Rui:Data Analysis

From ZhangLabWiki
Revision as of 22:31, 22 June 2011 by >RuiLiu (→‎Data)
Jump to navigation Jump to search

Examples

Jeff's samples:
Kun:LabNotes/Haplotyping/2010-9-17

[1]

Transcriptome:
Kun:LabNotes/SingleCellExpr/2011-6-13

[2]

Haplotyping:
Kun:LabNotes/Haplotyping/2011-6-14

[3]

Haplotyping 6.21.11

Data

ruiliu@genome-miner:~$ cd /home/kunzhang/haplotyping/Data/HL095/

ruiliu@genome-miner:/home/kunzhang/haplotyping/Data/HL095$ ls -l s_4_Indx1.*

-rw-r--r-- 1 kunzhang kunzhang 114237942 2011-06-13 22:07 s_4_Indx1.bowtie.pileup
-rw-r--r-- 1 kunzhang kunzhang  39977658 2011-06-13 21:28 s_4_Indx1.bowtie.sorted.bam
-rw-r--r-- 1 kunzhang kunzhang   4442210 2011-06-13 22:05 s_4_Indx1.bowtie.sorted.unique.bam
-rw-r--r-- 1 kunzhang kunzhang       278 2011-06-13 21:29 s_4_Indx1.bowtie.variants.txt
-rw-r--r-- 1 kunzhang kunzhang      7103 2011-06-14 15:58 s_4_Indx1.hapCompare.txt
-rw-r--r-- 1 kunzhang kunzhang      9399 2011-06-14 16:56 s_4_Indx1.idioGraph.txt
-rw-r--r-- 1 kunzhang kunzhang    675115 2011-06-13 21:29 s_4_Indx1.log
-rw-r--r-- 1 kunzhang kunzhang 273522562 2011-06-13 18:08 s_4_Indx1.txt

ruiliu@genome-miner:/home/kunzhang/haplotyping/Data/HL095$ less s_4_Indx1.bowtie.pileup

1       557399  C       A       25      25      25      1       ^:A     f
1       557400  A       A       25      0       25      1       .       f
1       557401  T       T       25      0       25      1       .       f
1       557402  C       C       25      0       25      1       .       f
1       557403  A       A       25      0       25      1       .       c
1       557404  C       C       25      0       25      1       .       g
1       557405  T       T       25      0       25      1       .       g
1       557406  A       A       25      0       25      1       .       g
1       557407  G       G       25      0       25      1       .       g
1       557408  A       A       25      0       25      1       .       e
1       557409  C       C       25      0       25      1       .       f

ruiliu@genome-miner:/home/kunzhang/haplotyping/Data/HL095$ more s_4_Indx11.log

/home/kunzhang/softwares/bowtie-latest/bowtie -k 1 -l 32 -n 2 -m 1 --best --quiet -p 4 /home/kunzhang/HsGenome/1KG.ref/HsB36m s_4_Indx
11.txt s_4_Indx11.bowtie.map.1 
/home/kunzhang/Exome/Data/scripts/trimTerminalMismatches.pl < s_4_Indx11.bowtie.map.1 > s_4_Indx11.bowtie.map.2
/home/kunzhang/softwares/samtools-0.1.7_x86_64-linux/bowtie2sam.pl  s_4_Indx11.bowtie.map.2 > s_4_Indx11.bowtie.sam
/home/kunzhang/softwares/samtools-0.1.7_x86_64-linux/samtools import /home/kunzhang/HsGenome/1KG.ref/human_b36_male.fa.gz.fai s_4_Indx
11.bowtie.sam s_4_Indx11.bowtie.bam
[sam_header_read2] 114 sequences loaded.
/home/kunzhang/softwares/samtools-0.1.7_x86_64-linux/samtools sort s_4_Indx11.bowtie.bam  s_4_Indx11.bowtie.sorted
java -Xmx2g -jar /home/kunzhang/softwares/picard-tools-1.38/MarkDuplicates.jar ASSUME_SORTED=TRUE REMOVE_DUPLICATES=TRUE INPUT=s_4_In
dx11.bowtie.sorted.bam OUTPUT=s_4_Indx11.bowtie.sorted.unique.bam METRICS_FILE=dup.metric
[Tue Jun 14 15:23:03 PDT 2011] net.sf.picard.sam.MarkDuplicates INPUT=s_4_Indx11.bowtie.sorted.bam OUTPUT=s_4_Indx11.bowtie.sorted.uni
que.bam METRICS_FILE=dup.metric REMOVE_DUPLICATES=true ASSUME_SORTED=true    MAX_SEQUENCES_FOR_DISK_READ_ENDS_MAP=50000 MAX_FILE_HANDL
ES_FOR_READ_ENDS_MAP=8000 READ_NAME_REGEX=[a-zA-Z0-9]+:[0-9]:([0-9]+):([0-9]+):([0-9]+).* OPTICAL_DUPLICATE_PIXEL_DISTANCE=100 TMP_DIR
=/tmp/kunzhang VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=fals
e CREATE_MD5_FILE=false
INFO	2011-06-14 15:23:03	MarkDuplicates	Start of doWork freeMemory: 374846832; totalMemory: 377225216; maxMemory: 1908932608
INFO	2011-06-14 15:23:03	MarkDuplicates	Reading input file and constructing read end information.
INFO	2011-06-14 15:23:03	MarkDuplicates	Will retain up to 7575129 data points before spilling to disk.
INFO	2011-06-14 15:23:03	MarkDuplicates	Assuming input is coordinate sorted.
INFO	2011-06-14 15:23:06	MarkDuplicates	Read 887559 records. 0 pairs never matched.
INFO	2011-06-14 15:23:07	MarkDuplicates	After buildSortedReadEndLists freeMemory: 733397640; totalMemory: 925368320; maxMemory
: 1908932608
INFO	2011-06-14 15:23:07	MarkDuplicates	Will retain up to 59654144 duplicate indices before spilling to disk.
INFO	2011-06-14 15:23:07	MarkDuplicates	Traversing read pair information and detecting duplicates.
INFO	2011-06-14 15:23:07	MarkDuplicates	Traversing fragment information and detecting duplicates.
INFO	2011-06-14 15:23:09	MarkDuplicates	Sorting list of duplicate records.
INFO	2011-06-14 15:23:09	MarkDuplicates	After generateDuplicateIndexes freeMemory: 923987040; totalMemory: 1410072576; maxMemo
ry: 1908932608
INFO	2011-06-14 15:23:09	MarkDuplicates	Marking 837865 records as duplicates.
INFO	2011-06-14 15:23:09	MarkDuplicates	Found 0 optical duplicate clusters.
INFO	2011-06-14 15:23:12	MarkDuplicates	Before output close freeMemory: 920285040; totalMemory: 1408237568; maxMemory: 1908932
608
INFO	2011-06-14 15:23:12	MarkDuplicates	After output close freeMemory: 876964080; totalMemory: 1364918272; maxMemory: 19089326
08
[Tue Jun 14 15:23:12 PDT 2011] net.sf.picard.sam.MarkDuplicates done.
Runtime.totalMemory()=1364918272
/home/kunzhang/softwares/samtools-0.1.7_x86_64-linux/samtools pileup -c -f /home/kunzhang/HsGenome/1KG.ref/human_b36_male.fa s_4_Indx1
1.bowtie.sorted.unique.bam > s_4_Indx11.bowtie.pileup
Use of uninitialized value in print at /home/kunzhang/Exome/Data/scripts/pileup2variants.pl line 145.
Use of uninitialized value in print at /home/kunzhang/Exome/Data/scripts/pileup2variants.pl line 145.

ruiliu@genome-miner:/home/kunzhang/haplotyping/Data/HL095$ /home/kunzhang/softwares/samtools-0.1.7_x86_64-linux/samtools view s_4_Indx11.bowtie.sorted.bam | wc -l

887559

ruiliu@genome-miner:/home/kunzhang/haplotyping/Data/HL095$ /home/kunzhang/softwares/samtools-0.1.7_x86_64-linux/samtools view s_4_Indx11.bowtie.sorted.unique.bam | wc -l

49694