Rui:Data Analysis: Difference between revisions

From ZhangLabWiki
Jump to navigation Jump to search
>RuiLiu
>RuiLiu
 
(71 intermediate revisions by the same user not shown)
Line 1: Line 1:
==Examples==
==Examples==
KUN's pipeline:
http://genome-tech.ucsd.edu/LabNotes/index.php/RNAseq_pipeline [http://genome-tech.ucsd.edu/LabNotes/index.php/RNAseq_pipeline]
  Jeff's samples:
  Jeff's samples:
  Kun:LabNotes/Haplotyping/2010-9-17
  Kun:LabNotes/Haplotyping/2010-9-17 [http://genome-tech.ucsd.edu/LabNotes/index.php/Kun:LabNotes/Haplotyping/2010-9-17]
[http://genome-tech.ucsd.edu/LabNotes/index.php/Kun:LabNotes/Haplotyping/2010-9-17]


  Transcriptome:
  Transcriptome:
  Kun:LabNotes/SingleCellExpr/2011-6-13
  Kun:LabNotes/SingleCellExpr/2011-6-13 [http://genome-tech.ucsd.edu/LabNotes/index.php/Kun:LabNotes/SingleCellExpr/2011-6-13]
[http://genome-tech.ucsd.edu/LabNotes/index.php/Kun:LabNotes/SingleCellExpr/2011-6-13]


  Haplotyping:
  Haplotyping:
  Kun:LabNotes/Haplotyping/2011-6-14
  Kun:LabNotes/Haplotyping/2011-6-14 [http://genome-tech.ucsd.edu/LabNotes/index.php/Kun:LabNotes/Haplotyping/2011-6-14]
[http://genome-tech.ucsd.edu/LabNotes/index.php/Kun:LabNotes/Haplotyping/2011-6-14]
 
==Haplotyping 6.21.11==
 
===Data===
ruiliu@genome-miner:~$ cd /home/kunzhang/haplotyping/Data/HL095/
 
ruiliu@genome-miner:/home/kunzhang/haplotyping/Data/HL095$ ls -l s_4_Indx1.*
-rw-r--r-- 1 kunzhang kunzhang 114237942 2011-06-13 22:07 s_4_Indx1.bowtie.pileup
-rw-r--r-- 1 kunzhang kunzhang  39977658 2011-06-13 21:28 s_4_Indx1.bowtie.sorted.bam
-rw-r--r-- 1 kunzhang kunzhang  4442210 2011-06-13 22:05 s_4_Indx1.bowtie.sorted.unique.bam
-rw-r--r-- 1 kunzhang kunzhang      278 2011-06-13 21:29 s_4_Indx1.bowtie.variants.txt
-rw-r--r-- 1 kunzhang kunzhang      7103 2011-06-14 15:58 s_4_Indx1.hapCompare.txt
-rw-r--r-- 1 kunzhang kunzhang      9399 2011-06-14 16:56 s_4_Indx1.idioGraph.txt
-rw-r--r-- 1 kunzhang kunzhang    675115 2011-06-13 21:29 s_4_Indx1.log
-rw-r--r-- 1 kunzhang kunzhang 273522562 2011-06-13 18:08 s_4_Indx1.txt
 
ruiliu@genome-miner:/home/kunzhang/haplotyping/Data/HL095$ less s_4_Indx1.bowtie.pileup
1      557399  C      A      25      25      25      1      ^:A    f
1      557400  A      A      25      0      25      1      .      f
1      557401  T      T      25      0      25      1      .      f
1      557402  C      C      25      0      25      1      .      f
1      557403  A      A      25      0      25      1      .      c
1      557404  C      C      25      0      25      1      .      g
1      557405  T      T      25      0      25      1      .      g
1      557406  A      A      25      0      25      1      .      g
1      557407  G      G      25      0      25      1      .      g
1      557408  A      A      25      0      25      1      .      e
1      557409  C      C      25      0      25      1      .      f
 
ruiliu@genome-miner:/home/kunzhang/haplotyping/Data/HL095$ more s_4_Indx11.log
/home/kunzhang/softwares/bowtie-latest/bowtie -k 1 -l 32 -n 2 -m 1 --best --quiet -p 4 /home/kunzhang/HsGenome/1KG.ref/HsB36m s_4_Indx
11.txt s_4_Indx11.bowtie.map.1
/home/kunzhang/Exome/Data/scripts/trimTerminalMismatches.pl < s_4_Indx11.bowtie.map.1 > s_4_Indx11.bowtie.map.2
/home/kunzhang/softwares/samtools-0.1.7_x86_64-linux/bowtie2sam.pl  s_4_Indx11.bowtie.map.2 > s_4_Indx11.bowtie.sam
/home/kunzhang/softwares/samtools-0.1.7_x86_64-linux/samtools import /home/kunzhang/HsGenome/1KG.ref/human_b36_male.fa.gz.fai s_4_Indx
11.bowtie.sam s_4_Indx11.bowtie.bam
[sam_header_read2] 114 sequences loaded.
/home/kunzhang/softwares/samtools-0.1.7_x86_64-linux/samtools sort s_4_Indx11.bowtie.bam  s_4_Indx11.bowtie.sorted
java -Xmx2g -jar /home/kunzhang/softwares/picard-tools-1.38/MarkDuplicates.jar ASSUME_SORTED=TRUE REMOVE_DUPLICATES=TRUE INPUT=s_4_In
dx11.bowtie.sorted.bam OUTPUT=s_4_Indx11.bowtie.sorted.unique.bam METRICS_FILE=dup.metric
[Tue Jun 14 15:23:03 PDT 2011] net.sf.picard.sam.MarkDuplicates INPUT=s_4_Indx11.bowtie.sorted.bam OUTPUT=s_4_Indx11.bowtie.sorted.uni
que.bam METRICS_FILE=dup.metric REMOVE_DUPLICATES=true ASSUME_SORTED=true    MAX_SEQUENCES_FOR_DISK_READ_ENDS_MAP=50000 MAX_FILE_HANDL
ES_FOR_READ_ENDS_MAP=8000 READ_NAME_REGEX=[a-zA-Z0-9]+:[0-9]:([0-9]+):([0-9]+):([0-9]+).* OPTICAL_DUPLICATE_PIXEL_DISTANCE=100 TMP_DIR
=/tmp/kunzhang VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=fals
e CREATE_MD5_FILE=false
INFO 2011-06-14 15:23:03 MarkDuplicates Start of doWork freeMemory: 374846832; totalMemory: 377225216; maxMemory: 1908932608
INFO 2011-06-14 15:23:03 MarkDuplicates Reading input file and constructing read end information.
INFO 2011-06-14 15:23:03 MarkDuplicates Will retain up to 7575129 data points before spilling to disk.
INFO 2011-06-14 15:23:03 MarkDuplicates Assuming input is coordinate sorted.
INFO 2011-06-14 15:23:06 MarkDuplicates Read '''887559''' records. 0 pairs never matched.
INFO 2011-06-14 15:23:07 MarkDuplicates After buildSortedReadEndLists freeMemory: 733397640; totalMemory: 925368320; maxMemory
: 1908932608
INFO 2011-06-14 15:23:07 MarkDuplicates Will retain up to 59654144 duplicate indices before spilling to disk.
INFO 2011-06-14 15:23:07 MarkDuplicates Traversing read pair information and detecting duplicates.
INFO 2011-06-14 15:23:07 MarkDuplicates Traversing fragment information and detecting duplicates.
INFO 2011-06-14 15:23:09 MarkDuplicates Sorting list of duplicate records.
INFO 2011-06-14 15:23:09 MarkDuplicates After generateDuplicateIndexes freeMemory: 923987040; totalMemory: 1410072576; maxMemo
ry: 1908932608
INFO 2011-06-14 15:23:09 MarkDuplicates Marking '''837865''' records as duplicates.
INFO 2011-06-14 15:23:09 MarkDuplicates Found 0 optical duplicate clusters.
INFO 2011-06-14 15:23:12 MarkDuplicates Before output close freeMemory: 920285040; totalMemory: 1408237568; maxMemory: 1908932
608
INFO 2011-06-14 15:23:12 MarkDuplicates After output close freeMemory: 876964080; totalMemory: 1364918272; maxMemory: 19089326
08
[Tue Jun 14 15:23:12 PDT 2011] net.sf.picard.sam.MarkDuplicates done.
Runtime.totalMemory()=1364918272
/home/kunzhang/softwares/samtools-0.1.7_x86_64-linux/samtools pileup -c -f /home/kunzhang/HsGenome/1KG.ref/human_b36_male.fa s_4_Indx1
1.bowtie.sorted.unique.bam > s_4_Indx11.bowtie.pileup
Use of uninitialized value in print at /home/kunzhang/Exome/Data/scripts/pileup2variants.pl line 145.
Use of uninitialized value in print at /home/kunzhang/Exome/Data/scripts/pileup2variants.pl line 145.


ruiliu@genome-miner:/home/kunzhang/haplotyping/Data/HL095$ /home/kunzhang/softwares/samtools-0.1.7_x86_64-linux/samtools view s_4_Indx11.bowtie.sorted.bam | wc -l
==Data analysis==
887559
*[[Rui:Haplotyping 6.21.11|Haplotyping 6.21.11]]
*[[Rui:RNAseq analysis from 7.18.11|RNAseq analysis from 7.18.11]]
*[[Rui:RNAseq analysis on HL098|RNAseq analysis on HL098]]
*[[Rui:RNAseq analysis on HL099|RNAseq analysis on HL099]]
*[[Rui:RNAseq analysis on HL101|RNAseq analysis on HL101]]
*[[Rui:RNAseq analysis on Hiseq111005|RNAseq_PGCsamples on Hiseq111005]]
*[[Rui:Hap analysis on HL108|Hap analysis on HL108]]
*[[Rui:RNAseq analysis on HL109|RNAseq analysis on HL109]]
*[[Rui:DNAseq analysis on HL114|DNAseq analysis on HL114]]
*[[Rui:DNAseq analysis on Hiseq120313|DNAseq analysis on Hiseq120313]]
*[[Rui:DNAseq analysis on HL118|DNAseq analysis on HL118]]
*[[Rui:totoRNAseq on 121226_HL140|totoRNAseq on 121226_HL140]]
*[[Rui:totoRNAseq on 130120_HL141|totoRNAseq on 130120_HL141]]
*[[Rui:totoRNAseq on 130220_HL144|totoRNAseq on 130220_HL144]]
*[[Rui:totoRNAseq on BL-UHRR-IVT-May28_2013|totoRNAseq on BL-UHRR-IVT-May28_2013]]
*[[Rui:totoRNAseq on 131009_HL161|totoRNAseq on 131009_HL161]]


ruiliu@genome-miner:/home/kunzhang/haplotyping/Data/HL095$ /home/kunzhang/softwares/samtools-0.1.7_x86_64-linux/samtools view s_4_Indx11.bowtie.sorted.unique.bam | wc -l
==RNAseq analysis key notes==
49694
* iGenome packages [http://tophat.cbcb.umd.edu/igenomes.html]
# UCSC genome package: [http://genome-tech.ucsd.edu/LabNotes/index.php/Rui:Tophat_mapping_10.27.11]
# Ensemble genome package: [http://genome-tech.ucsd.edu/LabNotes/index.php/Rui:Tophat_11.21.11#Re-download_Ensemble_package_from_iGenome_.5B1.5D]


===Reads===
* Comparison of RNAseq analysis programs: [http://genome-tech.ucsd.edu/LabNotes/index.php/Rui:RNAseq_analysis_on_Hiseq111005#Comparison_of_program]
        Sorted  Uniquely sorted  % 
Indx1  163556  15899           
Indx2  196625  16718
Indx3  127140  10123
Indx4  189464  13511
Indx5  9536  4856
Indx6  220670  12779
Indx7  185404  12479
Indx8  209445  14185
Indx9  175999  12100
Indx10  212683  13919
Indx11  188707  14591
Indx12  326082  16300

Latest revision as of 21:02, 31 October 2013

Examples[edit]

KUN's pipeline:
http://genome-tech.ucsd.edu/LabNotes/index.php/RNAseq_pipeline [1]
Jeff's samples:
Kun:LabNotes/Haplotyping/2010-9-17 [2]
Transcriptome:
Kun:LabNotes/SingleCellExpr/2011-6-13 [3]
Haplotyping:
Kun:LabNotes/Haplotyping/2011-6-14 [4]

Data analysis[edit]

RNAseq analysis key notes[edit]

  • iGenome packages [5]
  1. UCSC genome package: [6]
  2. Ensemble genome package: [7]
  • Comparison of RNAseq analysis programs: [8]