Editing
Matt:LabNotes/2013-7-26
Jump to navigation
Jump to search
Warning:
You are not logged in. Your IP address will be publicly visible if you make any edits. If you
log in
or
create an account
, your edits will be attributed to your username, along with other benefits.
Anti-spam check. Do
not
fill this in!
==Analysis of HL152: Representation Bias of CA12k Oligos (Corrected)== *I realized one of the parameters I have been using for bowtie2 ("-k 1") causes it to report the first good alignment but doesn't search for the best alignment *The other parameters that I changed from default (mismatch penalty, gap penalties) decreased the penalties from default allowing more alignments to be considered **The penalty for a single base gap is now equal to the penalty for single base mismatch while default a gap is about twice as high **This is okay for my purposes here because oligo synthesis introduces high rate of gaps but I still want to count those as aligned reads **I also want to consider as many reads as possible because I've shortened the reference to be very "specific"/defined (Not sure if this makes sense, I'll try to reword/explain better) Using the 50bp segments of the probe reference because it will create the fewest false alignments (aligning to full probe sequence resulted in ~10% more alignments): *Low alignment rate expected because did not use Illumina index so the reads included any junk reads without an index /home/kunzhang/softwares/bowtie2-latest/bowtie2 --phred64 --mp 1,0 --rdg 0,1 --rfg 0,1 -x /home/mzcai/InSitu_HL152_130524_Analysis/CAprobes_50bp -q /home/mzcai/InSitu_HL152_130524_Analysis/s_2_1_unassigned.txt > /home/mzcai/InSitu_HL152_130524_Analysis/Readsalign2probes_CA.txt & mzcai@genome-miner:~/InSitu_HL152_130524_Analysis$ 29716630 reads; of these: 29716630 (100.00%) were unpaired; of these: 22676072 (76.31%) aligned 0 times 1719956 (5.79%) aligned exactly 1 time 5320602 (17.90%) aligned >1 times 23.69% overall alignment rate Counted up the reads for each probe using [[Media:Count_reads_from_bowtie2_SAM_quantifyOligo.txt|Count_reads_from_bowtie2_SAM_quantifyOligo.pl]]: CountofCAOligosv4.txt *Some variation is to be expected even for a perfectly uniform starting oligo pool because PCR is a stochastic process and each cycle will double some oligos and not others *Majority of probes have between 100-1000 reads **747 probes with >1000 reads **8 probes with <100 reads [[File:RankOrderofCA12kOligoCounts.JPG|border]] [[File:BinsofCA12kOligoCounts.JPG|border]] ===Using Default Bowtie2 Parameters=== *My previous idea of loosening the mismatch to consider more reads to find optimal alignment is no good **As shown below, using the default parameters results in a more Gaussian distribution (Poisson distribution for large numbers) that is expected **Also results in less alignments, which means instead of finding optimal alignments before, the lower penalties just allowed many poor reads align to specific probes, which resulted in extremely high read counts for them /home/kunzhang/softwares/bowtie2-latest/bowtie2 --phred64 -x /home/mzcai/InSitu_HL152_130524_Analysis/CAprobes_50bp -q /home/mzcai/InSitu_HL152_130524_Analysis/s_2_1_unassigned.txt > /home/mzcai/InSitu_HL152_130524_Analysis/Readsalign2probes_CA_default.txt & 29716630 reads; of these: 29716630 (100.00%) were unpaired; of these: 24520654 (82.51%) aligned 0 times 5156375 (17.35%) aligned exactly 1 time 39601 (0.13%) aligned >1 times 17.49% overall alignment rate Counted up reads: CountofCAOligosv5.txt *Average: 420.6 *Stdev: 162.8 (Gaussian) *Coefficient of Variation: 0.387 (Gaussian, stdev/mean) *Total number of aligned reads: 5,195,976 [[File:RankOrderofCA12kOligoCounts_default.JPG]]<br> [[File:BinsofCA12kOligoCounts_default.JPG]]<br> ==Analysis of HL155: Representation Bias of Agi26k Oligos (Corrected)== *WAS USING THE WRONG END OF PROBES AS REFERENCE WHICH WAS WHY ALIGNMENT RATE WAS < 3% (see hidden comments for that analysis) **Create new fasta file with [[Media:Probelist_to_fasta_revcomp.txt|Probelist_to_fasta_revcomp.pl]] *Try both full reference (Agi26k_corrected.fa) and split between 0gap and 20gap (Agi26k0gap_corrected.fa & Agi26k20gap_corrected.fa) ===Alignment of Agi26k oligo reads to full reference=== *~33% alignment rate for 0gap expected because ~2/3 Indx10 of the reads are Dan's probes *~33% alignment rate for 20gap expected because ~2/3 Indx12 of the reads are Dan's probes *Using the same reasoning as for the CA12k Oligo alignment, redid alignment without "-k 1" Fasta reference file (50bp long): Agi26k_corrected.fa 0 gap aligned with custom parameters: /home/kunzhang/softwares/bowtie2-latest/bowtie2 --phred64 --mp 1,0 --rdg 0,1 --rfg 0,1 -x /home/mzcai/InSitu_HL155_130628_Analysis/Agi26k_corrected -q /home/mzcai/InSitu_HL155_130628_Analysis/s_3_1_Indx10.txt > /home/mzcai/InSitu_HL155_130628_Analysis/Readsalign2all_revcomp_0gap.txt & 16368416 reads; of these: 16368416 (100.00%) were unpaired; of these: 10066426 (61.50%) aligned 0 times 1491167 (9.11%) aligned exactly 1 time 4810823 (29.39%) aligned >1 times 38.50% overall alignment rate [[File:Agi26k_0gapAlignedtoAllCustomScorePenalties.JPG|700px]] *4,848,010 aligned to 0gap probes *1,453,980 aligned to 20gap probes 20 gap aligned with custom parameters: /home/kunzhang/softwares/bowtie2-latest/bowtie2 --phred64 --mp 1,0 --rdg 0,1 --rfg 0,1 -x /home/mzcai/InSitu_HL155_130628_Analysis/Agi26k_corrected -q /home/mzcai/InSitu_HL155_130628_Analysis/s_3_1_Indx12.txt > /home/mzcai/InSitu_HL155_130628_Analysis/Readsalign2all_revcomp_20gap.txt & 17308686 reads; of these: 17308686 (100.00%) were unpaired; of these: 12337494 (71.28%) aligned 0 times 1022448 (5.91%) aligned exactly 1 time 3948744 (22.81%) aligned >1 times 28.72% overall alignment rate [[File:Agi26k_20gapAlignedtoAllCustomScorePenalties.JPG|700px]] *1,033,711 aligned to 0gap probes *3,937,481 aligned to 20gap probes 0 gap aligned with default parameters: /home/kunzhang/softwares/bowtie2-latest/bowtie2 --phred64 -x /home/mzcai/InSitu_HL155_130628_Analysis/Agi26k_corrected -q /home/mzcai/InSitu_HL155_130628_Analysis/s_3_1_Indx10.txt > /home/mzcai/InSitu_HL155_130628_Analysis/Readsalign2all_revcomp_0gap_default.txt & 16368416 reads; of these: 16368416 (100.00%) were unpaired; of these: 11414712 (69.74%) aligned 0 times 4923826 (30.08%) aligned exactly 1 time 29878 (0.18%) aligned >1 times 30.26% overall alignment rate [[File:Agi26k_0gapAlignedtoAlldefault.JPG|700px]] *4,951,251 aligned to 0gap *2,453 aligned to 20gap probes 20 gap aligned with default parameters: /home/kunzhang/softwares/bowtie2-latest/bowtie2 --phred64 -x /home/mzcai/InSitu_HL155_130628_Analysis/Agi26k_corrected -q /home/mzcai/InSitu_HL155_130628_Analysis/s_3_1_Indx12.txt > /home/mzcai/InSitu_HL155_130628_Analysis/Readsalign2all_revcomp_20gap_default.txt & 17308686 reads; of these: 17308686 (100.00%) were unpaired; of these: 13257038 (76.59%) aligned 0 times 4027036 (23.27%) aligned exactly 1 time 24612 (0.14%) aligned >1 times 23.41% overall alignment rate [[File:Agi26k_20gapAlignedtoAlldefault.JPG|700px]] *2,627 aligned to 0gap *4,049,021 aligned to 20gap ===Alignment of Agi26k oligo reads to split reference=== *Since we know PCR step should have only selected one set of probes through primers, those that align to other set must be misalignments and we can prevent this by using only the reference for that set *Default parameters also seems to be better at accurate alignments Fasta reference files (50bp long): Agi26k0gap_corrected.fa, Agi26k20gap_corrected.fa 0 gap aligned with default parameters: /home/kunzhang/softwares/bowtie2-latest/bowtie2 --phred64 -x /home/mzcai/InSitu_HL155_130628_Analysis/Agi26k0gap_corrected -q /home/mzcai/InSitu_HL155_130628_Analysis/s_3_1_Indx10.txt > /home/mzcai/InSitu_HL155_130628_Analysis/Readsalign2split_revcomp_0gap_default.txt & 16368416 reads; of these: 16368416 (100.00%) were unpaired; of these: 11414774 (69.74%) aligned 0 times 4933186 (30.14%) aligned exactly 1 time 20456 (0.12%) aligned >1 times 30.26% overall alignment rate *Average number of reads/probe: 382.1075 *Stdev: 99.95655 (Gaussian) *Coefficient of Variation: 0.2616 (Gaussian, stdev/mean) *Total number of reads aligned: 4,953,642 [[File:RankOrderofAgi26k_0gapOligoCounts_default.JPG|550px]]<br> [[File:BinsofAgi26k_0gapOligoCounts_default.JPG]]<br> 20 gap aligned with default parameters: /home/kunzhang/softwares/bowtie2-latest/bowtie2 --phred64 -x /home/mzcai/InSitu_HL155_130628_Analysis/Agi26k20gap_corrected -q /home/mzcai/InSitu_HL155_130628_Analysis/s_3_1_Indx12.txt > /home/mzcai/InSitu_HL155_130628_Analysis/Readsalign2split_revcomp_20gap_default.txt & 17308686 reads; of these: 17308686 (100.00%) were unpaired; of these: 13255959 (76.59%) aligned 0 times 4036881 (23.32%) aligned exactly 1 time 15846 (0.09%) aligned >1 times 23.41% overall alignment rate *Average number of reads/probe: 307.514 *Stdev: 80.6908 (Gaussian) *Coefficient of Variation: 0.2624 (Gaussian, stdev/mean) *Total number of reads aligned: 4,052,727 [[File:RankOrderofAgi26k_20gapOligoCounts_default.JPG|550px]]<br> [[File:BinsofAgi26k_20gapOligoCounts_default.JPG]] <!--*Using the same reasoning as for the CA12k Oligo alignment, redid alignment without "-k 1" *Also separated 0gap and 20gap probes into two reference fasta files to prevent incorrect alignments **As seen in the tables of [http://genome-tech.ucsd.edu/LabNotes/index.php/Matt:LabNotes/2013-7-19|2013-7-19], using a single fasta file resulted in many incorrect alignments **This calls into the question the accuracy of reads/alignment Fasta reference files (50bp long): Agi26k0gap.fa & Agi26k20gap.fa ===Alignment of Agi26k oligo reads=== *Low alignment rate for 0gap expected because ~2/3 Indx10 of the reads are Dan's probes *Low alignment rate for 20gap expected because ~2/3 Indx12 of the reads are Dan's probes ====0 gap probes==== /home/kunzhang/softwares/bowtie2-latest/bowtie2 --phred64 --mp 1,0 --rdg 0,1 --rfg 0,1 -x /home/mzcai/InSitu_HL155_130628_Analysis/Agi26k0gap -q /home/mzcai/InSitu_HL155_130628_Analysis/s_3_1_Indx10.txt > /home/mzcai/InSitu_HL155_130628_Analysis/Readsalign2probes_0gap.txt & mzcai@genome-miner:~/InSitu_HL155_130628_Analysis$ 16368416 reads; of these: 16368416 (100.00%) were unpaired; of these: 15990987 (97.69%) aligned 0 times 127671 (0.78%) aligned exactly 1 time 249758 (1.53%) aligned >1 times 2.31% overall alignment rate Counted up reads for each probe using [[Media:Count_reads_from_bowtie2_SAM_quantifyOligo.txt|Count_reads_from_bowtie2_SAM_quantifyOligo.pl]]: CountsofAgi26k_0gap.txt *At least three reads for each probe *19 probes with >200 reads *154 probes with >100 reads *116 probes <10 reads *4 probes with <5 reads [[File:RankOrderofAgi26k_0gapOligoCounts.JPG|border]] [[File:BinsofAgi26k_0gapOligoCounts.JPG|border]] ====20 gap probes==== /home/kunzhang/softwares/bowtie2-latest/bowtie2 --phred64 --mp 1,0 --rdg 0,1 --rfg 0,1 -x /home/mzcai/InSitu_HL155_130628_Analysis/Agi26k20gap -q /home/mzcai/InSitu_HL155_130628_Analysis/s_3_1_Indx12.txt > /home/mzcai/InSitu_HL155_130628_Analysis/Readsalign2probes_20gap.txt & mzcai@genome-miner:~/InSitu_HL155_130628_Analysis$ 17308686 reads; of these: 17308686 (100.00%) were unpaired; of these: 17114997 (98.88%) aligned 0 times 63125 (0.36%) aligned exactly 1 time 130564 (0.75%) aligned >1 times 1.12% overall alignment rate Counted up reads for each probe using [[Media:Count_reads_from_bowtie2_SAM_quantifyOligo.txt|Count_reads_from_bowtie2_SAM_quantifyOligo.pl]]: CountsofAgi26k_20gap.txt *At least one read for each probe *18 probes with >100 reads *142 probes with >50 reads *311 probes with <5 reads [[File:RankOrderofAgi26k_20gapOligoCounts.JPG|border]] [[File:BinsofAgi26k_20gapOligoCounts.JPG|border]] -->
Summary:
Please note that all contributions to ZhangLabWiki may be edited, altered, or removed by other contributors. If you do not want your writing to be edited mercilessly, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource (see
ZhangLabWiki:Copyrights
for details).
Do not submit copyrighted work without permission!
Cancel
Editing help
(opens in new window)
Navigation menu
Personal tools
Not logged in
Talk
Contributions
Create account
Log in
Namespaces
Page
Discussion
English
Views
Read
Edit
View history
More
Search
Navigation
Main Page
Current events
Recent changes
Random page
Investigators
Matt Cai
Song Chen
Eric Chu
Dinh Diep
Elizabeth Duong
Shicheng Guo
Alan Fung
Daniel Jacobsen
Blue Lake
Huy Lam
Alice Li
Andrew Richards
Brandon Sos
Chris Wei
Yan Wu
Kun Zhang
Tools
What links here
Related changes
Special pages
Page information