Matt:LabNotes/2013-7-26
Analysis of HL152: Representation Bias of CA12k Oligos (Corrected)[edit]
- I realized one of the parameters I have been using for bowtie2 ("-k 1") causes it to report the first good alignment but doesn't search for the best alignment
- The other parameters that I changed from default (mismatch penalty, gap penalties) decreased the penalties from default allowing more alignments to be considered
- The penalty for a single base gap is now equal to the penalty for single base mismatch while default a gap is about twice as high
- This is okay for my purposes here because oligo synthesis introduces high rate of gaps but I still want to count those as aligned reads
- I also want to consider as many reads as possible because I've shortened the reference to be very "specific"/defined (Not sure if this makes sense, I'll try to reword/explain better)
Using the 50bp segments of the probe reference because it will create the fewest false alignments (aligning to full probe sequence resulted in ~10% more alignments):
- Low alignment rate expected because did not use Illumina index so the reads included any junk reads without an index
/home/kunzhang/softwares/bowtie2-latest/bowtie2 --phred64 --mp 1,0 --rdg 0,1 --rfg 0,1 -x /home/mzcai/InSitu_HL152_130524_Analysis/CAprobes_50bp -q /home/mzcai/InSitu_HL152_130524_Analysis/s_2_1_unassigned.txt > /home/mzcai/InSitu_HL152_130524_Analysis/Readsalign2probes_CA.txt &
mzcai@genome-miner:~/InSitu_HL152_130524_Analysis$ 29716630 reads; of these: 29716630 (100.00%) were unpaired; of these: 22676072 (76.31%) aligned 0 times 1719956 (5.79%) aligned exactly 1 time 5320602 (17.90%) aligned >1 times 23.69% overall alignment rate
Counted up the reads for each probe using Count_reads_from_bowtie2_SAM_quantifyOligo.pl: CountofCAOligosv4.txt
- Some variation is to be expected even for a perfectly uniform starting oligo pool because PCR is a stochastic process and each cycle will double some oligos and not others
- Majority of probes have between 100-1000 reads
- 747 probes with >1000 reads
- 8 probes with <100 reads
File:RankOrderofCA12kOligoCounts.JPG
File:BinsofCA12kOligoCounts.JPG
Using Default Bowtie2 Parameters[edit]
- My previous idea of loosening the mismatch to consider more reads to find optimal alignment is no good
- As shown below, using the default parameters results in a more Gaussian distribution (Poisson distribution for large numbers) that is expected
- Also results in less alignments, which means instead of finding optimal alignments before, the lower penalties just allowed many poor reads align to specific probes, which resulted in extremely high read counts for them
/home/kunzhang/softwares/bowtie2-latest/bowtie2 --phred64 -x /home/mzcai/InSitu_HL152_130524_Analysis/CAprobes_50bp -q /home/mzcai/InSitu_HL152_130524_Analysis/s_2_1_unassigned.txt > /home/mzcai/InSitu_HL152_130524_Analysis/Readsalign2probes_CA_default.txt & 29716630 reads; of these: 29716630 (100.00%) were unpaired; of these: 24520654 (82.51%) aligned 0 times 5156375 (17.35%) aligned exactly 1 time 39601 (0.13%) aligned >1 times 17.49% overall alignment rate
Counted up reads: CountofCAOligosv5.txt
- Average: 420.6
- Stdev: 162.8 (Gaussian)
- Coefficient of Variation: 0.387 (Gaussian, stdev/mean)
- Total number of aligned reads: 5,195,976
File:RankOrderofCA12kOligoCounts default.JPG
File:BinsofCA12kOligoCounts default.JPG
Analysis of HL155: Representation Bias of Agi26k Oligos (Corrected)[edit]
- WAS USING THE WRONG END OF PROBES AS REFERENCE WHICH WAS WHY ALIGNMENT RATE WAS < 3% (see hidden comments for that analysis)
- Create new fasta file with Probelist_to_fasta_revcomp.pl
- Try both full reference (Agi26k_corrected.fa) and split between 0gap and 20gap (Agi26k0gap_corrected.fa & Agi26k20gap_corrected.fa)
Alignment of Agi26k oligo reads to full reference[edit]
- ~33% alignment rate for 0gap expected because ~2/3 Indx10 of the reads are Dan's probes
- ~33% alignment rate for 20gap expected because ~2/3 Indx12 of the reads are Dan's probes
- Using the same reasoning as for the CA12k Oligo alignment, redid alignment without "-k 1"
Fasta reference file (50bp long): Agi26k_corrected.fa
0 gap aligned with custom parameters:
/home/kunzhang/softwares/bowtie2-latest/bowtie2 --phred64 --mp 1,0 --rdg 0,1 --rfg 0,1 -x /home/mzcai/InSitu_HL155_130628_Analysis/Agi26k_corrected -q /home/mzcai/InSitu_HL155_130628_Analysis/s_3_1_Indx10.txt > /home/mzcai/InSitu_HL155_130628_Analysis/Readsalign2all_revcomp_0gap.txt & 16368416 reads; of these: 16368416 (100.00%) were unpaired; of these: 10066426 (61.50%) aligned 0 times 1491167 (9.11%) aligned exactly 1 time 4810823 (29.39%) aligned >1 times 38.50% overall alignment rate
File:Agi26k 0gapAlignedtoAllCustomScorePenalties.JPG
- 4,848,010 aligned to 0gap probes
- 1,453,980 aligned to 20gap probes
20 gap aligned with custom parameters:
/home/kunzhang/softwares/bowtie2-latest/bowtie2 --phred64 --mp 1,0 --rdg 0,1 --rfg 0,1 -x /home/mzcai/InSitu_HL155_130628_Analysis/Agi26k_corrected -q /home/mzcai/InSitu_HL155_130628_Analysis/s_3_1_Indx12.txt > /home/mzcai/InSitu_HL155_130628_Analysis/Readsalign2all_revcomp_20gap.txt & 17308686 reads; of these: 17308686 (100.00%) were unpaired; of these: 12337494 (71.28%) aligned 0 times 1022448 (5.91%) aligned exactly 1 time 3948744 (22.81%) aligned >1 times 28.72% overall alignment rate
File:Agi26k 20gapAlignedtoAllCustomScorePenalties.JPG
- 1,033,711 aligned to 0gap probes
- 3,937,481 aligned to 20gap probes
0 gap aligned with default parameters:
/home/kunzhang/softwares/bowtie2-latest/bowtie2 --phred64 -x /home/mzcai/InSitu_HL155_130628_Analysis/Agi26k_corrected -q /home/mzcai/InSitu_HL155_130628_Analysis/s_3_1_Indx10.txt > /home/mzcai/InSitu_HL155_130628_Analysis/Readsalign2all_revcomp_0gap_default.txt & 16368416 reads; of these: 16368416 (100.00%) were unpaired; of these: 11414712 (69.74%) aligned 0 times 4923826 (30.08%) aligned exactly 1 time 29878 (0.18%) aligned >1 times 30.26% overall alignment rate
File:Agi26k 0gapAlignedtoAlldefault.JPG
- 4,951,251 aligned to 0gap
- 2,453 aligned to 20gap probes
20 gap aligned with default parameters:
/home/kunzhang/softwares/bowtie2-latest/bowtie2 --phred64 -x /home/mzcai/InSitu_HL155_130628_Analysis/Agi26k_corrected -q /home/mzcai/InSitu_HL155_130628_Analysis/s_3_1_Indx12.txt > /home/mzcai/InSitu_HL155_130628_Analysis/Readsalign2all_revcomp_20gap_default.txt & 17308686 reads; of these: 17308686 (100.00%) were unpaired; of these: 13257038 (76.59%) aligned 0 times 4027036 (23.27%) aligned exactly 1 time 24612 (0.14%) aligned >1 times 23.41% overall alignment rate
File:Agi26k 20gapAlignedtoAlldefault.JPG
- 2,627 aligned to 0gap
- 4,049,021 aligned to 20gap
Alignment of Agi26k oligo reads to split reference[edit]
- Since we know PCR step should have only selected one set of probes through primers, those that align to other set must be misalignments and we can prevent this by using only the reference for that set
- Default parameters also seems to be better at accurate alignments
Fasta reference files (50bp long): Agi26k0gap_corrected.fa, Agi26k20gap_corrected.fa
0 gap aligned with default parameters:
/home/kunzhang/softwares/bowtie2-latest/bowtie2 --phred64 -x /home/mzcai/InSitu_HL155_130628_Analysis/Agi26k0gap_corrected -q /home/mzcai/InSitu_HL155_130628_Analysis/s_3_1_Indx10.txt > /home/mzcai/InSitu_HL155_130628_Analysis/Readsalign2split_revcomp_0gap_default.txt & 16368416 reads; of these: 16368416 (100.00%) were unpaired; of these: 11414774 (69.74%) aligned 0 times 4933186 (30.14%) aligned exactly 1 time 20456 (0.12%) aligned >1 times 30.26% overall alignment rate
- Average number of reads/probe: 382.1075
- Stdev: 99.95655 (Gaussian)
- Coefficient of Variation: 0.2616 (Gaussian, stdev/mean)
- Total number of reads aligned: 4,953,642
File:RankOrderofAgi26k 0gapOligoCounts default.JPG
File:BinsofAgi26k 0gapOligoCounts default.JPG
20 gap aligned with default parameters:
/home/kunzhang/softwares/bowtie2-latest/bowtie2 --phred64 -x /home/mzcai/InSitu_HL155_130628_Analysis/Agi26k20gap_corrected -q /home/mzcai/InSitu_HL155_130628_Analysis/s_3_1_Indx12.txt > /home/mzcai/InSitu_HL155_130628_Analysis/Readsalign2split_revcomp_20gap_default.txt & 17308686 reads; of these: 17308686 (100.00%) were unpaired; of these: 13255959 (76.59%) aligned 0 times 4036881 (23.32%) aligned exactly 1 time 15846 (0.09%) aligned >1 times 23.41% overall alignment rate
- Average number of reads/probe: 307.514
- Stdev: 80.6908 (Gaussian)
- Coefficient of Variation: 0.2624 (Gaussian, stdev/mean)
- Total number of reads aligned: 4,052,727
File:RankOrderofAgi26k 20gapOligoCounts default.JPG
File:BinsofAgi26k 20gapOligoCounts default.JPG