Brandon:LabNotes/Project1/2013-3-19
Jump to navigation
Jump to search
Analysis of unamapples when RNase III/rare cells didn't work, other analyses[edit]
- in rare cells the run did not work. not sure why but probably because RNase III is not working. now checking polyA read counts in the complete fastq file with all read data, and in the unmappable file
- this analysis last time when the run didn't work in 121120_HL136_NXTA_seq_polyA_works_rnaseiii_doesnt showed polyA'ed 5' read sequences dominated the sequencing library when fragmentation didn't work
- also checking single alignment stats, and comparing to magnesium fragmentation to see how unmappable reads differ between magnesium and Rnase III fragmentation
- data analysis file: Media:2013-03-13 polyA analysis etc.xlsx
script to gather data
- used this script to count the polyA's in a file. if looking at a sam file have to use samtools view. could easily look at other sequence if needed.
for f in *.txt do var1=$(echo $1 | sed -e 's/.txt//g') echo "" #var2=$var1"_unmappable.txt" var3=$var1"_unique.bam" echo "" echo "total lines original fasta" >> nohup.out nohup wc -l $f echo "polyA counts, 10, 12, 14, 16, 18, 20" >> nohup.out nohup grep -c AAAAAAAAAA $f nohup grep -c AAAAAAAAAAAA $f nohup grep -c AAAAAAAAAAAAAA $f nohup grep -c AAAAAAAAAAAAAAAA $f nohup grep -c AAAAAAAAAAAAAAAAAA $f nohup grep -c AAAAAAAAAAAAAAAAAAAA $f echo "" >> nohup.out echo "total lines with polyA and correct 3' adatptor" >> nohup.out nohup grep -c AAAAAAAAAAAGATCGG $f echo "" >> nohup.out #echo "total lines polyA mapped unique reads file" >> nohup.out #samtools view $var3 | nohup wc -l #echo "polyA counts, 10, 12, 14, 16, 18, 20" >> nohup.out #samtools view $var3 | nohup grep -c AAAAAAAAAA #samtools view $var3 | nohup grep -c AAAAAAAAAAAA #samtools view $var3 | nohup grep -c AAAAAAAAAAAAAA #samtools view $var3 | nohup grep -c AAAAAAAAAAAAAAAA #samtools view $var3 | nohup grep -c AAAAAAAAAAAAAAAAAA #samtools view $var3 | nohup grep -c AAAAAAAAAAAAAAAAAAAA #echo "" >> nohup.out #echo "total lines with polyA and correct 3' adatptor" >> nohup.out #samtools view $var3 | nohup wc -l #samtools view $var3 | nohup grep -c AAAAAAAAAAAGATCGG echo "" >> nohup.out echo "______________" >> nohup.out echo "" >> nohup.out done;
results[edit]
rare cell samples not working
- results showing rare cells did not work. sampled libraries from different lanes, and did not work across the board. very high fail map and clonal read rates. something is not working.
' | total reads | one alignment | one alignment % | failed to map | fail map % |
s_1_1_Indx61 | 7,661,617 | 301,347 | 3.93% | 7,082,291 | 92.44% |
s_2_1_Indx63 | 7,758,720 | 627,974 | 8.09% | 6,854,875 | 88.35% |
s_2_1_Indx70 | 711,315 | 26,402 | 3.71% | 645,394 | 90.73% |
s_4_1_Indx69 | 4,043,909 | 252,998 | 6.26% | 3,601,051 | 89.05% |
s_5_1_Indx62 | 7,135,656 | 340,278 | 4.77% | 6,453,463 | 90.44% |
s_5_1_Indx72 | 162,631 | 7,079 | 4.35% | 138,227 | 84.99% |
s_7_1_Indx66 | 920,071 | 31,372 | 3.41% | 854,049 | 92.82% |
supressed | supressed % | clonal reads | clonal read% | unique reads | |
s_1_1_Indx61 | 277,979 | 3.63% | 156,320 | 51.87% | 145,027 |
s_2_1_Indx63 | 275,871 | 3.56% | 100,930 | 16.07% | 527,044 |
s_2_1_Indx70 | 39,519 | 5.56% | 13,852 | 52.47% | 12,550 |
s_4_1_Indx69 | 189,860 | 4.69% | 101,237 | 40.01% | 151,761 |
s_5_1_Indx62 | 341,915 | 4.79% | 198,387 | 58.30% | 141,891 |
s_5_1_Indx72 | 17,325 | 10.65% | 3,363 | 47.51% | 3,716 |
s_7_1_Indx66 | 34,650 | 3.77% | 15,903 | 50.69% | 15,469 |
- based on previous failed runs, the first thing to stop working is RNA fragmentation. In this case the transposon, which is double stranded DNA with the read1 sequence and T7 RNAP binding site, is transcribed and creates small RNAs. These RNAs are presumably then polyA'ed thus making a viable substrate for modification and amplification in downstream steps. This molecules dominates since unfragmented RNA is to long for RT, PCR etc.
- Thus ran the script on the rare cell sample data.
- after running polyA analysis, can see most of the sequenced library is composed of a polyA'ed read sequence. Based on this believe it is because of a defective RNase III, so ordered a new one.
- saw similar results across all samples. can also see unmappables are mostly polyA'ed too
- full fasta files, 130306_HL146_rare_cells, 3/13/2013
full fasta files, 130306_HL146_rare_cells | ' | ' |
3/13/2013 | ||
all reads | ||
s_1_1_Indx61 | counts | %_polyA |
total reads | 7,661,617 | |
10 A's | 7,447,421 | 97.2043% |
12 A's | 7,381,926 | 96.3495% |
14 A's | 7,292,209 | 95.1785% |
16 A's | 7,209,313 | 94.0965% |
18 A's | 7,092,015 | 92.5655% |
20 A's | 6,648,831 | 86.7810% |
11 A's, correct 3' adaptor | 3,784,363 | 49.3938% |
unmappables, 130306_HL146_rare_cells | ||
s_1_1_Indx61 | counts | %_polyA |
total reads | 7,360,270 | |
10 A's | 7,278,091 | 98.8835% |
12 A's | 7,212,973 | 97.9988% |
14 A's | 7,123,762 | 96.7867% |
16 A's | 7,041,809 | 95.6732% |
18 A's | 6,925,760 | 94.0965% |
20 A's | 6,485,128 | 88.1099% |
11 A's, correct 3' adaptor | 3,763,390 | 51.1311% |
results when samples do work
- polyA results when RNase III is working and fresh.
- from /BS_Scratch2/130128_HL142, which worked, 1/31/2013
- can see that >50% of unmappables are polyAed, while in samples that works less than 30% are.
all reads | ' | ' |
1. RNase III, 1000 cells IVT | ||
s_1_1_Indx49 | counts | %_polyA |
total reads | 6,013,132 | |
10 A's | 1,678,049 | 27.9064% |
12 A's | 1,646,862 | 27.3878% |
14 A's | 1,613,429 | 26.8318% |
16 A's | 1,574,486 | 26.1841% |
18 A's | 1,525,597 | 25.3711% |
20 A's | 1,383,588 | 23.0094% |
11 A's, correct 3' adaptor | 258,041 | 4.2913% |
unmappables | ||
1. RNase III, 1000 cells IVT | ||
Indx49 | counts | %_polyA |
total reads | 2,659,224 | |
10 A's | 1,600,714 | 60.1948% |
12 A's | 1,571,424 | 59.0933% |
14 A's | 1,538,997 | 57.8739% |
16 A's | 1,501,119 | 56.4495% |
18 A's | 1,453,160 | 54.6460% |
20 A's | 1,313,460 | 49.3926% |
11 A's, correct 3' adaptor | 255,275 | 9.5996% |
compilation of data from runs that work and didn't work
- data analysis file Media:2013-03-13 polyA analysis etc.xlsx
- 8/6/2012
- Custom transposon with IVT amplification, round 2
- Rnase III was about 2 months and 1.5 weeks old
- can see polyA percentages of total reads are about 10%. still best that I have seen so far. however with unmappables it is >30% which shows that polyAing problems have always been an issue.
- with the nextra data, which does not use polyA polymerase, it is not a problem.
all reads | ' | ' | ' | all reads | ' | ' |
N2 adaptor 1. 1000 cells IVT | Nxtra atpr 7. 1000 cells nextera tagmentation | |||||
dc_Indx49 | counts | %_polyA | dc_Indx55 | counts | %_polyA | |
total reads | 11,921,212 | total reads | 8,585,322 | |||
10 A's | 1,300,960 | 10.9130% | 10 A's | 25,196 | 0.2935% | |
12 A's | 1,250,762 | 10.4919% | 12 A's | 16,071 | 0.1872% | |
14 A's | 1,208,168 | 10.1346% | 14 A's | 10,927 | 0.1273% | |
16 A's | 1,165,975 | 9.7807% | 16 A's | 7,354 | 0.0857% | |
18 A's | 1,114,434 | 9.3483% | 18 A's | 4,898 | 0.0571% | |
20 A's | 1,016,658 | 8.5281% | 20 A's | 2,760 | 0.0321% | |
11 A's, correct 3' adaptor | 128,931 | 1.0815% | 11 A's, correct 3' adaptor | 49 | 0.0006% | |
unmappables | unmappables | |||||
N2 adaptor 1. 1000 cells IVT | Nxtra atpr 7. 1000 cells nextera tagmentation | |||||
dc_Indx49 | counts | %_polyA | dc_Indx55 | counts | %_polyA | |
total reads | 3,643,919 | total reads | 3,213,601 | |||
10 A's | 1,227,718 | 33.6922% | 10 A's | 16,298 | 0.5072% | |
12 A's | 1,187,242 | 32.5815% | 12 A's | 11,465 | 0.3568% | |
14 A's | 1,149,818 | 31.5544% | 14 A's | 8,402 | 0.2615% | |
16 A's | 1,111,262 | 30.4963% | 16 A's | 5,979 | 0.1861% | |
18 A's | 1,062,416 | 29.1559% | 18 A's | 4,083 | 0.1271% | |
20 A's | 967,660 | 26.5555% | 20 A's | 2,327 | 0.0724% | |
11 A's, correct 3' adaptor | 125,947 | 3.4564% | 11 A's, correct 3' adaptor | 49 | 0.0015% | |
- 11/24/2012
- 121120_HL136_NXTA_seq_polyA_works_rnaseiii_doesnt
- here Rnase III didn't work, and nextra data which did not use it was unaffected. nearly all the reads are polyA sequences and even more so with the unmappables. At this point Rnase III was about 6 months old
all reads | ' | ' | ' | all reads | ' | ' |
IVT 1000 cells | NXTA 150K cells | |||||
s_5_1_Indx49 | counts | %_polyA | s_5_1_Indx55 | counts | %_polyA | |
total reads | 266,353 | total reads | 1,164,659 | |||
10 A's | 222,379 | 83.4903% | 10 A's | 2,214 | 0.1901% | |
12 A's | 221,208 | 83.0507% | 12 A's | 1,548 | 0.1329% | |
14 A's | 220,013 | 82.6020% | 14 A's | 1,143 | 0.0981% | |
16 A's | 218,655 | 82.0922% | 16 A's | 823 | 0.0707% | |
18 A's | 216,718 | 81.3650% | 18 A's | 599 | 0.0514% | |
20 A's | 208,640 | 78.3321% | 20 A's | 427 | 0.0367% | |
11 A's, correct 3' adaptor | 31,536 | 11.8399% | 11 A's, correct 3' adaptor | 27 | 0.0023% | |
unmappables | unmappables | |||||
IVT 1000 cells | NXTA 150K cells | |||||
s_5_1_Indx49 | counts | %_polyA | s_5_1_Indx55 | counts | %_polyA | |
total reads | 228,365 | total reads | 682,757 | |||
10 A's | 215,342 | 94.2973% | 10 A's | 1,295 | 0.1897% | |
12 A's | 214,235 | 93.8125% | 12 A's | 1,041 | 0.1525% | |
14 A's | 213,069 | 93.3020% | 14 A's | 836 | 0.1224% | |
16 A's | 211,735 | 92.7178% | 16 A's | 650 | 0.0952% | |
18 A's | 209,817 | 91.8779% | 18 A's | 503 | 0.0737% | |
20 A's | 201,752 | 88.3463% | 20 A's | 372 | 0.0545% | |
11 A's, correct 3' adaptor | 31,532 | 13.8077% | 11 A's, correct 3' adaptor | 27 | 0.0040% | |
- 12/21/2012
- 121218_HL139_mg_vs_rnaseiii_frag_light_sequencing
- New RNase III bought, about 2-3 weeks old.
- can see the samples are working almost as good as with the first run that was done in august.
all reads | ' | ' | ' | all reads | ' | ' |
1. Rnase III normal, clean then PAP with BP | 5. Rnase III inactivate at 65C 20 mins, PAP then clean with BP no temp change | |||||
c_Indx65.txt | counts | %_polyA | c_Indx69.txt | counts | %_polyA | |
total reads | 358,328 | 323,236 | ||||
10 A's | 107,107 | 29.8908% | 10 A's | 75,004 | 23.2041% | |
12 A's | 104,528 | 29.1710% | 12 A's | 72,610 | 22.4635% | |
14 A's | 102,097 | 28.4926% | 14 A's | 70,338 | 21.7606% | |
16 A's | 99,660 | 27.8125% | 16 A's | 68,156 | 21.0855% | |
18 A's | 97,113 | 27.1017% | 18 A's | 65,699 | 20.3254% | |
20 A's | 93,013 | 25.9575% | 20 A's | 60,947 | 18.8553% | |
11 A's, correct 3' adaptor | 4,815 | 1.3437% | 11 A's, correct 3' adaptor | 8,894 | 2.7515% | |
unmappables | unmappables | |||||
1. Rnase III normal, clean then PAP with BP | 5. Rnase III inactivate at 65C 20 mins, PAP then clean with BP no temp change | |||||
c_Indx65.unmappable.txt | counts | %_polyA | c_Indx69.unmappable.txt | counts | %_polyA | |
total reads | 186,481 | total reads | 136,754 | |||
10 A's | 103,001 | 55.2340% | 10 A's | 70,323 | 51.4230% | |
12 A's | 100,673 | 53.9857% | 12 A's | 68,134 | 49.8223% | |
14 A's | 98,392 | 52.7625% | 14 A's | 66,000 | 48.2618% | |
16 A's | 96,090 | 51.5280% | 16 A's | 63,925 | 46.7445% | |
18 A's | 93,689 | 50.2405% | 18 A's | 61,575 | 45.0261% | |
20 A's | 89,767 | 48.1373% | 20 A's | 56,928 | 41.6280% | |
11 A's, correct 3' adaptor | 4,669 | 2.5037% | 11 A's, correct 3' adaptor | 8,744 | 6.3940% | |
- 1/31/2013
- from /BS_Scratch2/130128_HL142, which worked (rnase III and magnesium comparison)
- Rnase III now about 2 months old
- rnase III is working fine and gave good results in this case.
all reads | ' | ' |
1. RNase III, 1000 cells IVT | ||
s_1_1_Indx49 | counts | %_polyA |
total reads | 6,013,132 | |
10 A's | 1,678,049 | 27.9064% |
12 A's | 1,646,862 | 27.3878% |
14 A's | 1,613,429 | 26.8318% |
16 A's | 1,574,486 | 26.1841% |
18 A's | 1,525,597 | 25.3711% |
20 A's | 1,383,588 | 23.0094% |
11 A's, correct 3' adaptor | 258,041 | 4.2913% |
unmappables | ||
1. RNase III, 1000 cells IVT | ||
Indx49 | counts | %_polyA |
total reads | 2,659,224 | |
10 A's | 1,600,714 | 60.1948% |
12 A's | 1,571,424 | 59.0933% |
14 A's | 1,538,997 | 57.8739% |
16 A's | 1,501,119 | 56.4495% |
18 A's | 1,453,160 | 54.6460% |
20 A's | 1,313,460 | 49.3926% |
11 A's, correct 3' adaptor | 255,275 | 9.5996% |
- 3/13/2013
- full fasta files, 130306_HL146_rare_cells
- Rnase III is about 3.5 months old
- Now RNase III did not work. thus this post. seems you have to use it before it is 3 months old, otherwise it is not viable anymore.
full fasta files, 130306_HL146_rare_cells | ' | ' |
3/13/2013 | ||
all reads | ||
s_1_1_Indx61 | counts | %_polyA |
total reads | 7,661,617 | |
10 A's | 7,447,421 | 97.2043% |
12 A's | 7,381,926 | 96.3495% |
14 A's | 7,292,209 | 95.1785% |
16 A's | 7,209,313 | 94.0965% |
18 A's | 7,092,015 | 92.5655% |
20 A's | 6,648,831 | 86.7810% |
11 A's, correct 3' adaptor | 3,784,363 | 49.3938% |
unmappables, 130306_HL146_rare_cells | ||
s_1_1_Indx61 | counts | %_polyA |
total reads | 7,360,270 | |
10 A's | 7,278,091 | 98.8835% |
12 A's | 7,212,973 | 97.9988% |
14 A's | 7,123,762 | 96.7867% |
16 A's | 7,041,809 | 95.6732% |
18 A's | 6,925,760 | 94.0965% |
20 A's | 6,485,128 | 88.1099% |
11 A's, correct 3' adaptor | 3,763,390 | 51.1311% |
comparison between Rnase III and magnesium fragmentation in terms of polyA
- looked at the amount of unusable polyA sequences that happen between Rnase iii and magnesium fragmentation. Find that magnesium fragmentation results in alot more polyA sequence that are uninformative and unmappable. Seems as if most fragments that are generated and processed are due to bad/incomplete fragmentation during magnesium fragmentation. Thus probably why more reads are obtained with magnesium fragmentation, however the amount of unique reads between magnesium fragmentation and Rnase III is the same.
- 1/31/2013
- from /BS_Scratch2/130128_HL142, which worked (magnesium versus Rnase III frag)
- same type of results seen from other magnesium fragmented samples
' | ' | total reads | 10 A's | 12 A's |
1. RNase III, 1000 cells IVT | s_1_1_Indx49 | 6,013,132 | 1,678,049 | 1,646,862 |
2. RNase III, 1000 cells IVT Zymo <200 bp RNA removal | s_1_1_Indx50 | 7,050,458 | 2,339,221 | 2,304,762 |
3. RNase III, 1000 cells IVT 37C, 30 min tagmentation | s_1_1_Indx51 | 7,678,456 | 4,300,822 | 4,264,539 |
4. RNase III, 1000 cells IVT 3X [NT] | s_1_1_Indx52 | 8,131,941 | 4,017,086 | 3,967,862 |
8. Mg++, 1000 cells IVT Zymo <200 bp RNA removal | s_1_1_Indx56 | 12,400,893 | 8,973,674 | 8,893,555 |
9. Mg++, 1000 cells IVT 37C, 30 min tagmentation | s_1_1_Indx57 | 15,287,393 | 12,737,742 | 12,651,517 |
10. Mg++, 1000 cells IVT 3X [NT] | s_1_1_Indx58 | 21,590,487 | 15,157,802 | 14,982,567 |
1. RNase III, 1000 cells IVT | s_1_1_Indx49 | 27.9064% | 27.3878% | 26.8318% |
2. RNase III, 1000 cells IVT Zymo <200 bp RNA removal | s_1_1_Indx50 | 33.1783% | 32.6895% | 32.1664% |
3. RNase III, 1000 cells IVT 37C, 30 min tagmentation | s_1_1_Indx51 | 56.0115% | 55.5390% | 54.9753% |
4. RNase III, 1000 cells IVT 3X [NT] | s_1_1_Indx52 | 49.3989% | 48.7935% | 48.0837% |
8. Mg++, 1000 cells IVT Zymo <200 bp RNA removal | s_1_1_Indx56 | 72.3631% | 71.7171% | 70.9811% |
9. Mg++, 1000 cells IVT 37C, 30 min tagmentation | s_1_1_Indx57 | 83.3219% | 82.7578% | 82.0619% |
10. Mg++, 1000 cells IVT 3X [NT] | s_1_1_Indx58 | 70.2059% | 69.3943% | 68.4536% |
dates compilation of age of Rnase III
- 5/30/2012 purchase new Rnase III
- 11/28/2012 purchase new Rnase III
- 3/15/2013 purchase new RNase III
sequence_run | date | Rnase III age | months | viability |
IVT round 2 | 8/6/2012 | 78.00 | 2.516129032 | works |
IVT_round_3 | 10/26/2012 | 159 | 5.129032258 | not work |
121120_HL136_NXTA_seq_rniiii_round_3 | 11/24/2012 | 188.00 | 6.064516129 | not work |
121218_HL139_mg_vs_rnaseiii_frag | 12/21/2012 | 23.00 | 0.741935484 | works |
rnase III and magnesium comparison | 1/31/2013 | 64.00 | 2.064516129 | works |
_HL146_rare_cells | 3/13/2013 | 105.00 | 3.387096774 | not work |
conclusions[edit]
- assuming RNase III is not working anymore, ordered new RNase III. also magnesium fragmentation buffer is running low and is expired so ordered more. will tested on fresh versus frozen cells, which have shown good results but could have higher single alignment rates based on previous runs. Thus can compare directly and see if Rnase III is losing viability over time.
- Rnase III is touchy. order a new one every 2 months to ensure it is fresh and works. seems to not last more than 3 months before not working anymore. do not use if more than 2 months old.
- magnesium fragmentation results in incomplete fragmentation or unviable fragments that are polyA'ed that are unmappable and thus have high fail map rates, thus even though it has more reads, there are the same number of unique reads with Rnase III or magnesium fragmentation.