Brandon:LabNotes/Project1/2013-3-19

From ZhangLabWiki
Revision as of 02:48, 20 March 2013 by >Bsos (Created page with "==Analysis of unamapples when RNase III/rare cells didn't work, other analyses== *in rare cells the run did not work. not sure why but probably because RNase III is not work...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Analysis of unamapples when RNase III/rare cells didn't work, other analyses[edit]

  • in rare cells the run did not work. not sure why but probably because RNase III is not working. now checking polyA read counts in the complete fastq file with all read data, and in the unmappable file
    • this analysis last time when the run didn't work in 121120_HL136_NXTA_seq_polyA_works_rnaseiii_doesnt showed polyA'ed 5' read sequences dominated the sequencing library when fragmentation didn't work


  • also checking single alignment stats, and comparing to magnesium fragmentation to see how unmappable reads differ between magnesium and Rnase III fragmentation



script to gather data

  • used this script to count the polyA's in a file. if looking at a sam file have to use samtools view. could easily look at other sequence if needed.


for f in *.txt
	do 
	
	var1=$(echo $1 | sed -e 's/.txt//g')
	echo ""
	#var2=$var1"_unmappable.txt"
	var3=$var1"_unique.bam"
	echo ""
	
	echo "total lines original fasta" >> nohup.out
	nohup wc -l $f
	echo "polyA counts, 10, 12, 14, 16, 18, 20" >> nohup.out
	nohup grep -c AAAAAAAAAA $f
	nohup grep -c AAAAAAAAAAAA $f
	nohup grep -c AAAAAAAAAAAAAA $f
	nohup grep -c AAAAAAAAAAAAAAAA $f
	nohup grep -c AAAAAAAAAAAAAAAAAA $f
	nohup grep -c AAAAAAAAAAAAAAAAAAAA $f
	echo "" >> nohup.out
	
	
	echo "total lines with polyA and correct 3' adatptor" >> nohup.out
	nohup grep -c AAAAAAAAAAAGATCGG $f
	echo "" >> nohup.out
	
	#echo "total lines polyA mapped unique reads file" >> nohup.out
	#samtools view $var3 | nohup wc -l
	#echo "polyA counts, 10, 12, 14, 16, 18, 20" >> nohup.out
	#samtools view $var3 | nohup grep -c AAAAAAAAAA
	#samtools view $var3 | nohup grep -c AAAAAAAAAAAA
	#samtools view $var3 | nohup grep -c AAAAAAAAAAAAAA
	#samtools view $var3 | nohup grep -c AAAAAAAAAAAAAAAA
	#samtools view $var3 | nohup grep -c AAAAAAAAAAAAAAAAAA
	#samtools view $var3 | nohup grep -c AAAAAAAAAAAAAAAAAAAA
	#echo "" >> nohup.out
	
	#echo "total lines with polyA and correct 3' adatptor" >> nohup.out
	#samtools view $var3 | nohup wc -l
	#samtools view $var3 | nohup grep -c AAAAAAAAAAAGATCGG
	
	echo "" >> nohup.out
	echo "______________" >> nohup.out
	echo "" >> nohup.out
	
done;


results[edit]

rare cell samples not working


  • results showing rare cells did not work. sampled libraries from different lanes, and did not work across the board. very high fail map and clonal read rates. something is not working.
' total reads one alignment one alignment % failed to map fail map %
s_1_1_Indx61 7,661,617 301,347 3.93% 7,082,291 92.44%
s_2_1_Indx63 7,758,720 627,974 8.09% 6,854,875 88.35%
s_2_1_Indx70 711,315 26,402 3.71% 645,394 90.73%
s_4_1_Indx69 4,043,909 252,998 6.26% 3,601,051 89.05%
s_5_1_Indx62 7,135,656 340,278 4.77% 6,453,463 90.44%
s_5_1_Indx72 162,631 7,079 4.35% 138,227 84.99%
s_7_1_Indx66 920,071 31,372 3.41% 854,049 92.82%
supressed supressed % clonal reads clonal read% unique reads
s_1_1_Indx61 277,979 3.63% 156,320 51.87% 145,027
s_2_1_Indx63 275,871 3.56% 100,930 16.07% 527,044
s_2_1_Indx70 39,519 5.56% 13,852 52.47% 12,550
s_4_1_Indx69 189,860 4.69% 101,237 40.01% 151,761
s_5_1_Indx62 341,915 4.79% 198,387 58.30% 141,891
s_5_1_Indx72 17,325 10.65% 3,363 47.51% 3,716
s_7_1_Indx66 34,650 3.77% 15,903 50.69% 15,469



  • based on previous failed runs, the first thing to stop working is RNA fragmentation. In this case the transposon, which is double stranded DNA with the read1 sequence and T7 RNAP binding site, is transcribed and creates small RNAs. These RNAs are presumably then polyA'ed thus making a viable substrate for modification and amplification in downstream steps. This molecules dominates since unfragmented RNA is to long for RT, PCR etc.


  • Thus ran the script on the rare cell sample data.


  • after running polyA analysis, can see most of the sequenced library is composed of a polyA'ed read sequence. Based on this believe it is because of a defective RNase III, so ordered a new one.
  • saw similar results across all samples. can also see unmappables are mostly polyA'ed too
  • full fasta files, 130306_HL146_rare_cells, 3/13/2013
full fasta files, 130306_HL146_rare_cells ' '
3/13/2013
all reads
s_1_1_Indx61 counts %_polyA
total reads 7,661,617
10 A's 7,447,421 97.2043%
12 A's 7,381,926 96.3495%
14 A's 7,292,209 95.1785%
16 A's 7,209,313 94.0965%
18 A's 7,092,015 92.5655%
20 A's 6,648,831 86.7810%
11 A's, correct 3' adaptor 3,784,363 49.3938%
unmappables, 130306_HL146_rare_cells
s_1_1_Indx61 counts %_polyA
total reads 7,360,270
10 A's 7,278,091 98.8835%
12 A's 7,212,973 97.9988%
14 A's 7,123,762 96.7867%
16 A's 7,041,809 95.6732%
18 A's 6,925,760 94.0965%
20 A's 6,485,128 88.1099%
11 A's, correct 3' adaptor 3,763,390 51.1311%



results when samples do work

  • polyA results when RNase III is working and fresh.
  • from /BS_Scratch2/130128_HL142, which worked, 1/31/2013
  • can see that >50% of unmappables are polyAed, while in samples that works less than 30% are.
all reads ' '
1. RNase III, 1000 cells IVT
s_1_1_Indx49 counts %_polyA
total reads 6,013,132
10 A's 1,678,049 27.9064%
12 A's 1,646,862 27.3878%
14 A's 1,613,429 26.8318%
16 A's 1,574,486 26.1841%
18 A's 1,525,597 25.3711%
20 A's 1,383,588 23.0094%
11 A's, correct 3' adaptor 258,041 4.2913%
unmappables
1. RNase III, 1000 cells IVT
Indx49 counts %_polyA
total reads 2,659,224
10 A's 1,600,714 60.1948%
12 A's 1,571,424 59.0933%
14 A's 1,538,997 57.8739%
16 A's 1,501,119 56.4495%
18 A's 1,453,160 54.6460%
20 A's 1,313,460 49.3926%
11 A's, correct 3' adaptor 255,275 9.5996%




compilation of data from runs that work and didn't work


  • 8/6/2012
  • Custom transposon with IVT amplification, round 2
  • Rnase III was about 2 months and 1.5 weeks old
  • can see polyA percentages of total reads are about 10%. still best that I have seen so far. however with unmappables it is >30% which shows that polyAing problems have always been an issue.
  • with the nextra data, which does not use polyA polymerase, it is not a problem.
all reads ' ' ' all reads ' '
N2 adaptor 1. 1000 cells IVT Nxtra atpr 7. 1000 cells nextera tagmentation
dc_Indx49 counts %_polyA dc_Indx55 counts %_polyA
total reads 11,921,212 total reads 8,585,322
10 A's 1,300,960 10.9130% 10 A's 25,196 0.2935%
12 A's 1,250,762 10.4919% 12 A's 16,071 0.1872%
14 A's 1,208,168 10.1346% 14 A's 10,927 0.1273%
16 A's 1,165,975 9.7807% 16 A's 7,354 0.0857%
18 A's 1,114,434 9.3483% 18 A's 4,898 0.0571%
20 A's 1,016,658 8.5281% 20 A's 2,760 0.0321%
11 A's, correct 3' adaptor 128,931 1.0815% 11 A's, correct 3' adaptor 49 0.0006%
unmappables unmappables
N2 adaptor 1. 1000 cells IVT Nxtra atpr 7. 1000 cells nextera tagmentation
dc_Indx49 counts %_polyA dc_Indx55 counts %_polyA
total reads 3,643,919 total reads 3,213,601
10 A's 1,227,718 33.6922% 10 A's 16,298 0.5072%
12 A's 1,187,242 32.5815% 12 A's 11,465 0.3568%
14 A's 1,149,818 31.5544% 14 A's 8,402 0.2615%
16 A's 1,111,262 30.4963% 16 A's 5,979 0.1861%
18 A's 1,062,416 29.1559% 18 A's 4,083 0.1271%
20 A's 967,660 26.5555% 20 A's 2,327 0.0724%
11 A's, correct 3' adaptor 125,947 3.4564% 11 A's, correct 3' adaptor 49 0.0015%


  • 11/24/2012
  • 121120_HL136_NXTA_seq_polyA_works_rnaseiii_doesnt
  • here Rnase III didn't work, and nextra data which did not use it was unaffected. nearly all the reads are polyA sequences and even more so with the unmappables. At this point Rnase III was about 6 months old
all reads ' ' ' all reads ' '
IVT 1000 cells NXTA 150K cells
s_5_1_Indx49 counts %_polyA s_5_1_Indx55 counts %_polyA
total reads 266,353 total reads 1,164,659
10 A's 222,379 83.4903% 10 A's 2,214 0.1901%
12 A's 221,208 83.0507% 12 A's 1,548 0.1329%
14 A's 220,013 82.6020% 14 A's 1,143 0.0981%
16 A's 218,655 82.0922% 16 A's 823 0.0707%
18 A's 216,718 81.3650% 18 A's 599 0.0514%
20 A's 208,640 78.3321% 20 A's 427 0.0367%
11 A's, correct 3' adaptor 31,536 11.8399% 11 A's, correct 3' adaptor 27 0.0023%
unmappables unmappables
IVT 1000 cells NXTA 150K cells
s_5_1_Indx49 counts %_polyA s_5_1_Indx55 counts %_polyA
total reads 228,365 total reads 682,757
10 A's 215,342 94.2973% 10 A's 1,295 0.1897%
12 A's 214,235 93.8125% 12 A's 1,041 0.1525%
14 A's 213,069 93.3020% 14 A's 836 0.1224%
16 A's 211,735 92.7178% 16 A's 650 0.0952%
18 A's 209,817 91.8779% 18 A's 503 0.0737%
20 A's 201,752 88.3463% 20 A's 372 0.0545%
11 A's, correct 3' adaptor 31,532 13.8077% 11 A's, correct 3' adaptor 27 0.0040%



  • 12/21/2012
  • 121218_HL139_mg_vs_rnaseiii_frag_light_sequencing
  • New RNase III bought, about 2-3 weeks old.
  • can see the samples are working almost as good as with the first run that was done in august.
all reads ' ' ' all reads ' '
1. Rnase III normal, clean then PAP with BP 5. Rnase III inactivate at 65C 20 mins, PAP then clean with BP no temp change
c_Indx65.txt counts %_polyA c_Indx69.txt counts %_polyA
total reads 358,328 323,236
10 A's 107,107 29.8908% 10 A's 75,004 23.2041%
12 A's 104,528 29.1710% 12 A's 72,610 22.4635%
14 A's 102,097 28.4926% 14 A's 70,338 21.7606%
16 A's 99,660 27.8125% 16 A's 68,156 21.0855%
18 A's 97,113 27.1017% 18 A's 65,699 20.3254%
20 A's 93,013 25.9575% 20 A's 60,947 18.8553%
11 A's, correct 3' adaptor 4,815 1.3437% 11 A's, correct 3' adaptor 8,894 2.7515%
unmappables unmappables
1. Rnase III normal, clean then PAP with BP 5. Rnase III inactivate at 65C 20 mins, PAP then clean with BP no temp change
c_Indx65.unmappable.txt counts %_polyA c_Indx69.unmappable.txt counts %_polyA
total reads 186,481 total reads 136,754
10 A's 103,001 55.2340% 10 A's 70,323 51.4230%
12 A's 100,673 53.9857% 12 A's 68,134 49.8223%
14 A's 98,392 52.7625% 14 A's 66,000 48.2618%
16 A's 96,090 51.5280% 16 A's 63,925 46.7445%
18 A's 93,689 50.2405% 18 A's 61,575 45.0261%
20 A's 89,767 48.1373% 20 A's 56,928 41.6280%
11 A's, correct 3' adaptor 4,669 2.5037% 11 A's, correct 3' adaptor 8,744 6.3940%



  • 1/31/2013
  • from /BS_Scratch2/130128_HL142, which worked (rnase III and magnesium comparison)
  • Rnase III now about 2 months old
  • rnase III is working fine and gave good results in this case.


all reads ' '
1. RNase III, 1000 cells IVT
s_1_1_Indx49 counts %_polyA
total reads 6,013,132
10 A's 1,678,049 27.9064%
12 A's 1,646,862 27.3878%
14 A's 1,613,429 26.8318%
16 A's 1,574,486 26.1841%
18 A's 1,525,597 25.3711%
20 A's 1,383,588 23.0094%
11 A's, correct 3' adaptor 258,041 4.2913%
unmappables
1. RNase III, 1000 cells IVT
Indx49 counts %_polyA
total reads 2,659,224
10 A's 1,600,714 60.1948%
12 A's 1,571,424 59.0933%
14 A's 1,538,997 57.8739%
16 A's 1,501,119 56.4495%
18 A's 1,453,160 54.6460%
20 A's 1,313,460 49.3926%
11 A's, correct 3' adaptor 255,275 9.5996%


  • 3/13/2013
  • full fasta files, 130306_HL146_rare_cells
  • Rnase III is about 3.5 months old
  • Now RNase III did not work. thus this post. seems you have to use it before it is 3 months old, otherwise it is not viable anymore.
full fasta files, 130306_HL146_rare_cells ' '
3/13/2013
all reads
s_1_1_Indx61 counts %_polyA
total reads 7,661,617
10 A's 7,447,421 97.2043%
12 A's 7,381,926 96.3495%
14 A's 7,292,209 95.1785%
16 A's 7,209,313 94.0965%
18 A's 7,092,015 92.5655%
20 A's 6,648,831 86.7810%
11 A's, correct 3' adaptor 3,784,363 49.3938%
unmappables, 130306_HL146_rare_cells
s_1_1_Indx61 counts %_polyA
total reads 7,360,270
10 A's 7,278,091 98.8835%
12 A's 7,212,973 97.9988%
14 A's 7,123,762 96.7867%
16 A's 7,041,809 95.6732%
18 A's 6,925,760 94.0965%
20 A's 6,485,128 88.1099%
11 A's, correct 3' adaptor 3,763,390 51.1311%



comparison between Rnase III and magnesium fragmentation in terms of polyA


  • looked at the amount of unusable polyA sequences that happen between Rnase iii and magnesium fragmentation. Find that magnesium fragmentation results in alot more polyA sequence that are uninformative and unmappable. Seems as if most fragments that are generated and processed are due to bad/incomplete fragmentation during magnesium fragmentation. Thus probably why more reads are obtained with magnesium fragmentation, however the amount of unique reads between magnesium fragmentation and Rnase III is the same.


  • 1/31/2013
  • from /BS_Scratch2/130128_HL142, which worked (magnesium versus Rnase III frag)
  • same type of results seen from other magnesium fragmented samples
' ' total reads 10 A's 12 A's
1. RNase III, 1000 cells IVT s_1_1_Indx49 6,013,132 1,678,049 1,646,862
2. RNase III, 1000 cells IVT Zymo <200 bp RNA removal s_1_1_Indx50 7,050,458 2,339,221 2,304,762
3. RNase III, 1000 cells IVT 37C, 30 min tagmentation s_1_1_Indx51 7,678,456 4,300,822 4,264,539
4. RNase III, 1000 cells IVT 3X [NT] s_1_1_Indx52 8,131,941 4,017,086 3,967,862
8. Mg++, 1000 cells IVT Zymo <200 bp RNA removal s_1_1_Indx56 12,400,893 8,973,674 8,893,555
9. Mg++, 1000 cells IVT 37C, 30 min tagmentation s_1_1_Indx57 15,287,393 12,737,742 12,651,517
10. Mg++, 1000 cells IVT 3X [NT] s_1_1_Indx58 21,590,487 15,157,802 14,982,567
1. RNase III, 1000 cells IVT s_1_1_Indx49 27.9064% 27.3878% 26.8318%
2. RNase III, 1000 cells IVT Zymo <200 bp RNA removal s_1_1_Indx50 33.1783% 32.6895% 32.1664%
3. RNase III, 1000 cells IVT 37C, 30 min tagmentation s_1_1_Indx51 56.0115% 55.5390% 54.9753%
4. RNase III, 1000 cells IVT 3X [NT] s_1_1_Indx52 49.3989% 48.7935% 48.0837%
8. Mg++, 1000 cells IVT Zymo <200 bp RNA removal s_1_1_Indx56 72.3631% 71.7171% 70.9811%
9. Mg++, 1000 cells IVT 37C, 30 min tagmentation s_1_1_Indx57 83.3219% 82.7578% 82.0619%
10. Mg++, 1000 cells IVT 3X [NT] s_1_1_Indx58 70.2059% 69.3943% 68.4536%




dates compilation of age of Rnase III


  • 5/30/2012 purchase new Rnase III
  • 11/28/2012 purchase new Rnase III
  • 3/15/2013 purchase new RNase III
sequence_run date Rnase III age months viability
IVT round 2 8/6/2012 78.00 2.516129032 works
IVT_round_3 10/26/2012 159 5.129032258 not work
121120_HL136_NXTA_seq_rniiii_round_3 11/24/2012 188.00 6.064516129 not work
121218_HL139_mg_vs_rnaseiii_frag 12/21/2012 23.00 0.741935484 works
rnase III and magnesium comparison 1/31/2013 64.00 2.064516129 works
_HL146_rare_cells 3/13/2013 105.00 3.387096774 not work



conclusions[edit]

  • assuming RNase III is not working anymore, ordered new RNase III. also magnesium fragmentation buffer is running low and is expired so ordered more. will tested on fresh versus frozen cells, which have shown good results but could have higher single alignment rates based on previous runs. Thus can compare directly and see if Rnase III is losing viability over time.


  • Rnase III is touchy. order a new one every 2 months to ensure it is fresh and works. seems to not last more than 3 months before not working anymore. do not use if more than 2 months old.


  • magnesium fragmentation results in incomplete fragmentation or unviable fragments that are polyA'ed that are unmappable and thus have high fail map rates, thus even though it has more reads, there are the same number of unique reads with Rnase III or magnesium fragmentation.