Dinh 2011/NOTES/2011-11-1

From ZhangLabWiki
Jump to navigation Jump to search

330K re-normalization plan[edit]

  • How do the probe efficiency change across multiple experiments and cell types?
  • Calculate probes efficiency using library-free data, then for each probe, the average efficiency with standard deviation.
  • Since the capture experiments used normalized probes, I will only compare subset to subset.
    • Across multiple captures in one batch, how does the performance of probes compare?
    • Across multiple batches, how do the probes perform?
    • Which probes has consistent performance? Which probe has the most variation?
    • Out of the uniformly performing set, which are highest performing/which are lowest performing?
  • Group probes based on performance and variability

Counting probes efficiency for BSPP Library-Free libraries[edit]

RULES:

  • Reads mapping to Watson (bis.fwd) should always go to a target on Watson - read2 is upstream and read1 is downstream (mapped position should be offset by readlen +73)
  • Reads mapping to Crick (bis.rev) strand should always go to a target on Crick - read1 is upstream (mapped position is offset by read length -73) and read2 is downstream
  • Upstream = close to start of target, downstream = close to end of target
  • Easiest to use SOAP map file: (Hg18) File:MapReads2Probes.txt
mapReads2Probes W DMR330K_Annotated.txt < soap.fwd.mapped > fwd.counts.txt
mapReads2Probes C DMR330K_Annotated.txt < soap.rev.mapped > rev.counts.txt
  • SOAP map files for reads contain all alignments in '+' orientation - although PE reads are supposed to have one mapped to the '+' and the other mapped to '-'.

Map reads 2 probes[edit]

DMR330K PE100[edit]

  • I used 3 bp left offset and 3 bp right offset
> colon.N37.fastq.gz.fwd.soap.out.Watson.counts <==
Reads = 2414826 	 Mapped = 2139754 (0.886)
Reads mapping to positions which can't be uniquely counted to a probe = 124294	(0.051)
> colon.N37.fastq.gz.rev.soap.out.Crick.counts <==
Reads = 2066759 	 Mapped = 1861591 (0.901)
Reads mapping to positions which can't be uniquely counted to a probe = 80834	(0.039)
> CRBL.N37.fastq.gz.fwd.soap.out.Watson.counts <==
Reads = 2757404 	 Mapped = 2414054 (0.875)
Reads mapping to positions which can't be uniquely counted to a probe = 147505	(0.053)
> CRBL.N37.fastq.gz.rev.soap.out.Crick.counts <==
Reads = 2354820 	 Mapped = 2100747 (0.892)
Reads mapping to positions which can't be uniquely counted to a probe = 95071	(0.040)
> FL.N37.fastq.gz.fwd.soap.out.Watson.counts <==
Reads = 3582505 	 Mapped = 3146948 (0.878)
Reads mapping to positions which can't be uniquely counted to a probe = 152158	(0.042)
> FL.N37.fastq.gz.rev.soap.out.Crick.counts <==
Reads = 3064671 	 Mapped = 2741269 (0.894)
Reads mapping to positions which can't be uniquely counted to a probe = 91888	(0.030)
> heart.N37.fastq.gz.fwd.soap.out.Watson.counts <==
Reads = 2046522 	 Mapped = 1809707 (0.884)
Reads mapping to positions which can't be uniquely counted to a probe = 106214	(0.052)
> heart.N37.fastq.gz.rev.soap.out.Crick.counts <==
Reads = 1756595 	 Mapped = 1579355 (0.899)
Reads mapping to positions which can't be uniquely counted to a probe = 70161	(0.040)
> liver.N37.fastq.gz.fwd.soap.out.Watson.counts <==
Reads = 1491870 	 Mapped = 1292312 (0.866)
Reads mapping to positions which can't be uniquely counted to a probe = 85713	(0.057)
> liver.N37.fastq.gz.rev.soap.out.Crick.counts <==
Reads = 1272159 	 Mapped = 1117366 (0.878)
Reads mapping to positions which can't be uniquely counted to a probe = 59123	(0.046)
> lung.N37.fastq.gz.fwd.soap.out.Watson.counts <==
Reads = 3428623 	 Mapped = 3032793 (0.885)
Reads mapping to positions which can't be uniquely counted to a probe = 179679	(0.052)
> lung.N37.fastq.gz.rev.soap.out.Crick.counts <==
Reads = 2933482 	 Mapped = 2639062 (0.900)
Reads mapping to positions which can't be uniquely counted to a probe = 119363	(0.041)
> pancrease.N37.fastq.gz.fwd.soap.out.Watson.counts <==
Reads = 2595579 	 Mapped = 2269893 (0.875)
Reads mapping to positions which can't be uniquely counted to a probe = 135102	(0.052)
> pancrease.N37.fastq.gz.rev.soap.out.Crick.counts <==
Reads = 2228422 	 Mapped = 1977402 (0.887)
Reads mapping to positions which can't be uniquely counted to a probe = 90322	(0.041)
> SI.N37.fastq.gz.fwd.soap.out.Watson.counts <==
Reads = 3761894 	 Mapped = 3328791 (0.885)
Reads mapping to positions which can't be uniquely counted to a probe = 184598	(0.049)
> SI.N37.fastq.gz.rev.soap.out.Crick.counts <==
Reads = 3203385 	 Mapped = 2887979 (0.902)
Reads mapping to positions which can't be uniquely counted to a probe = 116754	(0.036)
> SM.N37.fastq.gz.fwd.soap.out.Watson.counts <==
Reads = 2416965 	 Mapped = 2153480 (0.891)
Reads mapping to positions which can't be uniquely counted to a probe = 127965	(0.053)
> SM.N37.fastq.gz.rev.soap.out.Crick.counts <==
Reads = 2077721 	 Mapped = 1879274 (0.904)
Reads mapping to positions which can't be uniquely counted to a probe = 87476	(0.042)
> stomach.N37.fastq.gz.fwd.soap.out.Watson.counts <==
Reads = 2815220 	 Mapped = 2477632 (0.880)
Reads mapping to positions which can't be uniquely counted to a probe = 145440	(0.052)
> stomach.N37.fastq.gz.rev.soap.out.Crick.counts <==
Reads = 2434011 	 Mapped = 2171407 (0.892)
Reads mapping to positions which can't be uniquely counted to a probe = 101191	(0.042)

> (UCLA)GK0290-002.gz.fwd.soap.out.Watson.counts <==
Reads = 8549352 	 Mapped = 7663379 (0.896)
Reads mapping to positions which can't be uniquely counted to a probe = 336593	(0.039)
> (UCLA)GK0290-002.gz.rev.soap.out.Crick.counts <==
Reads = 7368809 	 Mapped = 6692566 (0.908)
Reads mapping to positions which can't be uniquely counted to a probe = 204562	(0.028)
> (UCLA)GK0290-003.gz.fwd.soap.out.Watson.counts <==
Reads = 8691657 	 Mapped = 7653309 (0.881)
Reads mapping to positions which can't be uniquely counted to a probe = 348243	(0.040)
> (UCLA)GK0290-003.gz.rev.soap.out.Crick.counts <==
Reads = 7486891 	 Mapped = 6672288 (0.891)
Reads mapping to positions which can't be uniquely counted to a probe = 211972	(0.028)
> (UCLA)GK0290-004a.gz.fwd.soap.out.Watson.counts <==
Reads = 7805592 	 Mapped = 6990760 (0.896)
Reads mapping to positions which can't be uniquely counted to a probe = 316067	(0.040)
> (UCLA)GK0290-004a.gz.rev.soap.out.Crick.counts <==
Reads = 6737701 	 Mapped = 6114500 (0.908)
Reads mapping to positions which can't be uniquely counted to a probe = 186656	(0.028)
> UPENN_PE100_Indx11.fwd.soap.out.Watson.counts <==
Reads = 13104000         Mapped = 10625111 (0.811)
Reads mapping to positions which can't be uniquely counted to a probe = 629443  (0.048)
> UPENN_PE100_Indx11.rev.soap.out.Crick.counts <==
Reads = 12274998         Mapped = 10108328 (0.823)
Reads mapping to positions which can't be uniquely counted to a probe = 471469  (0.038)
> UPENN_PE100_Indx21.fwd.soap.out.Watson.counts <==
Reads = 10771331         Mapped = 8729892 (0.810)
Reads mapping to positions which can't be uniquely counted to a probe = 517327  (0.048)
> UPENN_PE100_Indx21.rev.soap.out.Crick.counts <==
Reads = 10076880         Mapped = 8298116 (0.823)
Reads mapping to positions which can't be uniquely counted to a probe = 385924  (0.038)
> UPENN_PE100_Indx36.fwd.soap.out.Watson.counts <==
Reads = 10803406         Mapped = 8840698 (0.818)
Reads mapping to positions which can't be uniquely counted to a probe = 513752  (0.048)
> UPENN_PE100_Indx36.rev.soap.out.Crick.counts <==
Reads = 10078671         Mapped = 8361505 (0.830)
Reads mapping to positions which can't be uniquely counted to a probe = 383216  (0.038)
*UPENN sample D03_CAMF022A has low on-target reads rate:
> UPENN_PE100_Indx20_1.fwd.soap.out.Watson.counts <==
Reads = 5632920 	 Mapped = 4467783 (0.793)
Reads mapping to positions which can't be uniquely counted to a probe = 262572	(0.047)
> UPENN_PE100_Indx20_1.rev.soap.out.Crick.counts <==
Reads = 5313396 	 Mapped = 4293686 (0.808)
Reads mapping to positions which can't be uniquely counted to a probe = 203878	(0.038)

DMR330K PE110[edit]

  • I used 3 bp left offset and 3 bp right offset
> AMD_PE110_1069001.fwd.soap.out.Watson.counts <==
Reads = 19146487 	 Mapped = 16522911 (0.863)
Reads mapping to positions which can't be uniquely counted to a probe = 906027	(0.047)
> AMD_PE110_1069001.rev.soap.out.Crick.counts <==
Reads = 16153762 	 Mapped = 14177970 (0.878)
Reads mapping to positions which can't be uniquely counted to a probe = 539169	(0.033)
> AMD_PE110_11201001c.fwd.soap.out.Watson.counts <==
Reads = 15142656 	 Mapped = 13786411 (0.910)
Reads mapping to positions which can't be uniquely counted to a probe = 703031	(0.046)
> AMD_PE110_11201001c.rev.soap.out.Crick.counts <==
Reads = 13059345 	 Mapped = 12028041 (0.921)
Reads mapping to positions which can't be uniquely counted to a probe = 431839	(0.033)

LC4K SE100[edit]

  • I used 5bp left offset and 8bp right offset
> H1-52C.gz.fwd.soap.out.Watson.counts <==
Reads = 1378189 	 Mapped = 540532 (0.392)
Reads mapping to positions which can't be uniquely counted to a probe = 986	(0.001)
> H1-52C.gz.rev.soap.out.Crick.counts <==
Reads = 1379820 	 Mapped = 539959 (0.391)
Reads mapping to positions which can't be uniquely counted to a probe = 3391	(0.002)
> H1-54C.gz.fwd.soap.out.Watson.counts <==
Reads = 1226710 	 Mapped = 603755 (0.492)
Reads mapping to positions which can't be uniquely counted to a probe = 1361	(0.001)
> H1-54C.gz.rev.soap.out.Crick.counts <==
Reads = 1242103 	 Mapped = 619599 (0.499)
Reads mapping to positions which can't be uniquely counted to a probe = 3578	(0.003)
> PGP1F-52C.gz.fwd.soap.out.Watson.counts <==
Reads = 2202867 	 Mapped = 840937 (0.382)
Reads mapping to positions which can't be uniquely counted to a probe = 1578	(0.001)
> PGP1F-52C.gz.rev.soap.out.Crick.counts <==
Reads = 2212586 	 Mapped = 834191 (0.377)
Reads mapping to positions which can't be uniquely counted to a probe = 4658	(0.002)
> PGP1F-54C.gz.fwd.soap.out.Watson.counts <==
Reads = 2205391 	 Mapped = 1071655 (0.486)
Reads mapping to positions which can't be uniquely counted to a probe = 2443	(0.001)
> PGP1F-54C.gz.rev.soap.out.Crick.counts <==
Reads = 2251836 	 Mapped = 1105802 (0.491)
Reads mapping to positions which can't be uniquely counted to a probe = 7099	(0.003)
> PGP1iPS-52C.gz.fwd.soap.out.Watson.counts <==
Reads = 1892249 	 Mapped = 844563 (0.446)
Reads mapping to positions which can't be uniquely counted to a probe = 1490	(0.001)
> PGP1iPS-52C.gz.rev.soap.out.Crick.counts <==
Reads = 1887968 	 Mapped = 854889 (0.453)
Reads mapping to positions which can't be uniquely counted to a probe = 4469	(0.002)
> PGP1iPS-54C.gz.fwd.soap.out.Watson.counts <==
Reads = 1547699 	 Mapped = 897668 (0.580)
Reads mapping to positions which can't be uniquely counted to a probe = 2399	(0.002)
> PGP1iPS-54C.gz.rev.soap.out.Crick.counts <==
Reads = 1549348 	 Mapped = 897127 (0.579)
Reads mapping to positions which can't be uniquely counted to a probe = 5484	(0.004)

Results[edit]

File:DMR330K probesMatrix.txt.gz
File:LC4K probesMatrix.txt

Grouping of probes[edit]

  • Only UPENN data had capture from Sub6 and Sub7, but I have only mapped one dataset so far. I will need to go back and map additional datasets and consider Sub6 and Sub7 separately
1) Probes that never get any capture in any dataset
Exp1-3 23,931
Sub1       40
Sub2       34
Sub3      246
Sub4      213
Sub5    6,137
================
        30,601 probes out of 237,128 (12.9%)
2) Probes that are inconsistent across datasets
I defined these as having 0 capture for at least 1 dataset and a normalized standard deviation of capture of 0.28 or greater 
To get the normalized standard deviation of capture: calculate the standard deviation of coverage across multiple captures and divide by range of capture (max capture coverage - min capture coverage).
I need to normalize the capture coverage within each sample by the total reads mapped.
3) Probes that are consistent across datasets and can further be subset by level of efficiency


Thought to determine capture bias on each base position:[edit]

  • Find all positions which is unique to only one probe, count and then take the average and standard deviation for those counts.
  • Easiest is to use take the pileup as input
  • Within one capture, how does the performance of one probe varies along it's target positions?