Editing
Dinh/Dinh 2015/NOTES/2015-1-7
Jump to navigation
Jump to search
Warning:
You are not logged in. Your IP address will be publicly visible if you make any edits. If you
log in
or
create an account
, your edits will be attributed to your username, along with other benefits.
Anti-spam check. Do
not
fill this in!
= 141216_HiSeqRapidRun = * Processed files are copied to genome-miner at: /media/Ext12T/DD_Ext12T/MONOD/141216_HiSeqRapidRunWGBS ** Sub-directories: BAMfiles, MethylFreq, BEDfiles * Library information from Noi: http://genome-tech.ucsd.edu/LabNotes/index.php/Noi/NOTES/2014-12-25 * Perform trimming with '''trim-galore''', mapping with '''bwa mem''', overlapping PE are clipped with '''bamUtils''' ** For trimming, use methylated adaptors sequences, quality trim with -q 20 from 3' ends, and trim 5 bp from 5' ends (methylation bias) * Do not remove clonal reads * Use Hg19_lambda reference (Hg19 plus LambdaDNA fasta files). * Use tscc with 4 processors per node ===Mapping steps=== * Make a table: {| class="wikitable" | PC-P_2||150106_HiSeqRapidWGBS/Fastq||s_1_1_ILMN_Indx01.txt,s_1_2_ILMN_Indx01.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | PC-P_4||150106_HiSeqRapidWGBS/Fastq||s_1_1_ILMN_Indx03.txt,s_1_2_ILMN_Indx03.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | NC-7||150106_HiSeqRapidWGBS/Fastq||s_1_1_ILMN_Indx07.txt,s_1_2_ILMN_Indx07.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | PC-P_9||150106_HiSeqRapidWGBS/Fastq||s_1_1_ILMN_Indx08.txt,s_1_2_ILMN_Indx08.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | 6-P-1||150106_HiSeqRapidWGBS/Fastq||s_1_1_ILMN_Indx09.txt,s_1_2_ILMN_Indx09.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | 6-P-2||150106_HiSeqRapidWGBS/Fastq||s_1_1_ILMN_Indx10.txt,s_1_2_ILMN_Indx10.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | 6-P-3||150106_HiSeqRapidWGBS/Fastq||s_1_1_ILMN_Indx11.txt,s_1_2_ILMN_Indx11.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | 7-P-3||150106_HiSeqRapidWGBS/Fastq||s_1_1_ILMN_Indx20.txt,s_1_2_ILMN_Indx20.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | 7-P-5||150106_HiSeqRapidWGBS/Fastq||s_1_1_ILMN_Indx21.txt,s_1_2_ILMN_Indx21.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | 7-P-6||150106_HiSeqRapidWGBS/Fastq||s_1_1_ILMN_Indx22.txt,s_1_2_ILMN_Indx22.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | 7-P-7||150106_HiSeqRapidWGBS/Fastq||s_1_1_ILMN_Indx23.txt,s_1_2_ILMN_Indx23.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | 7-P-8||150106_HiSeqRapidWGBS/Fastq||s_1_1_ILMN_Indx25.txt,s_1_2_ILMN_Indx25.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | 7-P-10||150106_HiSeqRapidWGBS/Fastq||s_1_1_ILMN_Indx27.txt,s_1_2_ILMN_Indx27.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | 6T-1||150106_HiSeqRapidWGBS/Fastq||s_2_1_Indx04.txt,s_2_2_Indx04.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | 6T-2||150106_HiSeqRapidWGBS/Fastq||s_2_1_Indx05.txt,s_2_2_Indx05.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | 6T-3||150106_HiSeqRapidWGBS/Fastq||s_2_1_Indx06.txt,s_2_2_Indx06.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | 6T-4||150106_HiSeqRapidWGBS/Fastq||s_2_1_Indx07.txt,s_2_2_Indx07.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | 6T-5||150106_HiSeqRapidWGBS/Fastq||s_2_1_Indx08.txt,s_2_2_Indx08.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | 7T-1||150106_HiSeqRapidWGBS/Fastq||s_2_1_Indx09.txt,s_2_2_Indx09.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | 7T-2||150106_HiSeqRapidWGBS/Fastq||s_2_1_Indx10.txt,s_2_2_Indx10.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | 7T-4||150106_HiSeqRapidWGBS/Fastq||s_2_1_Indx12.txt,s_2_2_Indx12.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | 7T-5||150106_HiSeqRapidWGBS/Fastq||s_2_1_Indx13.txt,s_2_2_Indx13.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | PCT-1||150106_HiSeqRapidWGBS/Fastq||s_2_1_Indx14.txt,s_2_2_Indx14.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | PCT-2||150106_HiSeqRapidWGBS/Fastq||s_2_1_Indx15.txt,s_2_2_Indx15.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | PCT-4||150106_HiSeqRapidWGBS/Fastq||s_2_1_Indx16.txt,s_2_2_Indx16.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | PCT-6||150106_HiSeqRapidWGBS/Fastq||s_2_1_Indx17.txt,s_2_2_Indx17.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |- | PCT-7||150106_HiSeqRapidWGBS/Fastq||s_2_1_Indx18.txt,s_2_2_Indx18.txt||64||none||AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC||AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT||WGBS |} * Split table file into 27 files, 1 line/sample per file: split -l 1 list_my_files aa_ * Run Go.Map.sh script on tscc, each sample will be processed on a separate node. ** New script added to simplify BisReadMapper pipeline: '''MasterBisReadMapper.pl''' in BisReadMapper_v1.4 <nowiki>#===Change the following paths===# cur_dir=`pwd` scripts_dir="/home/ddiep/scripts/BisReadMapper/src" #===Begin===# for n in aa_aa aa_ap aa_ab aa_ac aa_ad aa_ae aa_af aa_ag aa_ah aa_ai aa_aj aa_ak aa_al aa_am aa_an aa_ao aa_aq aa_ar aa_as aa_at aa_au aa_av aa_aw aa_ax aa_ay aa_az aa_ba do #1) Run mapper: echo "#!/bin/csh" > $n.job echo "#PBS -l nodes=1:ppn=4" >> $n.job echo "#PBS -l walltime=14:00:00" >> $n.job echo "#PBS -o $n.log" >> $n.job echo "#PBS -e $n.err" >> $n.job echo "#PBS -V" >> $n.job echo "#PBS -M diep.hue.dinh@gmail.com" >> $n.job echo "#PBS -m abe" >> $n.job echo "#PBS -A k4zhang-group" >> $n.job echo "cd /state/partition1/\$USER/\$PBS_JOBID" >> $n.job #echo "cd $cur_dir" >> $n.job echo "$scripts_dir/MasterBisReadMapper.pl -i $cur_dir/$n -s $cur_dir/list_paths_Hg19 -b yes > $cur_dir/$n.status" >> $n.job echo "cp -r * $cur_dir/" >> $n.job qsub -q hotel $n.job done #===End===# </nowiki> ===Mapping statistics=== {| class="wikitable" border=1 | align="center" style="background:#f0f0f0;"|'''SAMPLE ID''' | align="center" style="background:#f0f0f0;"|''' Total PE reads''' | align="center" style="background:#f0f0f0;"|''' Total reads''' | align="center" style="background:#f0f0f0;"|''' Total reads after trimming''' | align="center" style="background:#f0f0f0;"|'''Total mapped reads''' | align="center" style="background:#f0f0f0;"|'''%trimmed''' | align="center" style="background:#f0f0f0;"|'''%mapped''' |- | 6-P-1_map|| 6,304,203 || 12,608,406 || 11,953,618 || 10,355,738 ||5%||87% |- | 6-P-2_map|| 6,984,763 || 13,969,526 || 13,303,090 || 11,570,287 ||5%||87% |- | 6-P-3_map|| 7,758,987 || 15,517,974 || 14,491,124 || 12,482,525 ||7%||86% |- | 6T-1_map|| 7,086,511 || 14,173,022 || 13,834,798 || 11,869,709 ||2%||86% |- | 6T-2_map|| 7,489,616 || 14,979,232 || 14,638,550 || 12,820,323 ||2%||88% |- | 6T-3_map|| 23,044,384 || 46,088,768 || 45,125,296 || 36,126,640 ||2%||80% |- | 6T-4_map|| 7,345,257 || 14,690,514 || 14,302,984 || 12,351,984 ||3%||86% |- | 6T-5_map|| 8,242,846 || 16,485,692 || 16,084,486 || 13,871,915 ||2%||86% |- | 7-P-10_map|| 6,405,081 || 12,810,162 || 12,126,962 || 10,375,497 ||5%||86% |- | 7-P-3_map|| 5,117,607 || 10,235,214 || 9,714,174 || 8,343,291 ||5%||86% |- | 7-P-5_map|| 5,943,036 || 11,886,072 || 11,311,240 || 9,868,792 ||5%||87% |- | 7-P-6_map|| 8,151,067 || 16,302,134 || 15,454,040 || 13,420,765 ||5%||87% |- | 7-P-7_map|| 8,006,228 || 16,012,456 || 15,215,062 || 13,264,972 ||5%||87% |- | 7-P-8_map|| 6,321,921 || 12,643,842 || 11,948,422 || 10,309,480 ||6%||86% |- | 7T-1_map|| 7,397,882 || 14,795,764 || 14,414,610 || 12,579,473 ||3%||87% |- | 7T-2_map|| 7,913,136 || 15,826,272 || 15,471,842 || 13,386,508 ||2%||87% |- | 7T-4_map|| 7,428,250 || 14,856,500 || 14,495,860 || 12,632,367 ||2%||87% |- | 7T-5_map|| 7,933,340 || 15,866,680 || 15,474,202 || 13,465,056 ||2%||87% |- | NC-7_map|| 8,140,557 || 16,281,114 || 15,329,258 || 11,685,910 ||6%||76% |- | PC-P_2_map|| 6,236,300 || 12,472,600 || 11,848,168 || 10,306,946 ||5%||87% |- | PC-P_4_map|| 6,400,370 || 12,800,740 || 12,191,246 || 10,561,291 ||5%||87% |- | PC-P_9_map|| 7,500,258 || 15,000,516 || 14,213,460 || 12,272,000 ||5%||86% |- | PCT-1_map|| 7,092,180 || 14,184,360 || 13,848,186 || 11,921,316 ||2%||86% |- | PCT-2_map|| 8,566,243 || 17,132,486 || 16,739,144 || 14,465,075 ||2%||86% |- | PCT-4_map|| 8,189,668 || 16,379,336 || 16,003,166 || 13,781,204 ||2%||86% |- | PCT-6_map|| 9,228,450 || 18,456,900 || 17,987,752 || 15,681,405 ||3%||87% |- | PCT-7_map|| 7,697,260 || 15,394,520 || 15,030,202 || 12,980,554 ||2%||86% |} ===SNP matrix=== * Run MasterBisReadMapper with variant calling. Since I already have the sam files to give as input, MasterBisReadMapper will not need to re-map the data. * Homozygous SNPs are also called at dbSNP138 positions. * After getting the *filtered.SNP.txt files, generate the tped and tfam files for plink. /home/dinh/scripts/BisReadMapper/src/snp2tfiles.pl * Merge tped files into the large tped matrix for plink. /home/dinh/scripts/BisReadMapper/src/tped-merge.pl * Run plink: /home/nplongth/softwares/plink-1.07-x86_64/plink --noweb --geno 0 --maf 0.01 --tfile monod_141216 --recode --transpose --out monod_141216.plink **** 92639 SNPs failed missingness test ( GENO > 0 ) **** 84830 SNPs failed frequency test ( MAF < 0.01 ) 993 SNPs left. /home/nplongth/softwares/plink-1.07-x86_64/plink --noweb --tfile monod_141216.plink --cluster --matrix * the plink.mibs fie is a N x N matrix of genome-wide average IBS pairwise identities * use R to plot the dendrogram: A = read.table("plink.mibs", F) labels = read.table("monod_141216.plink.tfam",F) colnames(A) = labels$V2 plot(hclust(as.dist(1-A)), main="plink.mibs") dev.off() * SNPs dendrogram: [[File:monod_capture_plink_mibs.png | 800px]]
Summary:
Please note that all contributions to ZhangLabWiki may be edited, altered, or removed by other contributors. If you do not want your writing to be edited mercilessly, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource (see
ZhangLabWiki:Copyrights
for details).
Do not submit copyrighted work without permission!
Cancel
Editing help
(opens in new window)
Navigation menu
Personal tools
Not logged in
Talk
Contributions
Create account
Log in
Namespaces
Page
Discussion
English
Views
Read
Edit
View history
More
Search
Navigation
Main Page
Current events
Recent changes
Random page
Investigators
Matt Cai
Song Chen
Eric Chu
Dinh Diep
Elizabeth Duong
Shicheng Guo
Alan Fung
Daniel Jacobsen
Blue Lake
Huy Lam
Alice Li
Andrew Richards
Brandon Sos
Chris Wei
Yan Wu
Kun Zhang
Tools
What links here
Related changes
Special pages
Page information