Dinh/Dinh 2014/NOTES/2014-8-27

From ZhangLabWiki
Jump to navigation Jump to search

Running BisReadMapper with BWA[edit]

Paired-End reads[edit]

  • We must use some kind of PE reads stitching software to combine R1 and R2. However, whenever we do that, the forward and reverse strand correlation always show poorer data quality than when using only R1 or R2 data alone. I've tested COPE and PEAR on GP1V4 capture data where 99% of all paired end reads were overlapping with PE 150 reads, but the results were always bad. So I decided that clipping overlap in the BAM files would be better than stitching PE reads together since all the models for stitching seems to be based on non-bisulfite data (for genome assembly purposes.)
  • BamUtils have clipOverlap module that I can use for clipping, but must correctly flag all paired reads in BAM files for this to work.
 http://genome.sph.umich.edu/wiki/BamUtil#Releases
  • I incorporated this step into BamExtractor module for BisReadMapper so there is no need to perform stitching in the future.
  • Note tha BamUtils clipOverlap will only perform soft-clip, so the clipping is only indicated in the CIGAR string but not performed on the actual sequence.
  • clipOverlap also guarantees that any overlapping PE reads will get trimmed while stitching program may fail to perform stitching on some overlapping PEs.
Sample File Read 1 only Read 2 only COPE PEAR clipOverlap
NP-GP1V4-Ind1-16-Dec26-Ind1_S1 N=2140, r=0.944999843530629 N=1956, r=0.948678833946809 N=4493, r=0.905426295537918 N=4575, r=0.880593815314892 N=3766, r=0.947693618322778
NP-GP1V4-Ind1-16-Dec26-Ind2_S2 N=1307, r=0.947753281327104 N=1309, r=0.919215799033252 N=3292, r=0.872723283319447 N=3004, r=0.853236631821437 N=2627, r=0.930015463563508
NP-GP1V4-Ind1-16-Dec26-Ind3_S3 N=1238, r=0.923621136109205 N=1049, r=0.896750712776968 N=2892, r=0.877732324287496 N=2630, r=0.846694242580542 N=2340, r=0.918090849213536
NP-GP1V4-Ind1-16-Dec26-Ind4_S4 N=1410, r=0.965669325893385 N=1324, r=0.954974476737397 N=3182, r=0.90491763432851 N=3022, r=0.861150327721996 N=2674, r=0.958392440240385
NP-GP1V4-Ind1-16-Dec26-Ind5_S5 N=1437, r=0.895241800627052 N=1272, r=0.833958253713675 N=3468, r=0.82057943147337 N=3109, r=0.779392712114823 N=2839, r=0.858475255821648
NP-GP1V4-Ind1-16-Dec26-Ind6_S6 N=1114, r=0.915859349526694 N=963 , r=0.932666798629509 N=2459, r=0.753298536985006 N=2254, r=0.678976136765023 N=1994, r=0.914004826333877
NP-GP1V4-Ind1-16-Dec26-Ind7_S7 N=458 , r=0.853321515878049 N=340 , r=0.892339454612493 N=1178, r=0.692036233842428 N=766 , r=0.570623971432356 N=896 , r=0.800442726207723
NP-GP1V4-Ind1-16-Dec26-Ind8_S8 N=775 , r=0.923504016994822 N=581 , r=0.924650338321257 N=1809, r=0.760451037849 N=1432, r=0.654256010151288 N=1430, r=0.898359952429347

Obtain UMIs[edit]

Trim adaptors[edit]

  • Example of using trim-galore for padlock data:
 trim_galore --phred33 --paired -a CAGGCAGATGTTATCGAGGTCCGAC -a2 GTCGGACCTCGATAACATCTGCCTG read1.fq read2.fq
  • Example of using trim-galore for RRBS data (used M-adaptors):
 trim_galore --phred33 --rrbs --paired --dont_gzip -a AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC -a2 AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT read1.fq read2.fq
  • Example of using trim-galore for WGBS data (used M-adaptors):
 trim_galore --phred33 --paired --dont_gzip -a AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC -a2 AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT read1.fq read2.fq

Step by step[edit]

  • BWA and some other short reads aligner can read into memory large reference files so multiple chromosome files can be concatenated into one multi-FASTA reference file. Concatenate all the reference FASTA files into one file using the following command:
cat chr*fa > genome.fa
  • Generate the index file for the reference file (genome.fa.fai) using Samtools :
samtools faidx genome.fa
  • Prepare the genome using genomePrep.pl. To see the specific usage instructions for this script, see the README file included with the BisReadsMapper package.
/BisReadMapper_vX.X/src/genomePrep.pl genome.fa context=cg convert=yes
  • Concatenate all the CpG positions files into one file using the following terminal command:
cat genome*cpositions.txt > genome.cpg.positions.txt
  • Generate the two index references required for BWA using the following terminal commands:
bwa index –p genome.CT.bwa genome.fa.bis.CT
bwa index –p genome.GA.bwa genome.fa.bis.GA
  • Generate the list_path file. An example list_path_Hg19 file is provided in the Example directory for the BisReadMapper package. The required paths are: scripts_dir, samtools, bamUtils, cpg_list, ref_fai, and ref_fa. The file for cpg_list is the position list generated with genomePrep.pl and may contain CH positions. The file for snp_list is a two column table with (1) chromosome names and (2) positions for calling homozygous reference genotypes. The file for ref_dbsnp is the reference dbSNP file for filtering low confident genotypes. The file for target_bed is a BED format file of targeted regions used to generate the on or off target rate and the fraction of targeted bases covered.
  • Run the BisReadMapper.pl script for mapping reads following the usage instructions. To see the specific usage instructions for this script, see the README file included with the BisReadsMapper package. It is recommended to trim the 5’ end of reads by the average length of the capture arm sequences.
/BisReadMapper_vX.X/src/BisReadMapper.pl –r Reads_1.fq,Reads_2.fq –W genome.CT.bwa –C genome.GA.BWA –g genome.fa.fai –a bwa –p 4 –n Sample1 -5 27 –q 20 –b 33 > Sample1.log
  • Generate a list of SAM files using the following command:
ls *sorted.sam > list_sams
  • Run the BamExtractor.pl script for mapping reads following the usage instructions. To see the specific usage instructions for this script, see the README file included with the BisReadsMapper package.
/BisReadMapper_vX.X/src/BamExtractor.pl –i list_sams –s list_paths –o Sample1 –r none –v no –b yes –p no –d 5
  • For batch processing of multiple reads files, generate a shell script similar Go.MethylationPipelineSE.sh or Go.MethylationPipelinePE.sh for single-end or paired-end reads respectively. The Go.MethylationPipelineX.sh files are provided in the Example directory for the BisReadMapper package. All variables are required. For PE, the FILES variable is a space separated list of FASTQ files prefixes, such as “Sample1 Sample2 Sample3”. The reads filenames must end with “_1.fq” and “_2.fq”, otherwise, the variables Read1_ID and Read2_ID must be changed accordingly. For SE, the FILES variable must be a space separated list of FASTQ files.