Editing
Chris:LabNotes/FateMapping/Calendar/2014/2014-9-22
(section)
Jump to navigation
Jump to search
Warning:
You are not logged in. Your IP address will be publicly visible if you make any edits. If you
log in
or
create an account
, your edits will be attributed to your username, along with other benefits.
Anti-spam check. Do
not
fill this in!
==Procedure== ===Change targetFile to contain strands w/o C=== *Each microsatellite target derived from the UCSC genome browser contains the microsatellite subunit sequence along with the number of times that subunit is repeated (ex: 10xAG). However, because we want to run bisulfite conversion prior to probe capture, we want to take into account that unmethylated C's will be converted to U's that would consequently be converted to T's after amplification. Consequently, we want to avoid the strand containing C's in the MS target and use the opposite strand. *To do this, we use the fact that the MS targets listed in the bed file refer to the Watson (+) strand of DNA in the reference. So, if there is a C in the MS target sequence (ex: 10xAC), we want to choose the Crick strand (-). If there is a G in the MS target sequence (ex: 10xAG), we want to choose the Watson strand (+). If there are neither C or G in the MS target, we can allow ppDesigner to choose the strand, and if there are both C and G in the MS target, we have to throw out the target altogether. *To accomplish this, I wrote a perl script <[[File:cw_2014-09-22_Microsat_filter_all.txt]]> that takes in the bed file containing all MS targets <[[File:cw_2014_09_22_Hg19_2nt-3nt-4nt_MS_location.txt]]> and outputs three targetFiles (one for each type of MS, i.e. 2nt, 3nt, 4nt) with the fourth column of the targetFile indicating the desired strand and the probe target subunit sequence changed to reflect that strand. If the script works, the probe target subunit sequences should not contain any C's (as the script changes the MS subunit sequence to reflect the corresponding strand). The command to run this is as follows: perl microsat_filter_all.pl hg19_2nt-3nt-4nt_MS_location.txt *The following are the resulting MS targetFiles that will be inputted into ppDesigner: [[File: cw_2014-09-22_targetFile_2nt.txt]] [[File: cw_2014-09-22_TargetFile_3nt.txt]] [[File: cw_2014-09-22_TargetFile_4nt.txt]] ===Run ppDesigner=== *'''This part of the protocol has to be run on genome-miner, as Unafold is only installed there''' *I used ppDesigner v2 for this run (this can be found in genome-miner at </home/samchiang/Chris/software/ppDesigner_BSPP_v2.0/>) and can also be downloaded here [[File:cw_2014-09-22_PpDesigner_BSPP_v2.0.zip]] *In order to run ppDesigner for <u>bisulfite probes</u>, we perform the following steps (once I have the targetFiles for each type of microsatellite target): 1) Set the min/max melting temperature accepted for probes to 50/65 in the probe_parameters.pl file in ppDesigner by changing to the following lines: our $primerMaxTm = '''65'''; our $primerMinTm = '''50'''; 2) Use the paramters in the following example jobFile. These parameters are necessary as padlock probes targetting bisulfite converted DNA require longer probe arms for higher specificity. In addition, we have to also indicate that these probes are bisulfite probes [[File:cw_2014-09-22_jobFile_mod_90-110.txt]] 3) We then call and run ppDesigner using the following example command: /home/samchiang/Chris/software/ppDesigner_BSPP_v2.0-BSPP-params/src/ppDesigner.pl jobFile_mod_90-110.pl > hg19_2nt_ms_BSPP_20140922.txt *In order to run ppDesigner for <u>regular probes</u>, we perform the following steps (once I have the targetFiles fore ach type of microsatellite target): 1) Set the min/max melting temperature accepted for probes to 50/65 in the probe_parameters.pl file in ppDesigner by changing to the following lines: our $primerMaxTm = '''65'''; our $primerMinTm = '''55'''; 2) Use the paramters in the following example jobFile. These parameters are necessary as padlock probes targetting bisulfite converted DNA require longer probe arms for higher specificity. In addition, we have to also indicate that these probes are bisulfite probes [[File:cw_2014-09-22_regular_probes_JobFile_mod_90-110.txt]] 3) We then call and run ppDesigner using the following example command: /home/samchiang/Chris/software/ppDesigner_BSPP_v2.0/src/ppDesigner.pl jobFile_mod_90-110.pl > hg19_4nt_ms_20140922.txt *I ran each of these commands separately for each type of microsatellite (2nt, 3nt, 4nt) and the raw output files from ppDesigner as as follows: <u>Bisulfite Probes</u> [[File:Hg19_2nt_ms_BSPP_20140922.txt]] [[File:Hg19_3nt_ms_BSPP_20140922.txt]] [[File:Hg19_4nt_ms_BSPP_20140922.txt]] <u>Regular Probes</u> [[File:Hg19_2nt_ms_20140922.txt]] [[File:Hg19_3nt_ms_20140922.txt]] [[File:Hg19_4nt_ms_20140922.txt]] ===Filter by Mapping=== *All filtering was done in genemapster at </media/3TB_slot2/cjwei/Fate_Mapping/pp_design/ppDesigner_20140922> in each corresponding folder *For regular probes, we don't have to worry about strand specificity (i.e. there is no conversion in bases of sequence depending on the strand). Consequently, we can run the bash script: [[File:cw_20140922_1_filter-by-mapping.sh]] that performs the following steps 1) Creates simulated paired-end reads (one for each capture arm) from the original ppDesigner output using the perl script: [[File:cw_20140922_Probe_size.txt]] 2) We then verify that the capture arms map to the reference hg19 genome accordingly using Bowtie2 3) Once we have the output of Bowtie2 in sam format, we then want to filter out only those reads that mapped concordantly with both pairs mapping uniquely. Bowtie2 uses the tag "XS:i:" to denote unique mapping. Hence, we use samtools to filter only reads that mapped concordantly (using -Shf 0x2). Then we used grep to filter out any reads that had the "XS:i:" tag. We also remove header lines that start with "@." Finally, I used a python script from online that removes any orphan reads that mapped uniquely but the mate does not. This script is: [[File:cw_20140922_Extract_uniquely_mapped.txt]] 4) Next we extract only the read names from the sam file (i.e. extract only the targetIDs, one for each mate) using the script: [[File:cw_20140922_Sam2read-name.txt]] 5) We then extract only probe lines from the original ppDesigner output that were listed in the read names that passed the filter by mapping by using the custom script: [[File:cw_20140922_Merge_probe_files.txt]] *For bisulfite probes, we add a step in the beginning to divide the ppDesigner output into two files based on whether the probes target the Watson or the Crick strand. To accomplish this, I used the script: [[File:cw_20140922_Probe_split.txt]]. However, once we do that, we perform the subsequent steps listed above (steps 2-5) the same but for each strand separately then we combine all these in the end. The bash script that performs the steps is [[File:cw_20140922_1_filter-by-mapping-BSPP.sh]] *The resulting probe file after filter by mapping are as follows: <u>Bisulfite Probes</u> [[File:Hg19_2nt_ms_BSPP_20140922.txt.C.only_unique_alignment.txt]] [[File:Hg19_2nt_ms_BSPP_20140922.txt.W.only_unique_alignment.txt]] [[File:Hg19_3nt_ms_BSPP_20140922.txt.C.only_unique_alignment.txt]] [[File:Hg19_3nt_ms_BSPP_20140922.txt.W.only_unique_alignment.txt]] [[File:Hg19_4nt_ms_BSPP_20140922.txt.C.only_unique_alignment.txt]] [[File:Hg19_4nt_ms_BSPP_20140922.txt.W.only_unique_alignment.txt]] <u>Regular Probes</u> [[File:Hg19_2nt_ms_20140922.txt.only_unique_alignment.txt]] [[File:Hg19_3nt_ms_20140922.txt.only_unique_alignment.txt]] [[File:Hg19_4nt_ms_20140922.txt.only_unique_alignment.txt]] ===Filter by Alignability=== *For regular probes, alignability was determined using the following steps: 1) We again ran the probe_size.pl script in order to obtain a bed file of the capture arm start and end positions (this time using the starting point of the *.only_unique_alignment probe files produced by the previous filter by mapping step). This bed file contains two lines for each probe (one for each capture arm, i.e. ligation and extension arm) with the single base position for each. For example, the following is a pair of lines corresponding to one probe. Notice that the ligation and extension arm positions are calculated from knowledge of the flanking region (as you can see in the probe_size.pl script) chr10 1070927 1070928 chr10:001070950-1070992_LigArm chr10 1071050 1071051 chr10:001070950-1070992_ExtArm 2) We then found the intersection of the capture arm bed file with a bed file containing the alignability of all 24-mers in the hg19 genome. To do this, we run bedtools intersect agains the reference alignability bed file. This bed file can be found in genemapster at </media/3TB_slot2/cjwei/Fate_Mapping/pp_design/reference_genomes/hg19_alignability/wgEncodeCrgMapabilityAlign24mer.bed>. The output will the a file containing lines for only the capture arm position intersection. For example, the output for the same probe above would be: chr10 1070904 1070936 id-81105019 1 chr10 1070927 1070928 chr10:001070950-1070992_LigArm 1 chr10 1071022 1071072 id-81105048 1 chr10 1071050 1071051 chr10:001070950-1070992_ExtArm 1 3) Once we have the intersection file from step #2, I then used several custom perl scripts to parse the output. The first script is [[File:cw_20140922_Determine_alignability.txt]], which outputs the name of the probe (i.e. targetID) then the alignability for the ligation arm and extension arm. This contains all alignability scroes for all probes inputted, which can then be used to obtain some alignability statistics of the filter by mapping probes by using the script [[File:cw_20140922_Alignability_frequency.txt]]. However, the main output of the script is a file containing only the probe names in which both capture arms had alignability of 1. 4) Using the same merge_probe_files.pl script as used in step #5 in filter by mapping, we can then filter out only probes that had capture arm alignabilities of 1. *For bisulfite probes, the steps remain mostly the same, but we must also take into account strandedness. Consequently, we run the above steps twice, once for probes targetting the Watson strand and once for probes targetting the Crick strand. The step that changes is step #2 in which intersections for the Watson strand are against the reference at </media/3TB_slot2/cjwei/Fate_Mapping/pp_design/reference_genomes/hg19_alignability/hg19.bis.CT.mappability.bed>. The Crick strand reference is found at </media/3TB_slot2/cjwei/Fate_Mapping/pp_design/reference_genomes/hg19_alignability/hg19.bis.GA.mappability.bed>. The steps in creating these references are as follows: **'''This protocol was adapted from Dinh's old protocol for calculating mappability: <http://genome-tech.ucsd.edu/LabNotes/index.php/Dinh/Dinh_2014/NOTES/2014-4-30>''' 1) I created two hg19 references. One in which all C's were converted to T's (for the Watson strand after bisulfite conversioN) and one with G's converted to A's (Crick strand) 2) Using a package called GEM from <gemlibrary.sourceforge.net/>, we index each reference individually using the following commands: gem-indexer -i /oasis/tscc/scratch/cjwei/Fate_Mapping/hg19_reference.bis.CT.fa -o /oasis/tscc/scratch/cjwei/Fate_Mapping/hg19.bis.CT.GEM gem-indexer -i /oasis/tscc/scratch/cjwei/Fate_Mapping/hg19_reference.bis.GA.fa -o /oasis/tscc/scratch/cjwei/Fate_Mapping/hg19.bis.GA.GEM 3) We then used gem-mappability to create a mappability file (again, one for each strand) gem-mappability -T 8 -I hg19.bis.CT.GEM.gem -l 24 -o hg19.bis.CT.mappability gem-mappability -T 8 -I hg19.bis.GA.GEM.gem -l 24 -o hg19.bis.GA.mappability 4) I had to perform some file conversions of first converting the mappability file to wig using the following commands: gem-2-wig -I hg19.bis.CT.GEM.gem -i hg19.bis.CT.mappability.mappability -o hg19.bis.CT gem-2-wig -I hg19.bis.GA.GEM.gem -i hg19.bis.GA.mappability.mappability -o hg19.bis.GA 5) We then extracted necessary information using sed and wig2bed to obtain a bed file for each strand of reference hg19. Consequently, I used teh following commands: sed 's/_Watson//g' hg19.bis.CT.wig >hg19.bis.CT.wig.sed wig2bed < hg19.bis.CT.wig.sed > hg19.bis.CT.mappability.bed sed 's/_Crick//g' hg19.bis.GA.wig >hg19.bis.CT.wig.sed wig2bed < hg19.bis.CT.wig.sed > hg19.bis.CT.mappability.bed *The resulting probe files obtained from filter by alignability are as follows: <u>Bisulfite Probes</u> [[File:Hg19_2nt_ms_BSPP_20140922.txt.C.only_unique_alignment.only_unique_alignability.txt]] [[File:Hg19_2nt_ms_BSPP_20140922.txt.W.only_unique_alignment.only_unique_alignability.txt]] [[File:Hg19_3nt_ms_BSPP_20140922.txt.C.only_unique_alignment.only_unique_alignability.txt]] [[File:Hg19_3nt_ms_BSPP_20140922.txt.W.only_unique_alignment.only_unique_alignability.txt]] [[File:Hg19_4nt_ms_BSPP_20140922.txt.C.only_unique_alignment.only_unique_alignability.txt]] [[File:Hg19_4nt_ms_BSPP_20140922.txt.W.only_unique_alignment.only_unique_alignability.txt]] <u>Regular Probes</u> [[File:Hg19_2nt_ms_20140922.txt.only_unique_alignment.only_unique_alignability.txt]] [[File:Hg19_3nt_ms_20140922.txt.only_unique_alignment.only_unique_alignability.txt]] [[File:Hg19_4nt_ms_20140922.txt.only_unique_alignment.only_unique_alignability.txt]] ===Merge to determine intersection of BSPP and regular probes=== *This step is done in the folder in genemapster at </media/3TB_slot2/cjwei/Fate_Mapping/pp_design/ppDesigner_20140922/intersection_BSPP-regular_probes_noC> *The protocol for this step is short and only requires as input the concatenated files of each type of probe (Bisulfite and Regular), i.e. there will be two files [[File:Hg19_BSPP_probes_noC_20140922.txt]] and [[File:Hg19_regular_probes_noC_20140922.txt]] 1) We first sort each of these two input files based on targetID using the following commands (to make processing go by quicker): sort -k2 hg19_BSPP_probes_noC_20140922.txt >hg19_BSPP_probes_noC_20140922.txt.sorted sort -k2 hg19_regular_probes_noC_20140922.txt >hg19_regular_probes_noC_20140922.txt.sorted 2) We then obtain only the targetID's from each file by using the cut linux command cut -f2 hg19_BSPP_probes_noC_20140922.txt >hg19_BSPP_probes_noC_20140922.cut cut -f2 hg19_regular_probes_noC_20140922.txt >hg19_regular_probes_noC_20140922.cut 3) Next, I found the intersection between the two targetID *.cut files using grep grep -Fx -f hg19_BSPP_probes_noC_20140922.cut hg19_regular_probes_noC_20140922.cut | sort - >intersection.cut 4) Finally, I used the previous merge_probe_files.pl script to obtain only probes that target those specific targetID's perl merge_probe_files.pl intersection.cut hg19_BSPP_probes_noC_20140922.txt.sorted hg19_BSPP_probes_noC_20140922.intersected.txt perl merge_probe_files.pl intersection.cut hg19_regular_probes_noC_20140922.txt.sorted hg19_regular_probes_noC_20140922.intersected.txt *The output files can be used if we want a probe set that is directly comparable between bisulfite converted DNA and non-treated DNA, to determine if capture efficiency is different between one or the other.
Summary:
Please note that all contributions to ZhangLabWiki may be edited, altered, or removed by other contributors. If you do not want your writing to be edited mercilessly, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource (see
ZhangLabWiki:Copyrights
for details).
Do not submit copyrighted work without permission!
Cancel
Editing help
(opens in new window)
Navigation menu
Personal tools
Not logged in
Talk
Contributions
Create account
Log in
Namespaces
Page
Discussion
English
Views
Read
Edit
View history
More
Search
Navigation
Main Page
Current events
Recent changes
Random page
Investigators
Matt Cai
Song Chen
Eric Chu
Dinh Diep
Elizabeth Duong
Shicheng Guo
Alan Fung
Daniel Jacobsen
Blue Lake
Huy Lam
Alice Li
Andrew Richards
Brandon Sos
Chris Wei
Yan Wu
Kun Zhang
Tools
What links here
Related changes
Special pages
Page information