Chris:LabNotes/FateMapping/Calendar/2014/2014-9-22

From ZhangLabWiki
Jump to navigation Jump to search

Probe Design (filtered bisulfite and regular probes)[edit]

Background[edit]

  • Previously we found that the regular probes designed on 5/12 contained some capture arms that had very low alignability scores (i.e. will map to multiple places in the reference genome). This would consequently cause non-specific binding and may in turn be the cause of the very broad smear we're seeing in the probe capture gels. Because of this, I'll be redesigning the probes and applying a post-ppDesigner quality filtering.
  • In addition, after talking with Dr. Zhang, we want to also begin to look at DNA methylation at the same time as microsatellite data in order to obtain both phylogenetic and epigenetic data from each single cell. In order to do this, we'll need to perform bisulfite conversion on single cell DNA and perform probe capture. Consequently, I'll also be making another set of BSPP probes for bisulfite capture of microsatellite sites.
    • Because of the bisulfite conversion, all unmethylated C's will turn to U's, which in turn will be represented as T's after amplification. Consequently, when designing probes, we want to avoid the strand containing cytosines
  • The final padlock probes designed will fall into one of the following four categories (depending on the availability of space on the chip, we can order some of the following):
1) Bisulfite Padlock Probes (targetting strands that do not contain cytosine)
2) Regular probes (targetting strands that do not contain cytosine)
3) Intersection of BSPP and regular probes targetting strands that do not contain cytosine (for comparison purposes)
  • The general procedure is as follows
Change targetFile to contain strands w/o C
     |
     V
Run ppDesigner
     |
     V
Filter by Mapping (BSPP probes use strand specific mapping, regular probes map against general hg19 genome)
     |
     V
Filter by Alignability
     |
     V
Merge to determine intersection of BSPP and regular probes

Procedure[edit]

Change targetFile to contain strands w/o C[edit]

  • Each microsatellite target derived from the UCSC genome browser contains the microsatellite subunit sequence along with the number of times that subunit is repeated (ex: 10xAG). However, because we want to run bisulfite conversion prior to probe capture, we want to take into account that unmethylated C's will be converted to U's that would consequently be converted to T's after amplification. Consequently, we want to avoid the strand containing C's in the MS target and use the opposite strand.
  • To do this, we use the fact that the MS targets listed in the bed file refer to the Watson (+) strand of DNA in the reference. So, if there is a C in the MS target sequence (ex: 10xAC), we want to choose the Crick strand (-). If there is a G in the MS target sequence (ex: 10xAG), we want to choose the Watson strand (+). If there are neither C or G in the MS target, we can allow ppDesigner to choose the strand, and if there are both C and G in the MS target, we have to throw out the target altogether.
  • To accomplish this, I wrote a perl script <File:Cw 2014-09-22 Microsat filter all.txt> that takes in the bed file containing all MS targets <File:Cw 2014 09 22 Hg19 2nt-3nt-4nt MS location.txt> and outputs three targetFiles (one for each type of MS, i.e. 2nt, 3nt, 4nt) with the fourth column of the targetFile indicating the desired strand and the probe target subunit sequence changed to reflect that strand. If the script works, the probe target subunit sequences should not contain any C's (as the script changes the MS subunit sequence to reflect the corresponding strand). The command to run this is as follows:
perl microsat_filter_all.pl hg19_2nt-3nt-4nt_MS_location.txt
  • The following are the resulting MS targetFiles that will be inputted into ppDesigner:
File:Cw 2014-09-22 targetFile 2nt.txt
File:Cw 2014-09-22 TargetFile 3nt.txt
File:Cw 2014-09-22 TargetFile 4nt.txt

Run ppDesigner[edit]

  • This part of the protocol has to be run on genome-miner, as Unafold is only installed there
  • I used ppDesigner v2 for this run (this can be found in genome-miner at </home/samchiang/Chris/software/ppDesigner_BSPP_v2.0/>) and can also be downloaded here File:Cw 2014-09-22 PpDesigner BSPP v2.0.zip
  • In order to run ppDesigner for bisulfite probes, we perform the following steps (once I have the targetFiles for each type of microsatellite target):
1) Set the min/max melting temperature accepted for probes to 50/65 in the probe_parameters.pl file in ppDesigner by changing to the following lines:
     our $primerMaxTm = 65;
     our $primerMinTm = 50;
2) Use the paramters in the following example jobFile.  These parameters are necessary as padlock probes targetting bisulfite converted DNA require longer probe arms for higher specificity.  In addition, we have to also indicate that these probes are bisulfite probes
     File:Cw 2014-09-22 jobFile mod 90-110.txt
3) We then call and run ppDesigner using the following example command:
     /home/samchiang/Chris/software/ppDesigner_BSPP_v2.0-BSPP-params/src/ppDesigner.pl jobFile_mod_90-110.pl > hg19_2nt_ms_BSPP_20140922.txt
  • In order to run ppDesigner for regular probes, we perform the following steps (once I have the targetFiles fore ach type of microsatellite target):
1) Set the min/max melting temperature accepted for probes to 50/65 in the probe_parameters.pl file in ppDesigner by changing to the following lines:
     our $primerMaxTm = 65;
     our $primerMinTm = 55;
2) Use the paramters in the following example jobFile.  These parameters are necessary as padlock probes targetting bisulfite converted DNA require longer probe arms for higher specificity.  In addition, we have to also indicate that these probes are bisulfite probes
     File:Cw 2014-09-22 regular probes JobFile mod 90-110.txt
3) We then call and run ppDesigner using the following example command:
     /home/samchiang/Chris/software/ppDesigner_BSPP_v2.0/src/ppDesigner.pl jobFile_mod_90-110.pl > hg19_4nt_ms_20140922.txt
  • I ran each of these commands separately for each type of microsatellite (2nt, 3nt, 4nt) and the raw output files from ppDesigner as as follows:
Bisulfite Probes
File:Hg19 2nt ms BSPP 20140922.txt
File:Hg19 3nt ms BSPP 20140922.txt
File:Hg19 4nt ms BSPP 20140922.txt
Regular Probes
File:Hg19 2nt ms 20140922.txt
File:Hg19 3nt ms 20140922.txt
File:Hg19 4nt ms 20140922.txt

Filter by Mapping[edit]

  • All filtering was done in genemapster at </media/3TB_slot2/cjwei/Fate_Mapping/pp_design/ppDesigner_20140922> in each corresponding folder
  • For regular probes, we don't have to worry about strand specificity (i.e. there is no conversion in bases of sequence depending on the strand). Consequently, we can run the bash script: File:Cw 20140922 1 filter-by-mapping.sh that performs the following steps
1) Creates simulated paired-end reads (one for each capture arm) from the original ppDesigner output using the perl script: File:Cw 20140922 Probe size.txt
2) We then verify that the capture arms map to the reference hg19 genome accordingly using Bowtie2
3) Once we have the output of Bowtie2 in sam format, we then want to filter out only those reads that mapped concordantly with both pairs mapping uniquely.  Bowtie2 uses the tag "XS:i:" to denote unique mapping.  Hence, we use samtools to filter only reads that mapped concordantly (using -Shf 0x2).  Then we used grep to filter out any reads that had the "XS:i:" tag.  We also remove header lines that start with "@."  Finally, I used a python script from online that removes any orphan reads that mapped uniquely but the mate does not.  This script is: File:Cw 20140922 Extract uniquely mapped.txt
4) Next we extract only the read names from the sam file (i.e. extract only the targetIDs, one for each mate) using the script: File:Cw 20140922 Sam2read-name.txt
5) We then extract only probe lines from the original ppDesigner output that were listed in the read names that passed the filter by mapping by using the custom script: File:Cw 20140922 Merge probe files.txt
  • For bisulfite probes, we add a step in the beginning to divide the ppDesigner output into two files based on whether the probes target the Watson or the Crick strand. To accomplish this, I used the script: File:Cw 20140922 Probe split.txt. However, once we do that, we perform the subsequent steps listed above (steps 2-5) the same but for each strand separately then we combine all these in the end. The bash script that performs the steps is File:Cw 20140922 1 filter-by-mapping-BSPP.sh
  • The resulting probe file after filter by mapping are as follows:
Bisulfite Probes
File:Hg19 2nt ms BSPP 20140922.txt.C.only unique alignment.txt
File:Hg19 2nt ms BSPP 20140922.txt.W.only unique alignment.txt
File:Hg19 3nt ms BSPP 20140922.txt.C.only unique alignment.txt
File:Hg19 3nt ms BSPP 20140922.txt.W.only unique alignment.txt
File:Hg19 4nt ms BSPP 20140922.txt.C.only unique alignment.txt
File:Hg19 4nt ms BSPP 20140922.txt.W.only unique alignment.txt
Regular Probes
File:Hg19 2nt ms 20140922.txt.only unique alignment.txt
File:Hg19 3nt ms 20140922.txt.only unique alignment.txt
File:Hg19 4nt ms 20140922.txt.only unique alignment.txt

Filter by Alignability[edit]

  • For regular probes, alignability was determined using the following steps:
1) We again ran the probe_size.pl script in order to obtain a bed file of the capture arm start and end positions (this time using the starting point of the *.only_unique_alignment probe files produced by the previous filter by mapping step).  This bed file contains two lines for each probe (one for each capture arm, i.e. ligation and extension arm) with the single base position for each.  For example, the following is a pair of lines corresponding to one probe.  Notice that the ligation and extension arm positions are calculated from knowledge of the flanking region (as you can see in the probe_size.pl script)
     chr10	1070927	1070928	chr10:001070950-1070992_LigArm
     chr10	1071050	1071051	chr10:001070950-1070992_ExtArm
2) We then found the intersection of the capture arm bed file with a bed file containing the alignability of all 24-mers in the hg19 genome.  To do this, we run bedtools intersect agains the reference alignability bed file.  This bed file can be found in genemapster at </media/3TB_slot2/cjwei/Fate_Mapping/pp_design/reference_genomes/hg19_alignability/wgEncodeCrgMapabilityAlign24mer.bed>.  The output will the a file containing lines for only the capture arm position intersection.  For example, the output for the same probe above would be:
    chr10	1070904	1070936	id-81105019	1	chr10	1070927	1070928	chr10:001070950-1070992_LigArm	1
    chr10	1071022	1071072	id-81105048	1	chr10	1071050	1071051	chr10:001070950-1070992_ExtArm	1
3) Once we have the intersection file from step #2, I then used several custom perl scripts to parse the output.  The first script is File:Cw 20140922 Determine alignability.txt, which outputs the name of the probe (i.e. targetID) then the alignability for the ligation arm and extension arm.  This contains all alignability scroes for all probes inputted, which can then be used to obtain some alignability statistics of the filter by mapping probes by using the script File:Cw 20140922 Alignability frequency.txt.  However, the main output of the script is a file containing only the probe names in which both capture arms had alignability of 1.
4) Using the same merge_probe_files.pl script as used in step #5 in filter by mapping, we can then filter out only probes that had capture arm alignabilities of 1.
  • For bisulfite probes, the steps remain mostly the same, but we must also take into account strandedness. Consequently, we run the above steps twice, once for probes targetting the Watson strand and once for probes targetting the Crick strand. The step that changes is step #2 in which intersections for the Watson strand are against the reference at </media/3TB_slot2/cjwei/Fate_Mapping/pp_design/reference_genomes/hg19_alignability/hg19.bis.CT.mappability.bed>. The Crick strand reference is found at </media/3TB_slot2/cjwei/Fate_Mapping/pp_design/reference_genomes/hg19_alignability/hg19.bis.GA.mappability.bed>. The steps in creating these references are as follows:
1) I created two hg19 references.  One in which all C's were converted to T's (for the Watson strand after bisulfite conversioN) and one with G's converted to A's (Crick strand)
2) Using a package called GEM from <gemlibrary.sourceforge.net/>, we index each reference individually using the following commands:
     gem-indexer -i /oasis/tscc/scratch/cjwei/Fate_Mapping/hg19_reference.bis.CT.fa -o /oasis/tscc/scratch/cjwei/Fate_Mapping/hg19.bis.CT.GEM
     gem-indexer -i /oasis/tscc/scratch/cjwei/Fate_Mapping/hg19_reference.bis.GA.fa -o /oasis/tscc/scratch/cjwei/Fate_Mapping/hg19.bis.GA.GEM
3) We then used gem-mappability to create a mappability file (again, one for each strand)
     gem-mappability -T 8 -I hg19.bis.CT.GEM.gem -l 24 -o hg19.bis.CT.mappability
     gem-mappability -T 8 -I hg19.bis.GA.GEM.gem -l 24 -o hg19.bis.GA.mappability
4) I had to perform some file conversions of first converting the mappability file to wig using the following commands:
     gem-2-wig -I hg19.bis.CT.GEM.gem -i hg19.bis.CT.mappability.mappability -o hg19.bis.CT
     gem-2-wig -I hg19.bis.GA.GEM.gem -i hg19.bis.GA.mappability.mappability -o hg19.bis.GA
5) We then extracted necessary information using sed and wig2bed to obtain a bed file for each strand of reference hg19.  Consequently, I used teh following commands:
    sed 's/_Watson//g' hg19.bis.CT.wig >hg19.bis.CT.wig.sed
    wig2bed < hg19.bis.CT.wig.sed > hg19.bis.CT.mappability.bed
    sed 's/_Crick//g' hg19.bis.GA.wig >hg19.bis.CT.wig.sed
    wig2bed < hg19.bis.CT.wig.sed > hg19.bis.CT.mappability.bed
  • The resulting probe files obtained from filter by alignability are as follows:
Bisulfite Probes
File:Hg19 2nt ms BSPP 20140922.txt.C.only unique alignment.only unique alignability.txt
File:Hg19 2nt ms BSPP 20140922.txt.W.only unique alignment.only unique alignability.txt
File:Hg19 3nt ms BSPP 20140922.txt.C.only unique alignment.only unique alignability.txt
File:Hg19 3nt ms BSPP 20140922.txt.W.only unique alignment.only unique alignability.txt
File:Hg19 4nt ms BSPP 20140922.txt.C.only unique alignment.only unique alignability.txt
File:Hg19 4nt ms BSPP 20140922.txt.W.only unique alignment.only unique alignability.txt
Regular Probes
File:Hg19 2nt ms 20140922.txt.only unique alignment.only unique alignability.txt
File:Hg19 3nt ms 20140922.txt.only unique alignment.only unique alignability.txt
File:Hg19 4nt ms 20140922.txt.only unique alignment.only unique alignability.txt

Merge to determine intersection of BSPP and regular probes[edit]

  • This step is done in the folder in genemapster at </media/3TB_slot2/cjwei/Fate_Mapping/pp_design/ppDesigner_20140922/intersection_BSPP-regular_probes_noC>
  • The protocol for this step is short and only requires as input the concatenated files of each type of probe (Bisulfite and Regular), i.e. there will be two files File:Hg19 BSPP probes noC 20140922.txt and File:Hg19 regular probes noC 20140922.txt
1) We first sort each of these two input files based on targetID using the following commands (to make processing go by quicker):
     sort -k2 hg19_BSPP_probes_noC_20140922.txt >hg19_BSPP_probes_noC_20140922.txt.sorted
     sort -k2 hg19_regular_probes_noC_20140922.txt >hg19_regular_probes_noC_20140922.txt.sorted
2) We then obtain only the targetID's from each file by using the cut linux command
     cut -f2 hg19_BSPP_probes_noC_20140922.txt >hg19_BSPP_probes_noC_20140922.cut
     cut -f2 hg19_regular_probes_noC_20140922.txt >hg19_regular_probes_noC_20140922.cut
3) Next, I found the intersection between the two targetID *.cut files using grep
     grep -Fx -f hg19_BSPP_probes_noC_20140922.cut hg19_regular_probes_noC_20140922.cut | sort - >intersection.cut
4) Finally, I used the previous merge_probe_files.pl script to obtain only probes that target those specific targetID's
     perl merge_probe_files.pl intersection.cut hg19_BSPP_probes_noC_20140922.txt.sorted hg19_BSPP_probes_noC_20140922.intersected.txt
     perl merge_probe_files.pl intersection.cut hg19_regular_probes_noC_20140922.txt.sorted hg19_regular_probes_noC_20140922.intersected.txt
  • The output files can be used if we want a probe set that is directly comparable between bisulfite converted DNA and non-treated DNA, to determine if capture efficiency is different between one or the other.

Results[edit]