Editing
Athurva Gore:LabNotes/LabRotation
(section)
Jump to navigation
Jump to search
Warning:
You are not logged in. Your IP address will be publicly visible if you make any edits. If you
log in
or
create an account
, your edits will be attributed to your username, along with other benefits.
Anti-spam check. Do
not
fill this in!
=September= ==September 1== *Made notes for powerpoint slides at home; however, came up with new things to do before powerpoint is finished. *Will be presenting on Thursday. ==September 2== *'''TO DO:''' **Add in GC content, AT content for arms and target region for ''effects'' plots. **Make slides; show each factor's effect graphs with explanation and a comparison of Proximal/Distal Bases (including Dr. Li's data) ***Describe how these curves will be used to affect scoring in next set of probes ***Take how score is currently calculated into account, suggest modifications? ***Decide how best to scale everything. *'''PowerPoint''' **Analyzed data and made preliminary powerpoint of results. **Found that: ***1) For the proximal ligation character, G is preferred, followed by C, A, and then T. ***2) A high Tm for the ligation arm is good. ***3) A low Tm for the extension arm is good. ***4) Smaller targets are captured with higher efficiency. ***5) Low GC content is better for both arms according to the Cosmic set. I still need to look at the CES22K set. ***6) Conclusions cannot really be drawn from the frequency data for the Cosmic set or the CES22K set, as the confidence interval is very very large. In general, the only thing I can see is that the ligation arm frequency should be higher. **'''Here is the finished PowerPoint:''' [[Media:Cosmic_CES22K_Presentation.ppt]] **Still need to add target GC content data and arm GC content from CES22K set. *'''Incorporating Effect Plots into Scoring''' **Effect plots have allowed each factor to have an equation fitting its value to output. **This allows these equations to be directly used in determining probe design. **Perhaps compare current scoring method (that uses match between oligos A and B in code) to output as well? **Then, would be able to determine how much to scale this factor as well as the other, already fit factors. ==September 3== *'''TO DO:''' **Analyze arm GC content from CES22K Set and add to Powerpoint. **Add arm length data from CES22K Set and Cosmic Set to Powerpoint. **Implement new scoring system. *'''Data Analysis and [[Media:Cosmic_CES22K_Presentation.ppt|PowerPoint Presentation]]''' **Analyzed arm length from Cosmic set; found that longer arms (on both sides) result in a better capture. ***Will add this to PowerPoint and to scoring. **Need to extract and analyze GC Content from CES22K Set. ***Code to do this is running right now. After this, graphs will be created in R and added to PowerPoint and scoring system. **Turns out arm lengths were normalized to 20 for this set; cannot derive anything from this because of this. **GC results are different; it was found that the ligation arm having a high GC raised expression, while the extension arm having a high GC hurt expression. ***In the Cosmic set, higher GC in either arm led to decreased expression. ***I am not sure why this is the case; look into how probes were constructed for tips. *'''New Scoring System''' **For scoring, intercept of fit does not really matter, since all scores are just going to be compared to each other. We therefore just need to apply the slope. **New Score: ***p_score = m_LTm * Left_Tm + m_RTm * Right_Tm + m_LUnique * L_Unique + m_RUnique * R_Unique + m_TLength * T_Length + m_LLength * L_Length + m_RLength * R_Length + m_LGC * GC_Content_Left + m_RGC * GC_Content_Right + proxLigFactor **This scoring should produce much better probes than previous iterations, as all of these factors will be fitted. **Additional factors can then be analyzed as well. **Added new scoring system to GetCpGProbes.pl file. Just need to decide exactly which slopes to use for probe scoring. ***Will probably average results from Cosmic and CES22K (after making sure that other factors didn't cause differences; if this is the case, a weighted average will be used). ==September 4== *'''TO DO:''' **Give presentation on data analysis to Ida, Alice, and Dr. Zhang. **Finish up code for new probe design code (add in slopes from R analysis, finalize algorithm to select all probe sets). *'''Presentation''' **Added some finishing touches to PowerPoint; will go over it to make sure that I know it well. **Make sure that it looks good; make any changes necessary for style. *''New Probe Design Code'' **Added in framework for scoring yesterday. **Just need to add in slopes today. **Maybe have the code read them from a file? Won't have to edit code every time, can instead just edit a "scoring method" file. **Will average slopes from Cosmic and CES22K where both are present. **Will use average offset as well for the ligation base character. ==September 5== *'''TO DO:''' **Plug in final choices for slopes into Perl code. **Make sure new probe selection algorithm works. **Make sure new scoring works. **Try code out on some sequences. *'''Slope Choices''' **Based on the data in the above PowerPoint presentation, the following slopes were chosen for the code. $m_LLength=18.51978; #Ligation arm length - COSMIC $m_RLength=13.74286; #Extension arm length - COSMIC $m_TLength=-1.89955; #Target length - COSMIC $m_LTm=8.3619; #Ligation Tm - CES22K $m_RTm=-5.3790; #Extension Tm - CES22K $m_LGC=-326.17; #Ligation GC content - COSMIC - Maybe reduce a little if this is too much? $m_RGC=-217.60; #Extension GC content - COSMIC - Maybe reduce a little if this is too much? $m_LUnique=0; #Ignored for now. $m_RUnique=0; #Ignored for now. $g_Offset=268.845; #Offset for Proximal Ligation Base 'G'- Average of COSMIC and CES22K $a_Offset=169.95; #Offset for Proximal Ligation Base 'A'- Average of COSMIC and CES22K $t_Offset=123.78; #Offset for Proximal Ligation Base 'T'- Average of COSMIC and CES22K $c_Offset=180.505; #Offset for Proximal Ligation Base 'C'- Average of COSMIC and CES22K **This data was added to the GetCpgProbes.pl file. *'''Test probe selection algorithm and new scoring system''' **Chromosome 20, being the smallest chromosome, was used for testing (speed purposes). **A section was chosen for selection. ***'''Upload test file, link here''' **The code was tested and debugged. ==September 6-10== *'''TO DO:''' **Finish Perl probe generation script *Worked on Perl script *'''Learned several issues with how Perl passes arrays.''' **Implemented workarounds in code; used split and join in order to pass multiple arrays as multiple scalars. *'''Algorithm to make sure all possible probe combinations were used was implemented using recursion''' **Several bugs in recursion were present, causing infinite loops. **Fixed this; recursive algorithm to generate probe sets now works. *'''Looked around for list of Cosmic set genes''' **Found in Cosmic_3760_exons.list.txt ***Located in /home/kun/MIP/exonome on pgp.oxf freelogy server **Format of set is: exonID chromosome start end NM_032211-E2 chr10 100000811 100000923 NM_032211-E3 chr10 100001312 100001449 NM_032211-E4 chr10 100002099 100002215 ... ... ==September 11== *'''TO DO:''' **Modify Perl probe generation script to read files in Cosmic set format (like above) **Post new Perl script here **Run tests, compare new results to old Cosmic set results **Show Dr. Zhang the code *'''Cosmic File Reading''' **Need to modified Perl script so that Cosmic set type files can be read in successfully. **Also need to make sure that Gene and Exon IDs are output by the final program; otherwise it will be very difficult to keep track of probes. **These modifications have been made; an in-file setting now controls what file format to load from. *'''Fixing errors with some probes''' **It was discovered that some probe sets were causing errors while others were not; this was related to the way Perl's "return" function works. The probeList variables were not undefined, as they contained a single number with the error code for not finding any compatible probes. **This was fixed. *'''New Perl Script''' **[[Media:GetProbes 091108.txt|GetProbes 091108.txt]] *'''NEW: Redundant Sequences''' **Now, need to figure out a better way to deal with redundant portions of various genes. **Basically, different genes that include the same exon. **New data structure? ***Perhaps create exonIDs for various genes? Then, list genes as a combination of exons in this database? ***Prevents overlap. **Added refGene data from UCSC to a mysql table called refGene on laptop. ***Data can be ordered by chromosome, txStart and end regions. **However, multiple exons are stored in single rows. **A script could be used (PHP?) to extract each exon and create a new database table. **''Redundant Sequence Script Idea:'' ***Use PHP. ***Scan through table; grab all values from exonStarts and exonEnds (these are our exons.) ***For each exon, also grab the "name" field, the "chr" field, and the "strand" field. ***Reprint out new exons using a basic incrementing ID. Also reprint out gene data in a new table as well. '''Table Exon - Fields:''' exonID (autoincrement) chr strand start end '''Table ExonLink - Fields:''' geneID exonID position NOTE: There will be multiple entries for each geneID. In order to figure out which exons to sequence for a given gene, all exonIDs will be pulled out of the database. Probes for each exonID will be designed. Example: SELECT exonID FROM TABLE gene WHERE geneID=''geneid_value'' *'''After this is finished, probes can be ordered using new code.''' ==September 12== *'''TO DO:''' **Finish script that creates new tables. *'''Script to create refExon and refExonLink''' **Created a PHP script that can build the two new tables (refExon and refExonLink). **In order to differentiate between coding and non-coding regions, in the refExonLink table, I have added two columns: codeStart and codeEnd. **Basically, these two values allow us to figure out what is coding and what is not in the following way. **Cases: ***Out of bounds xxxxxx | | OR | | xxxxxx Both codeStart and codeEnd will be set to -1. ***In bounds | xxxxxx | Both codeStart and codeEnd will be set to -10. ***Overlap xxxx|xxx | codeStart will be set to the starting point of coding, codeEnd will be set to -10. | xxxx|xxx codeStart will be set to -10, codeEnd will be set to the end point of coding. xx|xxx|xx codeStart will be set to the starting point of coding, codeEnd will be set to the end point of coding. **This should allow coding and non-coding regions to be identified by the probe generation script. **Finished script; ran script on all RefSeq exons! **All refseq exons are now present in a mysql database on laptop. *'''Test with probe generation code''' **Using "SELECT INTO FILE," a set of exons was created to test the probe generation code. **Uploaded file to pgp.oxf, testing it now. ***Due to "brute-force" algorithm for genes, it seems to be taking a long time; make sure script is working. *'''Script test results''' **For now, I have disabled loading the KmerDist, as the code is not using Uniqueness calculations at the moment. This stops an initial delay during testing. **With small exons, script seems to work well; I had to lower the minimum target size to do this however, as some small exons were not working (small boundary arm length). **Large exons have a problem, however. ***Too many possible Oligos; code takes far too long for this to be feasible. **Options: ***Split large exons into two pieces. ***Find better algorithms to deal with large exons. ***Use much more stringent cutoffs for large exons.
Summary:
Please note that all contributions to ZhangLabWiki may be edited, altered, or removed by other contributors. If you do not want your writing to be edited mercilessly, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource (see
ZhangLabWiki:Copyrights
for details).
Do not submit copyrighted work without permission!
Cancel
Editing help
(opens in new window)
Navigation menu
Personal tools
Not logged in
Talk
Contributions
Create account
Log in
Namespaces
Page
Discussion
English
Views
Read
Edit
View history
More
Search
Navigation
Main Page
Current events
Recent changes
Random page
Investigators
Matt Cai
Song Chen
Eric Chu
Dinh Diep
Elizabeth Duong
Shicheng Guo
Alan Fung
Daniel Jacobsen
Blue Lake
Huy Lam
Alice Li
Andrew Richards
Brandon Sos
Chris Wei
Yan Wu
Kun Zhang
Tools
What links here
Related changes
Special pages
Page information