Athurva Gore: Difference between revisions

From ZhangLabWiki
Jump to navigation Jump to search
>Ajgore
No edit summary
>Ajgore
No edit summary
 
(44 intermediate revisions by the same user not shown)
Line 1: Line 1:
=Lab Rotation=
* [[Athurva_Gore:LabNotes/LabMeetings|Lab Presentations]]
*Will add daily entries once schedule settles down a bit.
* [http://genome-tech.ucsd.edu/LabNotes/probedesign Probe Designer]
* [[Athurva_Gore:LabNotes/PrimerDes|PCR Primer Design Instructions]]
* [[Athurva_Gore:LabNotes/ExomePipeline|Human Genome Resequencing Pipeline]]
=Current Tasks=
* [[Athurva_Gore:LabNotes/RnaEditing|RNA Editing]]
* [[Athurva_Gore:LabNotes/ExomeReseq|Exome Resequencing (PGP)]]


=August=
=Overall Notes=
==August 4th - August 10th==
* [[Athurva_Gore:IupacAmbiguity|IUPAC Ambiguity Codes for SNPs]]
* Received CpG island probe code from Dr. Zhang.
<calendar>
* Need to make modifications discussed in meeting:
name=Athurva_Gore
* Necessary modifications:
format=%name:LabNotes/%year-%month-%day
** '''Remove bisulfite conversion function - DONE'''
date=2012/01/03
*** Create dummy function that simply does not do a replacement, but creates fwd and rev strands
view=oneyear
*** Commented code to find/replace
</calendar>
*** Question:  parts we are looking at; do they have N's in them?  Probably shouldn't.
<calendar>
** '''Remove CG score penalty - DONE'''
name=Athurva_Gore
*** in function getProbeList
format=%name/LabNotes/%year-%month-%day
*** remove section starting with "my $CG_penalty"
date=2011/01/06
*** set CG_penalty as always zero for quick fix, remove CG finding code later
view=oneyear
** '''Add function to check sequence uniqueness in human genome (build hash table ahead of time, check Dr. Zhang's functions)'''
</calendar>
*** Looks like function oligoFreq does this; currently just uses log of each side
<calendar>
*** Modify probe score calculation to favor segment B being more unique
name=Athurva_Gore
*** See function getProbeList
format=%name/LabNotes/%year-%month-%day
** '''Modify score calculation so that Tm of segment A should be high and Tm of segment B should be low'''
date=2010/01/04
*** At the moment, Tm score is used in p_score calculation
view=oneyear
*** Simply modify to adjust so that A has high Tm, B has low Tm
</calendar>
*** See function getProbeList
<calendar>
*** Specific thresholds?
name=Athurva_Gore
** '''Exon database'''
format=%name/LabNotes/%year-%month-%day
*** Ask about our current UCSC data; it comes in format on PGP server, or is this modified?
date=2009/04/07
*** Found Exon-Intron Database; uses GenBank
view=oneyear
*** Show Dr. Zhang the readme
</calendar>


==August 11th - August 17th==
=ROTATION STUFF BELOW=
* Will copy from notebook
=DNA Folding Energies=
==August 18th - August 24th==
* Will copy from Notebook
==August 27th==
*Have code to read and analyze Cosmic set data; file is [[Media:ReadCosmic_20080827.txt‎|"readCosmic.pl"]]
*'''Comparison of Cosmic set data to Dr. Li's data (Harvard Collaborator)'''
**Dr. Li's data showed that the last base pair of the ligation arm was important.
***Dr. Li checked average expression for each base.
***G was preferred for ligation, followed by A, C, and T.
***[[Image:08_22_2008_JinLi_Ligation_Arm_Base_Dist.png]]
**Average expression was calculated using the [[Media:MakeHist_20080827.txt|"makeHist.pl"]] file.
***This file averages the abundance of the probes that used each base.
**The results were slightly different.
***For the COSMIC set, it was found that for the proximal ligation arm, G was preferred, followed by C, A, and T.
***This makes intuitive sense, as the bond between G and C is stronger than the bond between A and T.
***The results were also much less pronounced than Dr. Li's results; the difference in base performance is much smaller.
{|border="1" cellpadding="5" cellspacing="0"
| align="center" colspan="21" | '''Cosmic Set Results'''
|-
|'''Base:'''||A||T||G||C
|-
|'''Proximal Extension Arm:'''||98.2276826484018||105.4814504701790||97.8751164168995||105.1120477176380
|-
|'''Distal Extension Arm:'''||104.9745183977170||107.4126226040130||106.5526921141540||88.8525735059634
|-
|'''Proximal Ligation Arm:'''||95.1374247734938||83.7531848897751||135.360909000299||107.876484056988
|-
|'''Distal Ligation Arm:'''||103.1547820103830||113.0664396095970||96.5690024678530||96.7110820814283
|}
*[[Image:ProxDist_Base_Cosmic.png]]
**Analyze these results later; perhaps add in a preference on the Proximal Ligation Arm to favor G or C over A or T in the score.
*'''Implementation of new probe set algorithm.'''
**Algorithm is summarized above.
**Implement in Perl using hash table in getCpGIslands.pl file.
*'''Use R to analyze Cosmic Data Set'''
** Downloaded and installed GNU R and R-commander (GUI)
***Can load CSV and tab-separated files.
**''lm'' command can do a linear model fit to the data.
***May need to take the log of several pieces of data however in order to improve fit (if data is exponentially correlated).
***Initial ''lm'' results for Cosmic set show a very poor correlation...probably need to take log of several pieces of data set.
**Earlier, Dr. Zhang's analysis on CES22K set used the following factors:
***ProbeDist: distribution of oligo abundance determined by Solexa sequencing. - '''Don't have this for the Cosmic set'''
***H1_Tm, H2_Tm: H1 (ligation arm), H2 (extension arm), 12bp sub-sequences of H1 and H2. - '''Normalized for the Cosmic set; very close to 62 C in all cases'''
***H1R_dist, H2L_dist: K-mer distribution of H1&H2 12bp sub-sequences in the human genome. - '''Have this'''
***Core_25bp_genomic_homology: weighted score of the 25bp homologous sequences in the human genome, mapped by SOAP with <=3bp mismatches. - '''Not sure what this is...ask Dr. Zhang'''


==August 28th==
=Lab Rotation=
*TO DO TODAY:
*Will add daily entries once schedule settles down a bit.
**Take log of various factors in R, see if this affects ''lm'' results.
*[[Athurva_Gore:LabNotes/LabRotation|Lab Rotation Notes]]
**See if any other combinations of factors produce better fits in R.
**Implement algorithm to select probes in Perl.
*'''R Analysis of Cosmic Data Set'''
**Tried taking the log of all factors aside from Tm, checked fit.
**Fit is still not very good
***Very low R^2 value
***Checked by plotting in Octave; fit does not match up to data well at all.
**Perhaps a different, non-linear, non-logarithmic model is needed?
**One issue with the Cosmic Data Set is that there is no data for oligonucleotide bias on the chip.
***Agilent's chips have some inherent bias in what molecules are present and not present.
***This bias was measured for the CES22k set that Dr. Zhang worked on but ''not'' for the Cosmic set.
***This may be causing issues with fitting a curve to the Cosmic set, as this bias cannot be used in the fit.
**Came up with some ideas while working from home.
***Perhaps instead of log or linear fit, use a normal distribution type fit?
***Could fit a normal distribution to each factor and try and standardize it; this might result in a better overall fit.
***I am currently looking into how to easily do this in R or Octave; shouldn't be too hard to do a fit this way.
**Received data for CES22K set
***A preliminary look at the data in Octave makes it seem that R^2 values will remain very low.
***Confirmed that fit is not that great in Octave.
***However, data for bias is present; this means that this can be factored into the analysis.
***Work on this tomorrow.
 
==August 29th==
*TO DO TODAY:
**Make some slides for Cosmic and CES22k sets to illustrate findings for linear and log fits
**Hopefully find a good way to fit normal distributions to the data; create some plots for this
**Test CES22k set for proximal and distal bases as well (just as with Dr. Li's data and Cosmic set above)
*'''Analysis of Cosmic and CES22k Data Set'''
**R^2 value of fit is very low due to large number of data points.
***Idea from Dr. Zhang; check high and low end points for input factors (maybe outside 1 std. deviation?)
***This might increase Signal-to-Noise ratio for the data.
**Was doing some reading on R linear model modules; stumbled across ''effects'' module.
***''effects'' uses results from an ''lm'' command and generates "effect plots"
***It generates the effects of a set of variables on an output variable in individual graph form
***Red lines in graphs represent a 95% confidence interval
***A paper on the ''effects'' module can be found here: http://www.jstatsoft.org/v08/i15/paper
***'''Need to investigate this module and make sure that any assumptions it makes are valid for the data we are using.'''
****It seemed that it was being used in a variety of applications online that used linear models.
**Used the ''effects'' module on Dr. Zhang's earlier CES22k fit to determine effects of various factors on expression output.
***The results for this are very interesting, and provide a much nicer picture than just the fit data.
----
*'''CES22K SET:'''[[Image:CES22K Effect Plots.png]]
----
*'''COSMIC SET:'''[[Image:Cosmic Effect Plots.png]]
----
*'''Proximal and Distal Arm Base Comparison'''
**The above analysis (using [[Media:MakeHist_CES.txt|makeHist_CES.pl]]) was repeated for the CES22k set.
**The results are below.
{|border="1" cellpadding="5" cellspacing="0"
| align="center" colspan="21" | '''CES22k Set Results'''
|-
|'''Base:'''||A||T||G||C
|-
|'''Proximal Extension Arm:'''||305.94||326.29||236.49||247.1
|-
|'''Distal Extension Arm:'''||283.08||290.35||263.52||257.27
|-
|'''Proximal Ligation Arm:'''||244.76||163.81||402.33||253.13
|-
|'''Distal Ligation Arm:'''||280.15||303.46||245.07||263.64
|}
*[[Image:ProxDist_Base_CES22k.png]]
**Similar to Cosmic set results; G and C are preferred at the Proximal Ligation site over A and T.
**Differences are more pronounced, like Dr. Li's data.
**Some differences are present in the other bases as well...possibly analyze for significance (in Dr. Li's data, Cosmic data, and this data)?
*'''Slides to illustrate findings so far'''
**Will make tonight (8-29) and email out.
 
=September=
==September 1==
*Made notes for powerpoint slides at home; however, came up with new things to do before powerpoint is finished.
*Will be presenting on Thursday.
==September 2==
*'''TO DO:'''
**Add in GC content, AT content for arms and target region for ''effects'' plots.
**Make slides; show each factor's effect graphs with explanation and a comparison of Proximal/Distal Bases (including Dr. Li's data)
***Describe how these curves will be used to affect scoring in next set of probes
***Take how score is currently calculated into account, suggest modifications?
***Decide how best to scale everything.
*'''PowerPoint'''
**Analyzed data and made preliminary powerpoint of results.
**Found that:
***1)  For the proximal ligation character, G is preferred, followed by C, A, and then T.
***2)  A high Tm for the ligation arm is good.
***3)  A low Tm for the extension arm is good.
***4)  Smaller targets are captured with higher efficiency.
***5)  Low GC content is better for both arms according to the Cosmic set.  I still need to look at the CES22K set.
***6)  Conclusions cannot really be drawn from the frequency data for the Cosmic set or the CES22K set, as the confidence interval is very very large.  In general, the only thing I can see is that the ligation arm frequency should be higher.
**'''Here is the finished PowerPoint:''' [[Media:Cosmic_CES22K_Presentation.ppt]]
**Still need to add target GC content data and arm GC content from CES22K set.
*'''Incorporating Effect Plots into Scoring'''
**Effect plots have allowed each factor to have an equation fitting its value to output.
**This allows these equations to be directly used in determining probe design.
**Perhaps compare current scoring method (that uses match between oligos A and B in code) to output as well?
**Then, would be able to determine how much to scale this factor as well as the other, already fit factors.
 
==September 3==
*'''TO DO:'''
**Analyze arm GC content from CES22K Set and add to Powerpoint.
**Add arm length data from CES22K Set and Cosmic Set to Powerpoint.
**Implement new scoring system.
*'''Data Analysis and [[Media:Cosmic_CES22K_Presentation.ppt|PowerPoint Presentation]]'''
**Analyzed arm length from Cosmic set; found that longer arms (on both sides) result in a better capture.
***Will add this to PowerPoint and to scoring.
**Need to extract and analyze GC Content from CES22K Set.
***Code to do this is running right now.  After this, graphs will be created in R and added to PowerPoint and scoring system.
**Turns out arm lengths were normalized to 20 for this set; cannot derive anything from this because of this.
**GC results are different; it was found that the ligation arm having a high GC raised expression, while the extension arm having a high GC hurt expression.
***In the Cosmic set, higher GC in either arm led to decreased expression.
***I am not sure why this is the case; look into how probes were constructed for tips.
*'''New Scoring System'''
**For scoring, intercept of fit does not really matter, since all scores are just going to be compared to each other.  We therefore just need to apply the slope.
**New Score:
***p_score = m_LTm * Left_Tm + m_RTm * Right_Tm + m_LUnique * L_Unique + m_RUnique * R_Unique + m_TLength * T_Length + m_LLength * L_Length + m_RLength * R_Length + m_LGC * GC_Content_Left + m_RGC * GC_Content_Right + proxLigFactor
**This scoring should produce much better probes than previous iterations, as all of these factors will be fitted.
**Additional factors can then be analyzed as well.
**Added new scoring system to GetCpGProbes.pl file.  Just need to decide exactly which slopes to use for probe scoring.
***Will probably average results from Cosmic and CES22K (after making sure that other factors didn't cause differences; if this is the case, a weighted average will be used).
 
==September 4==
*'''TO DO:'''
**Give presentation on data analysis to Ida, Alice, and Dr. Zhang.
**Finish up code for new probe design code (add in slopes from R analysis, finalize algorithm to select all probe sets).
*'''Presentation'''
**Added some finishing touches to PowerPoint; will go over it to make sure that I know it well.
**Make sure that it looks good; make any changes necessary for style.
*''New Probe Design Code''
**Added in framework for scoring yesterday.
**Just need to add in slopes today.
**Maybe have the code read them from a file?  Won't have to edit code every time, can instead just edit a "scoring method" file.
**Will average slopes from Cosmic and CES22K where both are present.
**Will use average offset as well for the ligation base character.

Latest revision as of 19:30, 16 February 2012

Current Tasks[edit]

Overall Notes[edit]

<calendar> name=Athurva_Gore format=%name:LabNotes/%year-%month-%day date=2012/01/03 view=oneyear </calendar> <calendar> name=Athurva_Gore format=%name/LabNotes/%year-%month-%day date=2011/01/06 view=oneyear </calendar> <calendar> name=Athurva_Gore format=%name/LabNotes/%year-%month-%day date=2010/01/04 view=oneyear </calendar> <calendar> name=Athurva_Gore format=%name/LabNotes/%year-%month-%day date=2009/04/07 view=oneyear </calendar>

ROTATION STUFF BELOW[edit]

DNA Folding Energies[edit]

Lab Rotation[edit]