Noi/NOTES/2012-4-30: Difference between revisions

From ZhangLabWiki
Jump to navigation Jump to search
>Noi
No edit summary
>Noi
No edit summary
 
(16 intermediate revisions by the same user not shown)
Line 1: Line 1:
* [[http://genome-tech.ucsd.edu/LabNotes/index.php/noi:DMR220k_LabNotes '''Link to calendar''']]
* [[http://genome-tech.ucsd.edu/LabNotes/index.php/noi:DMR220k_LabNotes '''Link to calendar''']]
* Good tutorial for LDA by R: https://onlinecourses.science.psu.edu/stat857/book/export/html/17
* Good tutorial for LDA by R:  
** https://onlinecourses.science.psu.edu/stat857/book/export/html/17
** http://miner.chem.purdue.edu/Lectures/Lecture10.pdf
** http://little-book-of-r-for-multivariate-analysis.readthedocs.org/en/latest/src/multivariateanalysis.html#allocation-rules-and-misclassification-rate
** http://stat.ethz.ch/education/semesters/ss2012/ams/Rscript/scriptV6.1.R (good details)
== Regression analysis of UCLA SZ data, applying LDA function ==
== Regression analysis of UCLA SZ data, applying LDA function ==
* Continued from: http://genome-tech.ucsd.edu/LabNotes/index.php/Noi/NOTES/2012-4-17
* Continued from: http://genome-tech.ucsd.edu/LabNotes/index.php/Noi/NOTES/2012-4-17
* I repeated regression analysis again by changing some criteria to select the candidate significant CpG sites, based on FDR or with minimum raw p-value. For status regression analysis, I will not apply multiple test correction since I didn't get any signal. However with minimum t statistic of 2.5 from the previous analysis I could separate out the normal and affected samples. Dr. Suggested that this should be fine. In addition, I will change the color of the control samples in PCA analysis plots. LDA will be used then the number of samples in the two groups classified by the line will be counted.
* I repeated regression analysis again by changing some criteria to select the candidate significant CpG sites, based on FDR or with minimum raw p-value. For status regression analysis, I will not apply multiple test correction since I didn't get any signal. However with minimum t statistic of 2.5 from the previous analysis I could separate out the normal and affected samples. Dr. Zhang suggested that this should be fine. In addition, I will change the color of the control samples in PCA analysis plots. LDA will be used then the number of samples in the two groups classified by the line will be counted.
== Color code for PCA plot ==
 
== Color code for PCA plot (some of them have been changed) ==
*'''Age'''
*'''Age'''
  mycol=c(1,1,4,4,4,1,1,4,1,1,4,4,1,4,1,1,4,1,1,4,4,4,4,1,1,4,4,1,1,4,4,4,1,1,4,4,4,1,1,4,1,1,4,4,1,1,4,4,1,1,4,4,4,4,1,1,4,4,1,4,4,4,4,1,1,4,4,1,1,4,4,1,1,4,4,4,1,1,4,4,1,1,4,4,1,1,4,4,1,1,4,4,1,1,4,4,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2)
  mycol=c(1,1,4,4,4,1,1,4,1,1,4,4,1,4,1,1,4,1,1,4,4,4,4,1,1,4,4,1,1,4,4,4,1,1,4,4,4,1,1,4,1,1,4,4,1,1,4,4,1,1,4,4,4,4,1,1,4,4,1,4,4,4,4,1,1,4,4,1,1,4,4,1,1,4,4,4,1,1,4,4,1,1,4,4,1,1,4,4,1,1,4,4,1,1,4,4,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2)
  mycol[mycol==1]<- "black"         #1=UCLA parents
  mycol[mycol==1]<- "black"   #1=UCLA adult
  mycol[mycol==4]<- "#32CD32"    #4=UCLA children(limegreen)
  mycol[mycol==4]<- "#32CD32"    #4=UCLA young adult (limegreen)mycol[mycol==2]<- "#3A5FCD"    #2=UCSD-GA:all adult (royalblue3)
mycol[mycol==2]<- "#473C8B"    #2=UCSD-GA:all old adults(slateblue)
 
*'''sex'''
*'''sex'''
mycol=c(1,4,1,1,1,1,4,1,1,4,4,1,4,1,1,4,1,1,4,4,1,1,1,1,4,1,1,1,4,1,4,1,1,4,1,4,1,1,4,1,1,4,1,1,4,1,1,1,1,4,1,4,1,1,1,4,4,1,1,1,1,4,1,1,4,1,1,1,4,1,1,1,4,1,1,1,1,4,1,1,1,4,4,1,1,4,1,1,1,4,1,1,1,4,1,4,2,8,2,2,8,2,2,8,2,2,2,2,8,8,2,8,2,2,2,2,2,8,2,8,2,8,8,8,8,8,8,2,2,2,8,8,2,8,8,8,2,2,2,2,2,2,8,8,8,2,8,8,8,8,2,8,8,2,8,8,2,8,8,8,2,2,8,8,8,8,2,2,2)
mycol=c(1,4,1,1,1,1,4,1,1,4,4,1,4,1,1,4,1,1,4,4,1,1,1,1,4,1,1,1,4,1,4,1,1,4,1,4,1,1,4,1,1,4,1,1,4,1,1,1,1,4,1,4,1,1,1,4,4,1,1,1,1,4,1,1,4,1,1,1,4,1,1,1,4,1,1,1,1,4,1,1,1,4,4,1,1,4,1,1,1,4,1,1,1,4,1,4,2,8,2,2,8,2,2,8,2,2,2,2,8,8,2,8,2,2,2,2,2,8,2,8,2,8,8,8,8,8,8,2,2,2,8,8,2,8,8,8,2,2,2,2,2,2,8,8,8,2,8,8,8,8,2,8,8,2,8,8,2,8,8,8,2,2,8,8,8,8,2,2,2)
  mycol[mycol==1]<- "#000080"       #1=UCLA, male(navy)
  mycol[mycol==1]<- "#000080" #navy=UCLA male
  mycol[mycol==4]<- "#CD1076"       #4=UCLA, female(deeppink)
  mycol[mycol==4]<- "#CD0000" #red=UCLA female
  mycol[mycol==2]<- "#1874CD "     #2=UCSD-GA, male(dodgerblue)
  mycol[mycol==2]<- "#00BFFF" #lightblue=GA male
  mycol[mycol==8]<- "#CD69C9"     #8=UCSD-GA, female(orchid3)
  mycol[mycol==8]<- "#FF69B4" #pink=GA female
 
*'''status'''
*'''status'''
  mycol=c(4,4,1,4,1,4,4,4,4,4,4,4,4,1,4,4,4,4,4,4,1,4,1,4,4,4,1,4,4,4,4,1,4,4,4,4,1,4,4,4,4,4,4,1,4,4,1,4,4,4,1,4,4,1,4,4,4,4,4,4,1,4,4,4,4,1,4,4,4,4,1,4,4,1,4,1,4,4,1,1,4,4,4,4,4,4,1,4,4,4,1,4,4,4,1,4,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2)
  mycol=c(4,4,1,4,1,4,4,4,4,4,4,4,4,1,4,4,4,4,4,4,1,4,1,4,4,4,1,4,4,4,4,1,4,4,4,4,1,4,4,4,4,4,4,1,4,4,1,4,4,4,1,4,4,1,4,4,4,4,4,4,1,4,4,4,4,1,4,4,4,4,1,4,4,1,4,1,4,4,1,1,4,4,4,4,4,4,1,4,4,4,1,4,4,4,1,4,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2)
  mycol[mycol==1]<- "#8B0000"      #1=UCLA, affected (red4)
  mycol[mycol==1]<- "#EE0000"      #1=UCLA, affected (red2)
  mycol[mycol==4]<- "#0000CD"        #4=UCLA, unaffected (mediumblue)
  mycol[mycol==4]<- "#0000CD"        #4=UCLA, unaffected (mediumblue)
  mycol[mycol==2]<- "#7A67EE"    #2=UCSD-GA(slateblue2)
  mycol[mycol==2]<- "#00BFFF"    #2=UCSD-GA unaffected (deepskyblue)
* Directory: /home/nplongth/Noi_scratch/Data_analysis/UCLA_SZ_data_analysis_2012_04_05/UCLA_regression_analysis/2012_05_01_regression/
* I used the files generated from previous analyses. Age, sex and status data with the calculated p-value and q-value were printed out in separated files.


== PCA analysis, p-value <= 0.01 ==
* Print out the CpG sites, p-value <=0.01
awk '{if ($8<=0.01) print $0}' sorted_UCLA_regression_age_qval.txt > required_pval0.01/UCLA_regression_age_pval0.01
awk '{if ($8<=0.01) print $0}' sorted_UCLA_regression_sex_qval.txt > required_pval0.01/UCLA_regression_sex_pval0.01
awk '{if ($8<=0.01) print $0}' sorted_UCLA_regression_status.txt > required_pval0.01/UCLA_regression_status_pval0.01
* Extract methylation value into the matrix
../extract_methyFreqMatrix.pl ../UCLA-GA_methyl_min70_minSTD0.05 UCLA_regression_age_pval0.01 > UCLA_regression_age_pval0.01_MM
../extract_methyFreqMatrix.pl ../UCLA-GA_methyl_min70_minSTD0.05 UCLA_regression_sex_pval0.01 > UCLA_regression_sex_pval0.01_MM
../extract_methyFreqMatrix.pl ../UCLA-GA_methyl_min70_minSTD0.05 UCLA_regression_status_pval0.01 > UCLA_regression_status_pval0.01_MM
* After I run PCA and found the separation of the sample based on status was poor when compare to previous analysis I did with |t-stat| >2.5. One possible is that there were many missing values (NA), so the number of CpGs from regression and analyzed by PCA were very different.
* I reduced the stringency by increasing p-value to 0.015 to get more sites as I got from |t-stat| >2.5 and enough to separate the samples into two groups based on the regression factors. However, this could introduce false positive signal as well.


* Directory: /home/nplongth/Noi_scratch/Data_analysis/UCLA_SZ_data_analysis_2012_04_05/UCLA_regression_analysis/2012_05_01_regression/
== PCA analysis, p-value <= 0.015 ==
* I used the files generated from previous analyses. Age, sex and status data with the calculated p-value and q-value were printed out in separated files.
* Print out the CpG sites, p-value <=0.015
== PCA analysis, required p-value <= 0.01 ==
* Note: this cutoff was performed to repeat the previous analysis that I used cutoff at t-statistics of 2.5.
* Print out the CpG sites with the R square value > 0.1 and p-value <=0.01
  awk '{if ($8<=0.015) print $0}' sorted_UCLA_regression_age_qval.txt > pval0.015/UCLA_regression_age_pval0.015  
  awk '{if ($2>0.1 && $8<=0.01) print $0}' sorted_UCLA_regression_age_qval.txt>UCLA_regression_age_pval0.01
  awk '{if ($8<=0.015) print $0}' sorted_UCLA_regression_sex_qval.txt > pval0.015/UCLA_regression_sex_pval0.015
  1038 UCLA_regression_age_pval0.01
  awk '{if ($8<=0.015) print $0}' sorted_UCLA_regression_status.txt > pval0.015/UCLA_regression_status_pval0.015
  awk '{if ($2>0.1 && $8<=0.01) print $0}' sorted_UCLA_regression_sex_qval.txt > UCLA_regression_sex_pval0.01
678 UCLA_regression_sex_pval0.01
  awk '{if ($2>0.1 && $8<=0.01) print $0}' sorted_UCLA_regression_status.txt > UCLA_regression_status_pval0.01
110 UCLA_regression_status_pval0.01
* Extract methylation value into the matrix
* Extract methylation value into the matrix
  ./extract_methyFreqMatrix.pl ../UCLA-GA_methyl_min70_minSTD0.05 ./UCLA_regression_age_pval0.01 > UCLA_regression_age_pval0.01_MM
  ../extract_methyFreqMatrix.pl ../UCLA-GA_methyl_min70_minSTD0.05 UCLA_regression_age_pval0.015 > UCLA_regression_age_pval0.015_MM
  1038 UCLA_regression_age_pval0.01_MM
  ../extract_methyFreqMatrix.pl ../UCLA-GA_methyl_min70_minSTD0.05 UCLA_regression_sex_pval0.015 > UCLA_regression_sex_pval0.015_MM
./extract_methyFreqMatrix.pl ./UCLA-GA_methyl_min70_minSTD0.05 ./UCLA_regression_sex_pval0.01 > UCLA_regression_sex_pval0.01_MM
  ../extract_methyFreqMatrix.pl ../UCLA-GA_methyl_min70_minSTD0.05 UCLA_regression_status_pval0.015 > UCLA_regression_status_pval0.015_MM
  678 UCLA_regression_sex_pval0.01_MM
* Continued on: http://genome-tech.ucsd.edu/LabNotes/index.php/Noi/NOTES/2012-5-12#Regression_analysis_of_UCLA_SZ_data.2C_applying_LDA_function
./extract_methyFreqMatrix.pl ./UCLA-GA_methyl_min70_minSTD0.05 ./UCLA_regression_status_pval0.01 > UCLA_regression_status_pval0.01_MM
110 UCLA_regression_status_pval0.01_MM

Latest revision as of 04:37, 14 May 2012

Regression analysis of UCLA SZ data, applying LDA function[edit]

  • Continued from: http://genome-tech.ucsd.edu/LabNotes/index.php/Noi/NOTES/2012-4-17
  • I repeated regression analysis again by changing some criteria to select the candidate significant CpG sites, based on FDR or with minimum raw p-value. For status regression analysis, I will not apply multiple test correction since I didn't get any signal. However with minimum t statistic of 2.5 from the previous analysis I could separate out the normal and affected samples. Dr. Zhang suggested that this should be fine. In addition, I will change the color of the control samples in PCA analysis plots. LDA will be used then the number of samples in the two groups classified by the line will be counted.

Color code for PCA plot (some of them have been changed)[edit]

  • Age
mycol=c(1,1,4,4,4,1,1,4,1,1,4,4,1,4,1,1,4,1,1,4,4,4,4,1,1,4,4,1,1,4,4,4,1,1,4,4,4,1,1,4,1,1,4,4,1,1,4,4,1,1,4,4,4,4,1,1,4,4,1,4,4,4,4,1,1,4,4,1,1,4,4,1,1,4,4,4,1,1,4,4,1,1,4,4,1,1,4,4,1,1,4,4,1,1,4,4,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2)
mycol[mycol==1]<- "black"   	#1=UCLA adult
mycol[mycol==4]<- "#32CD32"     #4=UCLA young adult (limegreen)mycol[mycol==2]<- "#3A5FCD"     #2=UCSD-GA:all adult (royalblue3)
  • sex
mycol=c(1,4,1,1,1,1,4,1,1,4,4,1,4,1,1,4,1,1,4,4,1,1,1,1,4,1,1,1,4,1,4,1,1,4,1,4,1,1,4,1,1,4,1,1,4,1,1,1,1,4,1,4,1,1,1,4,4,1,1,1,1,4,1,1,4,1,1,1,4,1,1,1,4,1,1,1,1,4,1,1,1,4,4,1,1,4,1,1,1,4,1,1,1,4,1,4,2,8,2,2,8,2,2,8,2,2,2,2,8,8,2,8,2,2,2,2,2,8,2,8,2,8,8,8,8,8,8,2,2,2,8,8,2,8,8,8,2,2,2,2,2,2,8,8,8,2,8,8,8,8,2,8,8,2,8,8,2,8,8,8,2,2,8,8,8,8,2,2,2)
mycol[mycol==1]<- "#000080" 	#navy=UCLA male
mycol[mycol==4]<- "#CD0000"	#red=UCLA female
mycol[mycol==2]<- "#00BFFF"	#lightblue=GA male
mycol[mycol==8]<- "#FF69B4"  	#pink=GA female
  • status
mycol=c(4,4,1,4,1,4,4,4,4,4,4,4,4,1,4,4,4,4,4,4,1,4,1,4,4,4,1,4,4,4,4,1,4,4,4,4,1,4,4,4,4,4,4,1,4,4,1,4,4,4,1,4,4,1,4,4,4,4,4,4,1,4,4,4,4,1,4,4,4,4,1,4,4,1,4,1,4,4,1,1,4,4,4,4,4,4,1,4,4,4,1,4,4,4,1,4,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2)
mycol[mycol==1]<- "#EE0000"       #1=UCLA, affected (red2)
mycol[mycol==4]<- "#0000CD"        #4=UCLA, unaffected (mediumblue)
mycol[mycol==2]<- "#00BFFF"     #2=UCSD-GA unaffected (deepskyblue)
  • Directory: /home/nplongth/Noi_scratch/Data_analysis/UCLA_SZ_data_analysis_2012_04_05/UCLA_regression_analysis/2012_05_01_regression/
  • I used the files generated from previous analyses. Age, sex and status data with the calculated p-value and q-value were printed out in separated files.

PCA analysis, p-value <= 0.01[edit]

  • Print out the CpG sites, p-value <=0.01
awk '{if ($8<=0.01) print $0}' sorted_UCLA_regression_age_qval.txt > required_pval0.01/UCLA_regression_age_pval0.01
awk '{if ($8<=0.01) print $0}' sorted_UCLA_regression_sex_qval.txt > required_pval0.01/UCLA_regression_sex_pval0.01
awk '{if ($8<=0.01) print $0}' sorted_UCLA_regression_status.txt > required_pval0.01/UCLA_regression_status_pval0.01
  • Extract methylation value into the matrix
../extract_methyFreqMatrix.pl ../UCLA-GA_methyl_min70_minSTD0.05 UCLA_regression_age_pval0.01 > UCLA_regression_age_pval0.01_MM 
../extract_methyFreqMatrix.pl ../UCLA-GA_methyl_min70_minSTD0.05 UCLA_regression_sex_pval0.01 > UCLA_regression_sex_pval0.01_MM
../extract_methyFreqMatrix.pl ../UCLA-GA_methyl_min70_minSTD0.05 UCLA_regression_status_pval0.01 > UCLA_regression_status_pval0.01_MM
  • After I run PCA and found the separation of the sample based on status was poor when compare to previous analysis I did with |t-stat| >2.5. One possible is that there were many missing values (NA), so the number of CpGs from regression and analyzed by PCA were very different.
  • I reduced the stringency by increasing p-value to 0.015 to get more sites as I got from |t-stat| >2.5 and enough to separate the samples into two groups based on the regression factors. However, this could introduce false positive signal as well.

PCA analysis, p-value <= 0.015[edit]

  • Print out the CpG sites, p-value <=0.015
  • Note: this cutoff was performed to repeat the previous analysis that I used cutoff at t-statistics of 2.5.
awk '{if ($8<=0.015) print $0}' sorted_UCLA_regression_age_qval.txt > pval0.015/UCLA_regression_age_pval0.015  
awk '{if ($8<=0.015) print $0}' sorted_UCLA_regression_sex_qval.txt > pval0.015/UCLA_regression_sex_pval0.015
awk '{if ($8<=0.015) print $0}' sorted_UCLA_regression_status.txt > pval0.015/UCLA_regression_status_pval0.015
  • Extract methylation value into the matrix
../extract_methyFreqMatrix.pl ../UCLA-GA_methyl_min70_minSTD0.05 UCLA_regression_age_pval0.015 > UCLA_regression_age_pval0.015_MM 
../extract_methyFreqMatrix.pl ../UCLA-GA_methyl_min70_minSTD0.05 UCLA_regression_sex_pval0.015 > UCLA_regression_sex_pval0.015_MM
../extract_methyFreqMatrix.pl ../UCLA-GA_methyl_min70_minSTD0.05 UCLA_regression_status_pval0.015 > UCLA_regression_status_pval0.015_MM