Kun:LabNotes/SingleCellExpr/2014-6-5

From ZhangLabWiki
Jump to navigation Jump to search

Preparing the 2014Jun data set[edit]

Data source[edit]

  • The data came from 18 C1 experiments.
    • 20131219_C1_hNuc-20140222_Expt123Hiseq(oligo dT only)
    • 20140123_C1_hNuc-20140210_Expt119HiSeq
    • 20140205_C1_hNuc-20140222_Expt123Hiseq
    • 20140212_C1_hNuc-20140222_Expt123Hiseq
    • 20140213_C1_hNuc-20140222_Expt123Hiseq
    • 20140218_C1_hNuc-20140317_Expt128HiSeq
    • 20140220_C1_hNuc-20140317_Expt128HiSeq
    • 20140225_C1_hNuc-20140317_Expt128HiSeq
    • 20140226_C1_hNuc-20140317_Expt128HiSeq
    • 20140227_C1_hNuc-20140414_Expt131HiSeq
    • 20140313_C1_hNuc-20140414_Expt131HiSeq
    • 20140317_C1_hNuc-20140414_Expt131HiSeq
    • 20140304_C1_hNuc-20140521_Expt138HiSeq
    • 20140318_C1_hNuc-20140521_Expt138HiSeq
    • 20140320_C1_hNuc-20140521_Expt138HiSeq
    • 20140321_C1_hNuc-20140521_Expt138HiSeq
 cd /home/kunzhang/RNAseq/SCAP/expr_matrix
 ../scripts/merge_expr_matrix.pl 20131219_C1_hNuc-20140222_TPM_matrix_name_corrected.txt 20140210_96Samples_hNuc_Expt119HiSeq_C1_STAR_0123_TPM_matrix_name_corrected.txt 20140205_C1_hNuc-20140222_TPM_matrix_name_corrected.txt 20140212_C1_hNuc-20140222_TPM_matrix_name_corrected.txt 20140213_C1_hNuc-20140222_TPM_matrix_name_corrected.txt 20140218_C1_hNuc_expr_matrix.txt 20140220_C1_hNuc_expr_matrix.txt 20140225_C1_hNuc_expr_matrix.txt 20140226_C1_hNuc_expr_matrix.txt 20140227_C1_hNuc_expr_matrix.txt 20140311_C1_hNuc_expr_matrix.txt 20140313_C1_hNuc_expr_matrix.txt 20140317_C1_hNuc_expr_matrix.txt  20140304_C1_hNuc_expr_matrix.txt 20140318_C1_hNuc_expr_matrix.txt 20140320_C1_hNuc_expr_matrix.txt 20140321_C1_hNuc_expr_matrix.txt > 2014_1st_Quater_18_expr_matrix.txt
  cd ../summary_stats
  ../scripts/get_excluded_sample_id.pl 20131219_C1_hNuc-20140222_mappnig_stats_name_corrected.txt 20140123_C1_hNuc-20140210_mappnig_stats_name_corrected.txt 20140205_C1_hNuc-20140222_mapping_stats_name_corrected.txt 20140212_C1_hNuc-20140222_mapping_stats_name_corrected.txt 20140212_C1_hNuc-20140222_mapping_stats.txt 20140213_C1_hNuc-20140222_mapping_stats_name_corrected.txt 20140218_C1_mapping_stat.txt 20140220_C1_mapping_stat.txt 20140225_C1_mapping_stat.txt 20140226_C1_mapping_stat.txt 20140227_C1_hNuc_mapping_stats.txt 20140311_C1_hNuc_mapping_stats.txt 20140313_C1_hNuc_mapping_stats.txt 20140317_C1_hNuc_mapping_stats.txt 20140304_C1_mapping_stat.txt 20140318_C1_mapping_stat.txt  20140320_C1_mapping_stat.txt 20140321_C1_mapping_stat.txt > 2014_1st_Quater_18_expr_matrix_1N_excluded_ids.txt
 ../scripts/filterExprMatrix.pl 2014_1st_Quater_18_expr_matrix.txt ../summary_stats/2014_1st_Quater_18_expr_matrix_1N_excluded_ids.txt > 2014_1st_Quater_18_1N_expr_matrix_filtered.txt

A total of 728 data sets (10323 genes) were obtained after the filtering (at least 1M total reads; mapping rate >0.4; # ERCC detected > 24; % ERCC reads < 0.3; <30% missing values per gene).

Iterative over-dispersion analysis[edit]

Round 1[edit]

Identify over-dispersed genes[edit]
 #Find over-dispersed genes for the entire data set, using a Z cutoff of 2.
 ../scripts/exprMatrix2overDispersedGenes.pl 2014_1st_Quater_18_1N_expr_matrix_filtered.txt 2 > 2014_1st_Quater_18_1N_Z2_overDispersed_genes.txt
 #Extract the gene expression matrix for the over-dispersed genes.
 ../scripts/extract_gene_set_ENSG.pl  2014_1st_Quater_18_1N_Z2_overDispersed_genes.txt 2014_1st_Quater_18_1N_expr_matrix_filtered.txt > 2014_1st_Quater_18_1N_Z2_overDispersed_expr_matrix.txt
Hierarchical clustering of samples based on these genes[edit]
 #Visualize the sample clustering.
 library(gplots)
 x=read.table("2014_1st_Quater_18_1N_Z2_overDispersed_expr_matrix.txt",header=TRUE,row.names=1);
 x.cor = cor(log10(x+1),use="pairwise.complete.obs",method="pearson")
 heatmap.2(as.matrix(x.cor), col=bluered(128), scale="none", cexCol=0.1, cexRow=0.1 ,key=T, symkey=F,density.info="histogram",trace="none",dendrogram="both",Rowv=TRUE,Colv=TRUE)
 File:2014 1st Quater 18 over-dispersed-Z2 sample clustering.png
 #Cut the dendrogram, and extract the sample names in each cluster
 hc.rows<- hclust(dist(x.cor))
 plot(hc.rows,cex=0.1) 
 ct<- cutree(hc.rows, h=11) # the threshold (11) was selected by visually inspect the dendrogram.
 rect.hclust(hc.rows, h=11) # draw red rectangles to mark the subgroups
 write.table(ct, file="2014_1st_Quater_18_1N_Z2_overDispersed_cutree_3_clusters.txt", sep = "\t", row.names=TRUE, col.names=TRUE)
 File:2014 1st Quater 18 over-dispersed-Z2 dendrogram cutree h11.png
 #Generate gene expression matrix for each cluster
 ../scripts/extractSampleClusterExprMatrix.pl 2014_1st_Quater_18_expr_matrix.txt 2014_1st_Quater_18_1N_Z2_overDispersed_cutree_3_clusters.txt 1 > 2014_1st_Quater_18_expr_matrix_cutree-h11_c1.txt
 ../scripts/extractSampleClusterExprMatrix.pl 2014_1st_Quater_18_expr_matrix.txt 2014_1st_Quater_18_1N_Z2_overDispersed_cutree_3_clusters.txt 2 > 2014_1st_Quater_18_expr_matrix_cutree-h11_c2.txt &
 ../scripts/extractSampleClusterExprMatrix.pl 2014_1st_Quater_18_expr_matrix.txt 2014_1st_Quater_18_1N_Z2_overDispersed_cutree_3_clusters.txt 3 > 2014_1st_Quater_18_expr_matrix_cutree-h11_c3.txt &
 #Check whether samples in these cluster have specific neural transmitters.
 ./make_gene_set_heatmap_Hs.pl neural_transmitter_metabolism.txt ../expr_matrix/2014_1st_Quater_18_expr_matrix_cutree-h11_c1.txt
 ./make_gene_set_heatmap_Hs.pl neural_transmitter_metabolism.txt ../expr_matrix/2014_1st_Quater_18_expr_matrix_cutree-h11_c2.txt
 ./make_gene_set_heatmap_Hs.pl neural_transmitter_metabolism.txt ../expr_matrix/2014_1st_Quater_18_expr_matrix_cutree-h11_c3.txt
                            Cluster #1                                               Cluster #2                                               Cluster #3
 File:2014 1st Quater 18 expr matrix cutree-h11 c1 neural transmitter metabolism expr heatmap.pngFile:2014 1st Quater 18 expr matrix cutree-h11 c2 neural transmitter metabolism expr heatmap.pngFile:2014 1st Quater 18 expr matrix cutree-h11 c3 neural transmitter metabolism expr heatmap.png
  • Clear Cluster #1 are mostly GAD1-/GAD2- neurons, and Cluster #3 are mostly GAD1+/GAD2+ neurons. Cluster #2 is a mixture of both. While overall there is a separate between excitatory versus inhibitory neurons, the clustering is not exactly based on such a difference. There is probably another feature that determines the clustering.
K-mean partitioning based on the over-dispersed genes[edit]
 library(cluster) 
 library(fpc)
 x=read.table("2014_1st_Quater_18_1N_Z2_overDispersed_expr_matrix.txt",header=TRUE,row.names=1)
 x[is.na(x)] <- 0
 y=log10(x+1)
 tys = scale(t(y))
 k_5=kmeans(tys, 5)
 k_4=kmeans(tys, 4)
 k_3=kmeans(tys, 3)
 k_2=kmeans(tys, 2)
 plotcluster(tys,k_2$cluster)
 plotcluster(tys,k_3$cluster)
 plotcluster(tys,k_4$cluster)
 plotcluster(tys,k_5$cluster)
 write.table(k_3$cluster, file="2014_1st_Quater_18_1N_Z2_overDispersed_k-mean_3_clusters.txt", sep = "\t", row.names=TRUE, col.names=TRUE)
 write.table(k_2$cluster, file="2014_1st_Quater_18_1N_Z2_overDispersed_k-mean_2_clusters.txt", sep = "\t", row.names=TRUE, col.names=TRUE)
 File:2014 1st Quater 18 over-dispersed-Z2 k2 clusters.pngFile:2014 1st Quater 18 over-dispersed-Z2 k3 clusters.png
 File:2014 1st Quater 18 over-dispersed-Z2 k4 clusters.pngFile:2014 1st Quater 18 over-dispersed-Z2 k5 clusters.png
 #Generate gene expression matrix for each cluster
 ../scripts/extractSampleClusterExprMatrix.pl 2014_1st_Quater_18_expr_matrix.txt 2014_1st_Quater_18_1N_Z2_overDispersed_k-mean_2_clusters.txt 1 > 2014_1st_Quater_18_expr_matrix_k-mean2_c1.txt &
 ../scripts/extractSampleClusterExprMatrix.pl 2014_1st_Quater_18_expr_matrix.txt 2014_1st_Quater_18_1N_Z2_overDispersed_k-mean_2_clusters.txt 2 > 2014_1st_Quater_18_expr_matrix_k-mean2_c2.txt &
 ../scripts/extractSampleClusterExprMatrix.pl 2014_1st_Quater_18_expr_matrix.txt 2014_1st_Quater_18_1N_Z2_overDispersed_k-mean_3_clusters.txt 1 > 2014_1st_Quater_18_expr_matrix_k-mean3_c1.txt &
 ../scripts/extractSampleClusterExprMatrix.pl 2014_1st_Quater_18_expr_matrix.txt 2014_1st_Quater_18_1N_Z2_overDispersed_k-mean_3_clusters.txt 2 > 2014_1st_Quater_18_expr_matrix_k-mean3_c2.txt &
 ../scripts/extractSampleClusterExprMatrix.pl 2014_1st_Quater_18_expr_matrix.txt 2014_1st_Quater_18_1N_Z2_overDispersed_k-mean_3_clusters.txt 3 > 2014_1st_Quater_18_expr_matrix_k-mean3_c3.txt &
  ./make_gene_set_heatmap_Hs.pl neural_transmitter_metabolism.txt ../expr_matrix/2014_1st_Quater_18_expr_matrix_k-mean2_c1.txt
  ./make_gene_set_heatmap_Hs.pl neural_transmitter_metabolism.txt ../expr_matrix/2014_1st_Quater_18_expr_matrix_k-mean2_c2.txt
  ./make_gene_set_heatmap_Hs.pl neural_transmitter_metabolism.txt ../expr_matrix/2014_1st_Quater_18_expr_matrix_k-mean3_c1.txt
  ./make_gene_set_heatmap_Hs.pl neural_transmitter_metabolism.txt ../expr_matrix/2014_1st_Quater_18_expr_matrix_k-mean3_c2.txt
  ./make_gene_set_heatmap_Hs.pl neural_transmitter_metabolism.txt ../expr_matrix/2014_1st_Quater_18_expr_matrix_k-mean3_c3.txt
                         Cluster #1 (k=2)                                               Cluster #2  (k=2)                      
  File:2014 1st Quater 18 expr matrix k-mean2 c1 neural transmitter metabolism expr heatmap.pngFile:2014 1st Quater 18 expr matrix k-mean2 c2 neural transmitter metabolism expr heatmap.png
                        Cluster #1(k=3)                                          Cluster #2(k=3)                                         Cluster #3(k=3)
  File:2014 1st Quater 18 expr matrix k-mean3 c1 neural transmitter metabolism expr heatmap.pngFile:2014 1st Quater 18 expr matrix k-mean3 c2 neural transmitter metabolism expr heatmap.pngFile:2014 1st Quater 18 expr matrix k-mean3 c3 neural transmitter metabolism expr heatmap.png
  • The sample clusters obtained by cutting the dendrogram are not entirely consistent with the k-mean generated clusters. For example, Cluster #1 in the k-3 clustering result include all #3, plus some #2 samples in the dendrogram. Similarly, GAD1/GAD2 seem to contribute to the separation, but there is also something else.
Identify over-dispersed genes using a less stringent threshold[edit]
  • When looked into the matrix of over-dispersed genes, I found that only 95 genes were in the list. Perhaps that's too few and not sufficient for robust sample clustering. Therefore I repeated the whole analysis using a Z cutoff of 1, which allowed me to identify 564 genes.
 ../scripts/exprMatrix2overDispersedGenes.pl 2014_1st_Quater_18_1N_expr_matrix_filtered.txt 1 > 2014_1st_Quater_18_1N_Z1_overDispersed_genes.txt
 #Extract the gene expression matrix for the over-dispersed genes.
 ../scripts/extract_gene_set_ENSG.pl  2014_1st_Quater_18_1N_Z1_overDispersed_genes.txt 2014_1st_Quater_18_1N_expr_matrix_filtered.txt > 2014_1st_Quater_18_1N_Z1_overDispersed_expr_matrix.txt
 x=read.table("2014_1st_Quater_18_1N_Z1_overDispersed_expr_matrix.txt",header=TRUE,row.names=1);
 x.cor = cor(log10(x+1),use="pairwise.complete.obs",method="pearson")
 hm<-heatmap.2(as.matrix(x.cor), col=bluered(128), scale="none", cexCol=0.1, cexRow=0.1 ,key=T, symkey=F,density.info="histogram",trace="none",dendrogram="both",Rowv=TRUE,Colv=TRUE)
 hc.rows<- hclust(dist(x.cor))
 plot(hc.rows,cex=0.1) 
 ct<- cutree(hc.rows, h=11) # 
 rect.hclust(hc.rows, h=11) # draw red rectangles to mark the subgroups
 write.table(ct, file="2014_1st_Quater_18_1N_Z1_overDispersed_cutree_2_clusters.txt", sep = "\t", row.names=TRUE, col.names=TRUE)
 x=read.table("2014_1st_Quater_18_1N_Z1_overDispersed_expr_matrix.txt",header=TRUE,row.names=1)
 x[is.na(x)] <- 0
 y=log10(x+1)
 tys = scale(t(y))
 k_5=kmeans(tys, 5)
 k_4=kmeans(tys, 4)
 k_3=kmeans(tys, 3)
 k_2=kmeans(tys, 2)
 plotcluster(tys,k_2$cluster)
 plotcluster(tys,k_3$cluster)
 plotcluster(tys,k_4$cluster)
 plotcluster(tys,k_5$cluster)
 write.table(k_3$cluster, file="2014_1st_Quater_18_1N_Z1_overDispersed_k-mean_3_clusters.txt", sep = "\t", row.names=TRUE, col.names=TRUE)
 write.table(k_2$cluster, file="2014_1st_Quater_18_1N_Z1_overDispersed_k-mean_2_clusters.txt", sep = "\t", row.names=TRUE, col.names=TRUE)
 
 ../scripts/extractSampleClusterExprMatrix.pl 2014_1st_Quater_18_expr_matrix.txt 2014_1st_Quater_18_1N_Z1_overDispersed_cutree_2_clusters.txt 1 > 2014_1st_Quater_18_expr_matrix_Z1_cutree-h11_c1.txt
 ../scripts/extractSampleClusterExprMatrix.pl 2014_1st_Quater_18_expr_matrix.txt 2014_1st_Quater_18_1N_Z1_overDispersed_cutree_2_clusters.txt 2 > 2014_1st_Quater_18_expr_matrix_Z1_cutree-h11_c2.txt 
 ../scripts/extractSampleClusterExprMatrix.pl 2014_1st_Quater_18_expr_matrix.txt 2014_1st_Quater_18_1N_Z1_overDispersed_k-mean_2_clusters.txt 1 > 2014_1st_Quater_18_expr_matrix_Z1_k-mean2_c1.txt 
 ../scripts/extractSampleClusterExprMatrix.pl 2014_1st_Quater_18_expr_matrix.txt 2014_1st_Quater_18_1N_Z1_overDispersed_k-mean_2_clusters.txt 2 > 2014_1st_Quater_18_expr_matrix_Z1_k-mean2_c2.txt 
 ../scripts/extractSampleClusterExprMatrix.pl 2014_1st_Quater_18_expr_matrix.txt 2014_1st_Quater_18_1N_Z1_overDispersed_k-mean_3_clusters.txt 1 > 2014_1st_Quater_18_expr_matrix_Z1_k-mean3_c1.txt 
 ../scripts/extractSampleClusterExprMatrix.pl 2014_1st_Quater_18_expr_matrix.txt 2014_1st_Quater_18_1N_Z1_overDispersed_k-mean_3_clusters.txt 2 > 2014_1st_Quater_18_expr_matrix_Z1_k-mean3_c2.txt 
 ../scripts/extractSampleClusterExprMatrix.pl 2014_1st_Quater_18_expr_matrix.txt 2014_1st_Quater_18_1N_Z1_overDispersed_k-mean_3_clusters.txt 3 > 2014_1st_Quater_18_expr_matrix_Z1_k-mean3_c3.txt 
  File:2014 1st Quater 18 over-dispersed-Z1 sample clustering.png
  File:2014 1st Quater 18 over-dispersed-Z1 dendrogram cutree h11.png
 File:2014 1st Quater 18 over-dispersed-Z1 k2 clusters.pngFile:2014 1st Quater 18 over-dispersed-Z1 k3 clusters.png
 File:2014 1st Quater 18 over-dispersed-Z1 k4 clusters.pngFile:2014 1st Quater 18 over-dispersed-Z1 k5 clusters.png
                        Cluster #1(k=2)                                          Cluster #2(k=2) 
  File:2014 1st Quater 18 expr matrix Z1 k-mean2 c1 neural transmitter metabolism expr heatmap.pngFile:2014 1st Quater 18 expr matrix Z1 k-mean2 c2 neural transmitter metabolism expr heatmap.png
                        Cluster #1(k=3)                                          Cluster #2(k=3)                                         Cluster #3(k=3)
  File:2014 1st Quater 18 expr matrix Z1 k-mean3 c1 neural transmitter metabolism expr heatmap.pngFile:2014 1st Quater 18 expr matrix Z1 k-mean3 c2 neural transmitter metabolism expr heatmap.pngFile:2014 1st Quater 18 expr matrix Z1 k-mean3 c3 neural transmitter metabolism expr heatmap.png
  • Clearly the separate is much cleaner with 565 genes that have Z-score above 1. However, GAD1/GAD2 positive neurons are still not fully separated. For example, with k=2, Cluster #1 still have a fraction of GAD1/GAD2 positive neurons. Such neurons seem to have dual identities. There could be true neurons like these, or perhaps these data sets came from two nuclei when they overlapped on each other and looked like one nucleus under the microscope. We might need to use RNA FISH to distinguish these two hypotheses.
Identify differentially expressed genes between clusters[edit]
  • I took the over-dispersed genes for all samples (2014_1st_Quater_18_1N_Z1_overDispersed_expr_matrix.txt), and assigned each sample to one of the two clusters based on the k-mean analysis (2014_1st_Quater_18_1N_Z1_overDispersed_k-mean_2_clusters.txt).
  • Then I ran SAM to identify differentially expressed genes. The majority of the genes were differentially expressed, which is not surprising since these genes were pre-selected based on the variability, and the two clusters were well separated.
  Media:2014_1st_Quater_18_1N_Z1_overDispersed_expr_matrix_for_SAM.xlsx
  File:2014 1st Quater 18 expr matrix Z1 k-mean2 two-clusters-SAM-plot.png
  Top ten up/down genes:
Gene ID Fold Change q-value(%) HUGO Gene ID Fold Change q-value(%) HUGO
ENSG00000157103.6 14770220922 0 SLC6A1 ENSG00000135426.9 2.56E-10 0 TESPA1
ENSG00000136750.7 9609687782 0 GAD2 ENSG00000179456.9 6.60E-10 0 ZBTB18
ENSG00000128683.8 9106160761 0 GAD1 ENSG00000198963.5 9.72E-10 0 RORB
ENSG00000077044.5 2678606715 0 DGKD ENSG00000156140.4 1.21E-09 0 ADAMTS3
ENSG00000129244.4 1642874016 0 ATP1B2 ENSG00000120738.7 1.32E-09 0 EGR1
ENSG00000164512.12 1351632660 0 ANKRD55 ENSG00000060718.14 1.43E-09 0 COL11A1
ENSG00000175352.6 1338510457 0 NRIP3 ENSG00000135919.8 1.52E-09 0 SERPINE2
ENSG00000103460.12 1303571887 0 TOX3 ENSG00000167614.8 1.56E-09 0 TTYH1
ENSG00000017427.11 938250314.6 0 IGF1 ENSG00000169507.5 1.96E-09 0 SLC38A11
ENSG00000127328.17 863574011.2 0 RAB3IP ENSG00000135636.8 2.18E-09 0 DYSF
  • I then manually annotated some of the genes. Note that I used /home/kunzhang/RNAseq/SCAP/scripts/ENSG2HUGO.pl to convert a list of ENSG IDs into HUGO IDs. The top most up-regulated genes in Cluster #2 are mostly GABAergic neuron related genes, which is not surprising. The ones that are up-regulated in Cluster #1 have more diverse functions. Many are glycoproteins, related to calcium binding or cell adhesion.
  • I did found a few interesting transcriptional regulators, such as ZBTB18/SATB2 for Cluster #1 and TOX3 for Cluster #2.
  • Then I decided to focus the analysis only on known transcription factors.
    • Extract the gene expression matrix for all known TFs (based on the list in nrg2538).
 ./make_gene_set_heatmap_Hs.pl nrg2538_TF_genes.txt ../expr_matrix/2014_1st_Quater_18_1N_expr_matrix_filtered.txt
    • Renamed nrg2538_TF_genes_expr_matrix.txt to 2014_1st_Quater_18_1N_nrg2538_TF_genes_expr_matrix.txt, then assigned each sample to one of the two clusters(2014_1st_Quater_18_1N_Z1_overDispersed_k-mean_2_clusters.txt).
    • Run SAM analysis.
  Media:2014_1st_Quater_18_1N_nrg2538_TF_genes_expr_matrix_for_SAM.xlsx
  File:2014 1st Quater 18 expr matrix nrg2538 k-mean2 two-clusters-SAM-plot.png
    • Top ten differentially expressed genes:
Up in Cluster 1 Up in Cluser 2
Gene ID Fold Change q-value(%) Gene ID Fold Change q-value(%)
RORB 9.85E-10 0 TOX3 1.07E+09 0
EGR1 1.31E-09 0 RXRA 2.77E+08 0
ZBTB7C 2.75E-09 0 ZNF641 9.95E+07 0.663375427
HOPX 2.92E-09 0 PPARA 8.41E+07 0
NEUROD2 3.60E-09 0 ETS1 7.65E+07 0
STAT6 3.86E-09 0 BACH1 2.60E+01 0
PLXND1 4.18E-09 0 ZNF385D 1.89E+01 0
ONECUT2 4.50E-09 0 SOX6 7.97E+00 0
LHX2 4.59E-09 0 TOX2 7.29E+00 0
MKX 5.16E-09 0 ZNF536 6.96E+00 0

Round 2[edit]

K-mean cluster 1 (mostly GAD1-/GAD2- neurons)[edit]
../scripts/exprMatrix2overDispersedGenes.pl 2014_1st_Quater_18_expr_matrix_Z1_k-mean2_c1.txt 1 > 2014_1st_Quater_18_k-mean2_c1_Z1_overDispersed_genes.txt
../scripts/extract_gene_set_ENSG.pl  2014_1st_Quater_18_k-mean2_c1_Z1_overDispersed_genes.txt 2014_1st_Quater_18_1N_expr_matrix_filtered.txt > 2014_1st_Quater_18_k-mean2_c1_Z1_overDispersed_genes_expr_matrix.txt
library(gplots)
x=read.table("2014_1st_Quater_18_k-mean2_c1_Z1_overDispersed_genes_expr_matrix.txt",header=TRUE,row.names=1);
x.cor = cor(log10(x+1),use="pairwise.complete.obs",method="pearson")
heatmap.2(as.matrix(x.cor), col=bluered(128), scale="none", cexCol=0.1, cexRow=0.1 ,key=T, symkey=F,density.info="histogram",trace="none",dendrogram="both",Rowv=TRUE,Colv=TRUE)
File:2014 1st Quater 18 k-mean2 c1 Z1 overDispersed genes sample clustering.png 
hc.rows<- hclust(dist(x.cor))
plot(hc.rows,cex=0.1) 
ct<- cutree(hc.rows, h=8) # 
rect.hclust(hc.rows, h=8) # draw red rectangles to mark the subgroups
write.table(ct, file="2014_1st_Quater_18_k-mean2_c1_Z1_overDispersed_genes_cutree_2_clusters.txt", sep = "\t", row.names=TRUE, col.names=TRUE)
File:2014 1st Quater 18 k-mean2 c1 Z1 overDispersed genes sample clustering cutree2.png

library(cluster) 
library(fpc)
x[is.na(x)] <- 0
y=log10(x+1)
tys = scale(t(y))
k_5=kmeans(tys, 5)
k_4=kmeans(tys, 4)
k_3=kmeans(tys, 3)
k_2=kmeans(tys, 2)
plotcluster(tys,k_2$cluster)
plotcluster(tys,k_3$cluster)
plotcluster(tys,k_4$cluster)
plotcluster(tys,k_5$cluster)
write.table(k_3$cluster, file="2014_1st_Quater_18_k-mean2_c1_Z1_overDispersed_genes_k-mean_3_clusters.txt", sep = "\t", row.names=TRUE, col.names=TRUE)
write.table(k_2$cluster, file="2014_1st_Quater_18_k-mean2_c1_Z1_overDispersed_genes_k-mean_2_clusters.txt", sep = "\t", row.names=TRUE, col.names=TRUE)
 File:2014 1st Quater 18 k-mean2 c1 Z1 overDispersed genes k2 clusters.pngFile:2014 1st Quater 18 k-mean2 c1 Z1 overDispersed genes k3 clusters.png
 File:2014 1st Quater 18 k-mean2 c1 Z1 overDispersed genes k4 clusters.pngFile:2014 1st Quater 18 k-mean2 c1 Z1 overDispersed genes k5 clusters.png
  • It appeared that these neurons can be divided into four cleanly separated clusters. The next questions would be to find out the identity or meaning of these clusters.
Finding genes up-regulated in the four sub-groups of K-mean cluster 1[edit]
  • For this I took the over-dispersed genes in all k-mean (k=2) cluster 1 samples from Round 1 (2014_1st_Quater_18_k-mean2_c1_Z1_overDispersed_genes_expr_matrix.txt), label the samples with four groups based on the Round 2 k-mean clustering results (2014_1st_Quater_18_k-mean2_c1_Z1_overDispersed_genes_k-mean_4_clusters), and run the SAM analysis under the multi-class model. Obviously most of the genes (365/486) are differentially expressed in one or more groups. I sorted these genes based on the "contrast" for each of the four groups, then selected the top 10 genes that have the highest contrast for each group (note that I skipped a few that have no annotations, such as RP11-123O10.4).
 Media:2014_1st_Quater_18_k-mean2_c1_Z1_overDispersed_genes_expr_matrix_for_SAM_4-clusters.xlsx
 Cluster c1.1
HUGO contrast-1 contrast-2 contrast-3 contrast-4
KCNIP3 11.16739647 -4.478728361 -2.350967986 -8.986883423
PTPRK 10.01585583 -8.448597715 3.281430094 -10.8506507
EGR1 9.747299154 -4.684003738 -1.824346768 -7.36895931
LINGO1 9.58144463 -4.624228762 -0.40094134 -9.099549439
HECW1 9.394975421 -5.855567107 -1.766355741 -5.738812994
SEMA4D 8.81652661 -2.666120885 -2.904782293 -6.55968048
NGEF 8.802471142 -4.446578057 1.310832307 -10.42218722
ZNF804B 8.5895529 -6.751014558 -0.494902477 -5.345801831
RORB 8.54586603 -3.975389056 -2.334804946 -5.602362517
CPNE4 8.480061688 -3.478772801 -1.500439455 -7.129478032
 Cluster c1.2
HUGO contrast-1 contrast-2 contrast-3 contrast-4
EYA4 -5.321417894 12.84154833 -6.951889857 4.354034676
PID1 -4.831615771 12.56961797 -5.8966099 2.475220629
PHKB -4.678402329 12.53723223 -6.323235779 2.853976955
SEC14L1 -4.063798258 12.32036809 -6.629922109 2.568378636
PDE3A -3.022438352 12.1453072 -9.026753944 4.419686124
HFM1 -4.719507944 12.04260411 -5.660492381 2.521496057
PPEF2 -3.137958388 12.00043463 -7.707046659 2.960175498
SLCO5A1 -3.904449914 11.97087414 -6.695142422 2.771008518
GRIP1 -3.072843559 8.984872116 -12.26343691 12.04528531
FSTL5 -0.971506882 8.634044019 -7.959248851 3.462288967
 Cluster c1.3
HUGO contrast-1 contrast-2 contrast-3 contrast-4
XIST -7.68079175 -2.540266439 12.60231218 -2.948850821
SYN3 4.509683673 -7.724417995 8.033069861 -9.762531396
STXBP5L 6.297609932 -8.539816682 7.94343846 -11.4885019
CBLN2 7.536329826 -9.613307235 7.650603365 -11.86021
CA10 7.632543482 -10.16386485 7.578192948 -11.35073153
FAM19A1 7.740036874 -9.88390345 7.393307367 -11.54462373
TESPA1 8.398699091 -10.48531557 7.16022654 -11.60713383
DGKB 5.667978001 -6.42129736 7.073847338 -11.51450378
ANKDD1A -3.551719544 -0.094358135 6.717202522 -3.652511861
SLIT3 4.749934938 -6.512804085 6.654470337 -9.486357108
  Cluster c1.4
HUGO contrast-1 contrast-2 contrast-3 contrast-4
ERBB4 -5.47746011 6.466422889 -8.491198974 13.09425661
ZNF385D -5.612887639 6.706045987 -8.288955267 12.78215485
GAD1 -3.992672286 7.380574545 -10.18753447 12.2404974
GRIP1 -3.072843559 8.984872116 -12.26343691 12.04528531
GAD2 -4.399414548 6.847386992 -9.173866991 12.02002257
KLHL5 -2.873803987 3.718134302 -8.032442931 11.36189782
SLC6A1 -3.64699617 8.167746887 -10.4614637 11.29912374
QKI -3.926567386 5.118752239 -7.596491468 10.93301198
SAMD5 -1.299998198 2.202125809 -8.017337836 10.52149593
TENM3 -1.967305855 1.641470543 -6.793668393 10.43483179
  • Now I combined all the "top-ten" differentially expressed genes that I found between Cluster 1/2 and four sub-clusters in 1, and compiled a "cluster-specific gene list". With this list, I made a heat map for the expression levels of these genes across all samples.
  File:2014 1st Quater 18 1N cluster specific genes expr heatmap.png

Did I miss any gene by over-filtering[edit]

  • In the previous analyses, I filtered the raw expression matrix by removing any gene that has >30% missing values. Was that too stringent? What if certain genes express exclusive in a sub-group of neurons?
  • To answer this question, I repeated the filtering by keeping genes with as high as 70% missing values.
 ../scripts/filterExprMatrix.pl 2014_1st_Quater_18_expr_matrix.txt ../summary_stats/2014_1st_Quater_18_expr_matrix_1N_excluded_ids.txt > 2014_1st_Quater_18_1N_expr_matrix_filtered_70pct.txt &
 ../scripts/exprMatrix2overDispersedGenes.pl 2014_1st_Quater_18_1N_expr_matrix_filtered_70pct.txt 1 > 2014_1st_Quater_18_1N_70pct_Z1_overDispersed_genes.txt
  • This time I obtain 537 over-dispersed genes, slightly less than the number 564 that I got before. 325 genes were shared between the two lists. By eye-balling the genes no present in the previous list, I did find a number of interesting ones. So I decided to analyze this list further.
 ../scripts/extract_gene_set_ENSG.pl  2014_1st_Quater_18_1N_70pct_Z1_overDispersed_genes.txt 2014_1st_Quater_18_1N_expr_matrix_filtered_70pct.txt > 2014_1st_Quater_18_1N_70pct_Z1_overDispersed_expr_matrix.txt
 x=read.table("2014_1st_Quater_18_1N_70pct_Z1_overDispersed_expr_matrix.txt",header=TRUE,row.names=1);
 x.cor = cor(log10(x+1),use="pairwise.complete.obs",method="pearson")
 hm<-heatmap.2(as.matrix(x.cor), col=bluered(128), scale="none", cexCol=0.1, cexRow=0.1 ,key=T, symkey=F,density.info="histogram",trace="none",dendrogram="both",Rowv=TRUE,Colv=TRUE)
 File:2014 1st Quater 18 70pct Z1 overDispersed genes sample clustering.png
 x[is.na(x)] <- 0
 y=log10(x+1)
 tys = scale(t(y))
 k_5=kmeans(tys, 5)
 k_4=kmeans(tys, 4)
 k_3=kmeans(tys, 3)
 k_2=kmeans(tys, 2)
 plotcluster(tys,k_2$cluster)
 plotcluster(tys,k_3$cluster)
 plotcluster(tys,k_4$cluster)
 plotcluster(tys,k_5$cluster)
 write.table(k_2$cluster, file="2014_1st_Quater_18_1N_70pct_Z1_overDispersed_k-mean_2_clusters.txt", sep = "\t", row.names=TRUE, col.names=TRUE)
 File:2014 1st Quater 18 70pct over-dispersed-Z1 k2 clusters.pngFile:2014 1st Quater 18 70pct over-dispersed-Z1 k3 clusters.png
 File:2014 1st Quater 18 70pct over-dispersed-Z1 k4 clusters.pngFile:2014 1st Quater 18 70pct over-dispersed-Z1 k5 clusters.png
  • I repeated the SAM analysis on the two clusters based on K-mean(k=2). Comparison of top-15 up-regulated genes in each cluster. Apparently a few known brain related genes, such as CXCL14, GRIK1 and KCNIP1 showed up in the list when I relaxed the filtering criterion.
<30pct missing <70pct missing
C2_up C1_up C2_up C1_up
SLC6A1 TESPA1 SLC6A1 TESPA1
GAD2 ZBTB18 GAD1 EGR1
GAD1 CTC-552D5 GAD2 TTYH1
DGKD RORB ADRA1A ADAMTS16
ATP1B2 ADAMTS3 MAF COL5A2
ANKRD55 EGR1 CXCL14 PWWP2B
NRIP3 COL11A1 ARX CBLN2
TOX3 SERPINE2 ERBB4 RORB
IGF1 TTYH1 GRIK1 FAM19A1
RAB3IP SLC38A11 RBMS3 CA10
GOLIM4 DYSF KCNIP1 MLIP
TACR1 ADAMTS16 BTBD11 SATB2
LYPD6B DUSP6 MYO16 PTPRK
RASSF8 COL5A2 DNER GULP1
FREM1 SLC22A24 PTPRM ZBTB18
  • Second round analysis of the sub-clusters in C1.
 ../scripts/extractSampleClusterExprMatrix.pl 2014_1st_Quater_18_1N_expr_matrix_filtered_70pct.txt 2014_1st_Quater_18_1N_70pct_Z1_overDispersed_k-mean_2_clusters.txt 1 > 2014_1st_Quater_18_1N_70pct_expr_matrix_k-mean2_c1.txt 
 ../scripts/exprMatrix2overDispersedGenes.pl 2014_1st_Quater_18_1N_70pct_expr_matrix_k-mean2_c1.txt 1 > 2014_1st_Quater_18_1N_70pct_k-mean2_c1_Z1_overDispersed_genes.txt
 ../scripts/extract_gene_set_ENSG.pl  2014_1st_Quater_18_1N_70pct_k-mean2_c1_Z1_overDispersed_genes.txt 2014_1st_Quater_18_1N_70pct_expr_matrix_k-mean2_c1.txt > 2014_1st_Quater_18_1N_70pct_k-mean2_c1_Z1_overDispersed_genes_expr_matrix.txt
 x=read.table("2014_1st_Quater_18_1N_70pct_k-mean2_c1_Z1_overDispersed_genes_expr_matrix.txt",header=TRUE,row.names=1);
 x.cor = cor(log10(x+1),use="pairwise.complete.obs",method="pearson")
 hm<-heatmap.2(as.matrix(x.cor), col=bluered(128), scale="none", cexCol=0.1, cexRow=0.1 ,key=T, symkey=F,density.info="histogram",trace="none",dendrogram="both",Rowv=TRUE,Colv=TRUE)
 #For some reason that I couldn't figure out, K-mean clustering didn't work on this data set. All samples collapsed on two straight lines or 3-5 define spots. So I further divide C1 into four sub-clusters based on tree cutting.
 ct<- cutree(hc.rows, h=3.8) # 
 rect.hclust(hc.rows, h=3.8) # draw red rectangles to mark the subgroups
 write.table(ct, file="2014_1st_Quater_18_1N_70pct_c1_Z1_overDispersed_cutree_4_clusters.txt", sep = "\t", row.names=TRUE, col.names=TRUE)
 File:2014 1st Quater 18 70pct over-dispersed-Z1 c1 cutree 4 clusters.png
  • Then I did the SAM analysis on four sub-clusters, and hand-picked the most up-regulated genes for each cluster.
Hugo contrast-1 contrast-2 contrast-3 contrast-4
GULP1 8.425237569 -8.046609482 -3.600322514 -0.557093767
GLIS3 8.343548625 -5.563345218 -1.192143799 -4.861766435
PDZD2 8.24768625 -8.659361328 -9.103894161 5.443518451
GPC6 7.454229827 -5.095298333 -4.376936986 -1.077364549
PTPRK 7.444544177 -6.183656599 2.248062283 -6.460351566
CUX2 7.410971043 -8.776620023 -7.187091414 4.790494964
ST6GALNAC5 7.386061028 -4.663803517 -2.542527965 -3.106939335
DGKB 7.370751451 -8.571181072 -8.594561568 6.010402456
HS6ST3 7.27057172 -8.107257569 -7.610239863 4.805509098
ADAMTS3 7.076801907 -8.42261044 -4.098819873 1.972586834
Hugo contrast-1 contrast-2 contrast-3 contrast-4
SLC35F1 -6.472144655 8.272579443 6.147406638 -4.577265042
PTCHD4 -5.502784325 7.796328744 0.786530817 -0.303972278
FOXP2 -3.860427522 6.951464815 6.996421917 -7.622723144
FSTL5 -6.125676731 6.820074944 5.252446902 -2.93360098
VWC2L -4.266060283 6.501114113 4.461321639 -4.299193058
GRIP1 -7.256540308 6.344769187 4.881292263 -0.720788007
FNBP1L -5.068946724 6.250833375 4.557613564 -3.145588857
MYRIP -1.97257783 6.110724268 5.336333823 -7.751749713
SLC44A5 -2.444135501 6.071122614 5.030114017 -6.819412197
KLHL5 -4.988223503 5.894461437 2.629056693 -1.106223833
Hugo contrast-1 contrast-2 contrast-3 contrast-4
RXFP1 -2.585159705 4.851245621 8.81861737 -9.215244171
CLSTN2 -2.882489285 3.539838122 8.210887065 -7.137631463
NRG1 0.581508264 1.277217783 8.141319499 -9.602031634
DCC -1.338380764 2.699355287 7.937058101 -8.147450676
IL1RAPL2 -5.207244176 3.833199619 7.741449575 -3.949301642
CADPS2 2.1536228 0.248433806 7.30501552 -9.951046312
CPNE4 -0.205159683 3.640818046 7.221434895 -9.722838787
POU6F2 0.98733985 3.616163654 7.220286284 -11.23448076
RORB 0.243334211 1.93511208 7.055460544 -8.6905897
FOXP2 -3.860427522 6.951464815 6.996421917 -7.622723144
Hugo contrast-1 contrast-2 contrast-3 contrast-4
GNAL 2.152713882 -4.204862434 -8.66612006 9.076171648
GRIA4 3.222367558 -4.333159768 -9.315812969 8.429467979
MRC1L1 -2.557892185 -3.230970989 -1.86243514 7.814728707
RGS12 -2.103405117 -2.204978885 -3.28629215 7.716132243
RPS6KA2 -3.299256284 1.582562861 -4.970998816 7.640257898
THSD7A 0.418925677 -2.269664451 -6.47728784 7.571726232
HPCAL1 1.004808105 -2.915489262 -6.670063124 7.551415141
SLCO2A1 2.230426314 -5.854172552 -5.600667599 7.454133784
ANKRD30BL -2.887431523 -1.875881406 -2.240940892 7.447192944
GRIK4 1.879214978 -5.679658971 -5.193369988 7.368797978
  • Take the top up-regulated genes for all groups, and generate a expression matrix for all samples. Hierarchical clustering based on these genes showed a very clear separate of the five groups. A small subset of genes can be selected to "label" these groups. Next we need to validate the in situ expression patterns of these gens by RNA FISH.
 File:2014 1st Quater 18 1N 70pct C1 cluster specific genes expr heatmap.png

DESeq vs SAM[edit]

  • I decided to compare DESeq with SAM on identifying differentially expressed genes.
  • I installed the latest DESeq2 v1.4.5 in R.
  source("http://bioconductor.org/biocLite.R")
  biocLite("DESeq2")
  #run the following commands in R, the DESeq step will take a while
  library('DESeq2')
  x=read.table("2014_1st_Quater_18_1N_70pct_Z1_overDispersed_expr_matrix.txt",header=TRUE,row.names=1);
  colData=read.table("2014_1st_Quater_18_1N_70pct_Z1_overDispersed_k-mean_2_clusters_labels.txt",header=TRUE,row.names=1);
  x[is.na(x)] <- 0
  dds <- DESeqDataSetFromMatrix(countData = round(x), colData = colData, design =~ clusters)
  dds <- DESeq(dds)
  res <- results(dds)
  #show the most significant genes
  resOrdered <- res[order(res$padj),] 
  head(resOrdered,20)
  report_df=data.frame(resOrdered$baseMean,resOrdered$log2FoldChange,resOrdered$padj)
  #report the top 100 DE genes.
  write.table(head(report_df,100), file="2014_1st_Quater_18_1N_70pct_Z1_overDispersed_C1_C2_DE.txt", sep = "\t", row.names=TRUE, col.names=TRUE)
  #set a cutoff for padj, then pick genes based on the fold changes
  resSig <- res[ which(res$padj<0.01),]
  head(resSig[order(resSig$log2FoldChange ),], 10)
  tail(resSig[order(resSig$log2FoldChange ),], 10)
  #To identify the HUGO gene names for these genes
  /home/kunzhang/RNAseq/SCAP//scripts/ENSG2HUGO.pl  2014_1st_Quater_18_1N_70pct_Z1_overDispersed_C1_C2_DE.txt

Heatmap clustering[edit]

  • Rui noticed a difference between clustering using hclust and heatmap.2: [1].
  • After some investigation, I realized that by doing clustering on x.cor using heatmap.2, the dendrogram was actually generated by calculating the correlation matrix on the correlation matrix, which is wrong.
  • The right way is to generate a dendrogram based on the correlation matrix of the samples, and feed the dendrogram to heatmap.2.
x=read.table("2014_1st_Quater_18_1N_70pct_Z1_overDispersed_expr_matrix.txt",header=TRUE,row.names=1);
x.cor = cor(log10(x+1),use="pairwise.complete.obs",method="pearson") 
hc.rows<- hclust(as.dist(1 - x.cor), method="complete")
colv <- as.dendrogram(hc.rows)
rowv <- colv
heatmap.2(as.matrix(x.cor), col=bluered(128), scale="none", cexCol=0.1, cexRow=0.1 ,key=T, symkey=F,density.info="histogram",trace="none",dendrogram="both",Rowv=rowv,Colv=colv)
                       Modified heatmap                                              Previous heatmap
 File:2014 1st Quater 18 70pct Z1 overDispersed genes sample clustering hclust.pngFile:2014 1st Quater 18 70pct Z1 overDispersed genes sample clustering.png 
plot(hc.rows,cex=0.1) 
ct<- cutree(hc.rows, h=0.9) # 
rect.hclust(hc.rows, h=0.9) # draw red rectangles to mark the subgroups
 File:2014 1st Quater 18 70pct over-dispersed-Z1 c1 cutree 2 clusters.png