Dinh/Dinh 2012/NOTES/2012-3-22
Jump to navigation
Jump to search
BSPP data trimming and cleaning rules using 48 African (UPenn) as an example
Methylation
- Script to generate methylation matrix:
- From our March 16 meeting, we finalized ways to clean data using the simulation results from the 9 technical replicates.
Inclusion/Exclusion rule
- Take the array of QC (pearson's correlation value) and perform a Grubbs' test
Reject samples with low outlying QC value by p<0.05.
- Take the array of number of CpGs characterized within each BED file and perform a Grubbs' test
Reject samples with low outlying CpG captured by p<0.05.
- Rationale: Samples with low QC are problematic because they contain many low-quality sites. We should also reject samples with low CpG captured because this means that the capture did not worked well and there may be many PCR duplicates.
Remove low confidence sites
- The methylation matrix needs to be cleaned by removing all high technical variability sites:
Accept site if techRepSTDEV < 0.1 OR p_value > 0.1
- Since not all CpG sites have been characterized by the 9 technical replicates (only ~200K can be used each time), we need to figure out a way to infer techRevSTDEV and p_value from probe efficiency measures.
- Script to append p-value from techical replicate
mQTL
- As a standard requirement, CpGs for mQTL should contain no more than 95% individuals with hypo (<0.2) or hypermethylation (>0.8) since such sites are probably within CpG islands.
- Script to filter by hypo or hypermethylation
PCA
- As a standard to perform PCA, we need to calculate a subset of sites with low error.
- Find STDEV = x such that number of sites with STDEV=x in data (A) is greater than the number of sites with STDEV=x in technical replicates (B)
- When using err ratio (B/A) <= 0.2, this means that 20% of variabilities are contributed by sampling variabilities (maybe some technical variabilities as well).
- Script to find x:
BSPP SNP
- Script to generate SNP matrix (.tped/.tfam):
Use PLINK for trimming SNP matrix and distance calculations
- Filter matrix to allow maximum 25% missing values (25% for QTL, may require less for clustering):
(suffix name for tped/tfam is UPenn44.filteredSNP134) plink-1.07-x86_64/plink --noweb --tfile UPenn44.filteredSNP134 --geno 0.25 --recode --out cleaned
- Tranpose the clean matrix to tped/tfam again:
plink-1.07-x86_64/plink --noweb --file cleaned --recode --transpose --out min33.UPenn44.filteredSNP134
- To get the IBD distance matrix:
plink-1.07-x86_64/plink --tfile min33.UPenn44.filteredSNP134 --cluster --distance-matrix --noweb (plink.mdist will be generated, rename the file:) mv plink.mdist UPenn44_IBD
- Note that this matrix does not have headers for both rows and columns, so when clustering using R
A = read.table("*.tfam",F) B = read.table("UPenn44_IBD",F) rownames(B) = A$V2 colnames(B) = A$V2 plot(hclust(as.dist(B), method="average"), main="SNP cluster, n=44")