Noi/NOTES/2011-12-25
HL111 (NP-BSPP-Hapmap-Nov15) mapping[edit]
- Note:
- In sequencing sample spreadsheet, these libraries were recored in lane 1-3, but they were actually loaded in lane 5-7.
- Libraries from two families (1362 and 1454) from HAPMAP plate, the 9 replicates for probe normalization and PGP1iPS treated with different bisulfite conversion kits (Zymoresearch and Imprint from Sigma). Also, one of UCLA sample (Indx94) was included in these libraries.
- NP-BSPP-Hapmap-Nov15 library normalization: [[1]]
Indx | Sample ID |
Indx10 | GM11995 (wellID G7) |
Indx11 | GM11993 (wellID F2) |
Indx12 | **GM10861 (wellID A9) |
Indx13 | GM11992 (wellID C2 ) |
Indx14 | GM11994 (wellID G9 ) |
Indx15 | GM10860 (wellID F10) |
Indx16 | GM12813 (wellID E8) |
Indx17 | GM12802 (wellID E7) |
Indx18 | GM12815 (wellID G12) |
Indx19 | GM12812 (wellID F11) |
Indx20 | GM12814 (wellID A7) |
Indx21 | GM12801 (wellID D9) |
Indx22 | NA12156 (wellID B3)-1 |
Indx23 | NA12156 (wellID B3)-2 |
Indx24 | NA12156 (wellID B3)-3 |
Indx25 | NA12156 (wellID B3)-4 |
Indx26 | NA12156 (wellID B3)-5 |
Indx27 | NA12156 (wellID B3)-6 |
Indx28 | NA12156 (wellID B3)-7 |
Indx29 | NA12156 (wellID B3)-8 |
Indx30 | NA12156 (wellID B3)-9 |
Indx41 | **PGP1iPS |
Indx42 | PGP1iPS-IMPRINT |
Indx94 | UCLA-RO_Indx94 |
- Sequencing data is in genome-miner: /home/kunzhang/FreshReads/111209_SN1001
Merge and trim 27bp from 5' end[edit]
- Note: the last four digits in sample ID represent the family number
less s_5_1_Indx10.txt s_5_2_Indx10.txt s_6_1_Indx10.txt s_6_2_Indx10.txt s_7_1_Indx10.txt s_7_2_Indx10.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM11995-1362
less s_5_1_Indx11.txt s_5_2_Indx11.txt s_6_1_Indx11.txt s_6_2_Indx11.txt s_7_1_Indx11.txt s_7_2_Indx11.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM11993-1362
less s_5_1_Indx12.txt s_5_2_Indx12.txt s_6_1_Indx12.txt s_6_2_Indx12.txt s_7_1_Indx12.txt s_7_2_Indx12.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM10861-1362
less s_5_1_Indx13.txt s_5_2_Indx13.txt s_6_1_Indx13.txt s_6_2_Indx13.txt s_7_1_Indx13.txt s_7_2_Indx13.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM11992-1362
less s_5_1_Indx14.txt s_5_2_Indx14.txt s_6_1_Indx14.txt s_6_2_Indx14.txt s_7_1_Indx14.txt s_7_2_Indx14.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM11994-1362
less s_5_1_Indx15.txt s_5_2_Indx15.txt s_6_1_Indx15.txt s_6_2_Indx15.txt s_7_1_Indx15.txt s_7_2_Indx15.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM10860-1362
less s_5_1_Indx16.txt s_5_2_Indx16.txt s_6_1_Indx16.txt s_6_2_Indx16.txt s_7_1_Indx16.txt s_7_2_Indx16.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM12813-1454
less s_5_1_Indx17.txt s_5_2_Indx17.txt s_6_1_Indx17.txt s_6_2_Indx17.txt s_7_1_Indx17.txt s_7_2_Indx17.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM12802-1454
less s_5_1_Indx18.txt s_5_2_Indx18.txt s_6_1_Indx18.txt s_6_2_Indx18.txt s_7_1_Indx18.txt s_7_2_Indx18.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM12815-1454
less s_5_1_Indx19.txt s_5_2_Indx19.txt s_6_1_Indx19.txt s_6_2_Indx19.txt s_7_1_Indx19.txt s_7_2_Indx19.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM12812-1454
less s_5_1_Indx20.txt s_5_2_Indx20.txt s_6_1_Indx20.txt s_6_2_Indx20.txt s_7_1_Indx20.txt s_7_2_Indx20.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM12814-1454
less s_5_1_Indx21.txt s_5_2_Indx21.txt s_6_1_Indx21.txt s_6_2_Indx21.txt s_7_1_Indx21.txt s_7_2_Indx21.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM12801-1454
less s_5_1_Indx22.txt s_5_2_Indx22.txt s_6_1_Indx22.txt s_6_2_Indx22.txt s_7_1_Indx22.txt s_7_2_Indx22.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/NA12156-1408_1
less s_5_1_Indx23.txt s_5_2_Indx23.txt s_6_1_Indx23.txt s_6_2_Indx23.txt s_7_1_Indx23.txt s_7_2_Indx23.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/NA12156-1408_2
less s_5_1_Indx24.txt s_5_2_Indx24.txt s_6_1_Indx24.txt s_6_2_Indx24.txt s_7_1_Indx24.txt s_7_2_Indx24.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/NA12156-1408_3
less s_5_1_Indx25.txt s_5_2_Indx25.txt s_6_1_Indx25.txt s_6_2_Indx25.txt s_7_1_Indx25.txt s_7_2_Indx25.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/NA12156-1408_4
less s_5_1_Indx26.txt s_5_2_Indx26.txt s_6_1_Indx26.txt s_6_2_Indx26.txt s_7_1_Indx26.txt s_7_2_Indx26.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/NA12156-1408_5
less s_5_1_Indx27.txt s_5_2_Indx27.txt s_6_1_Indx27.txt s_6_2_Indx27.txt s_7_1_Indx27.txt s_7_2_Indx27.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/NA12156-1408_6
less s_5_1_Indx28.txt s_5_2_Indx28.txt s_6_1_Indx28.txt s_6_2_Indx28.txt s_7_1_Indx28.txt s_7_2_Indx28.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/NA12156-1408_7
less s_5_1_Indx29.txt s_5_2_Indx29.txt s_6_1_Indx29.txt s_6_2_Indx29.txt s_7_1_Indx29.txt s_7_2_Indx29.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/NA12156-1408_8
less s_5_1_Indx30.txt s_5_2_Indx30.txt s_6_1_Indx30.txt s_6_2_Indx30.txt s_7_1_Indx30.txt s_7_2_Indx30.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/NA12156-1408_9
less s_5_1_Indx41.txt s_5_2_Indx41.txt s_6_1_Indx41.txt s_6_2_Indx41.txt s_7_1_Indx41.txt s_7_2_Indx41.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/PGP1iPS-ZYMO
less s_5_1_Indx42.txt s_5_2_Indx42.txt s_6_1_Indx42.txt s_6_2_Indx42.txt s_7_1_Indx42.txt s_7_2_Indx42.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/PGP1iPS-IMPRINT
less s_5_1_Indx94.txt s_5_2_Indx94.txt s_6_1_Indx94.txt s_6_2_Indx94.txt s_7_1_Indx94.txt s_7_2_Indx94.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GK0281-004
Mapping to hg19 on Triton cluster[edit]
- Note:
- Since some files are very big, they failed to get the output files. I need to split those files (on Triton cluster) and do mapping again.
- I used head and tail commands instead of split command.
head -111017824 GM10861-1362 > GM10861-1362-1
head -106854184 GM11992-1362 > GM11992-1362-1
head -122652416 GM11995-1362 > GM11995-1362-1
head -108824480 GM12802-1454 > GM12802-1454-1
head -109761668 GM12812-1454 > GM12812-1454-1
head -114040252 GM12813-1454 > GM12813-1454-1
head -109525956 GM12814-1454 > GM12814-1454-1
head -124399152 GM12815-1454 > GM12815-1454-1
head -113341284 NA12156-1408_1 > NA12156-1408_1-1
head -104618848 NA12156-1408_3 > NA12156-1408_3-1
head -109221100 NA12156-1408_4 > NA12156-1408_4-1
head -114618216 NA12156-1408_5 > NA12156-1408_5-1
head -111514412 NA12156-1408_6 > NA12156-1408_6-1
tail -111017824 GM10861-1362 > GM10861-1362-2
tail -106854184 GM11992-1362 > GM11992-1362-2
tail -122652416 GM11995-1362 > GM11995-1362-2
tail -108824480 GM12802-1454 > GM12802-1454-2
tail -109761668 GM12812-1454 > GM12812-1454-2
tail -114040252 GM12813-1454 > GM12813-1454-2
tail -109525956 GM12814-1454 > GM12814-1454-2
tail -124399152 GM12815-1454 > GM12815-1454-2
tail -113341284 NA12156-1408_1 > NA12156-1408_1-2
tail -104618848 NA12156-1408_3 > NA12156-1408_3-2
tail -109221100 NA12156-1408_4 > NA12156-1408_4-2
tail -114618216 NA12156-1408_5 > NA12156-1408_5-2
tail -111514412 NA12156-1408_6 > NA12156-1408_6-2
- After mapping, the methylFreq files were concatenated (on genome-miner)
cat GM10861-1362-1.fwd.pileup.methylFreq GM10861-1362-2.fwd.pileup.methylFreq > GM10861-1362.fwd.pileup.methylFreq
cat GM11992-1362-1.fwd.pileup.methylFreq GM11992-1362-2.fwd.pileup.methylFreq > GM11992-1362.fwd.pileup.methylFreq
cat GM11995-1362-1.fwd.pileup.methylFreq GM11995-1362-2.fwd.pileup.methylFreq > GM11995-1362.fwd.pileup.methylFreq
cat GM12802-1454-1.fwd.pileup.methylFreq GM12802-1454-2.fwd.pileup.methylFreq > GM12802-1454.fwd.pileup.methylFreq
cat GM12812-1454-1.fwd.pileup.methylFreq GM12812-1454-2.fwd.pileup.methylFreq > GM12812-1454.fwd.pileup.methylFreq
cat GM12813-1454-1.fwd.pileup.methylFreq GM12813-1454-2.fwd.pileup.methylFreq > GM12813-1454.fwd.pileup.methylFreq
cat GM12814-1454-1.fwd.pileup.methylFreq GM12814-1454-2.fwd.pileup.methylFreq > GM12814-1454.fwd.pileup.methylFreq
cat GM12815-1454-1.fwd.pileup.methylFreq GM12815-1454-2.fwd.pileup.methylFreq > GM12815-1454.fwd.pileup.methylFreq
cat NA12156-1408_1-1.fwd.pileup.methylFreq NA12156-1408_1-2.fwd.pileup.methylFreq > NA12156-1408_1.fwd.pileup.methylFreq
cat NA12156-1408_3-1.fwd.pileup.methylFreq NA12156-1408_3-2.fwd.pileup.methylFreq > NA12156-1408_3.fwd.pileup.methylFreq
cat NA12156-1408_4-1.fwd.pileup.methylFreq NA12156-1408_4-2.fwd.pileup.methylFreq > NA12156-1408_4.fwd.pileup.methylFreq
cat NA12156-1408_5-1.fwd.pileup.methylFreq NA12156-1408_5-2.fwd.pileup.methylFreq > NA12156-1408_5.fwd.pileup.methylFreq
cat NA12156-1408_6-1.fwd.pileup.methylFreq NA12156-1408_6-2.fwd.pileup.methylFreq > NA12156-1408_6.fwd.pileup.methylFreq
- Generated BED files by methylFreq2BED.pl script (on genome-miner).
perl /home/kunzhang/bin/methylFreq2BED.pl GM10861-1362 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/GM10861-1362.fwd.pileup.methylFreq > GM10861-1362.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl GM11992-1362 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/GM11992-1362.fwd.pileup.methylFreq > GM11992-1362.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl GM11995-1362 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/GM11995-1362.fwd.pileup.methylFreq > GM11995-1362.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl GM12802-1454 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/GM12802-1454.fwd.pileup.methylFreq > GM12802-1454.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl GM12812-1454 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/GM12812-1454.fwd.pileup.methylFreq > GM12812-1454.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl GM12813-1454 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/GM12813-1454.fwd.pileup.methylFreq > GM12813-1454.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl GM12814-1454 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/GM12814-1454.fwd.pileup.methylFreq > GM12814-1454.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl GM12815-1454 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/GM12815-1454.fwd.pileup.methylFreq > GM12815-1454.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl NA12156-1408_1 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/NA12156-1408_1.fwd.pileup.methylFreq > NA12156-1408_1.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl NA12156-1408_3 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/NA12156-1408_3.fwd.pileup.methylFreq > NA12156-1408_3.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl NA12156-1408_4 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/NA12156-1408_4.fwd.pileup.methylFreq > NA12156-1408_4.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl NA12156-1408_5 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/NA12156-1408_5.fwd.pileup.methylFreq > NA12156-1408_5.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl NA12156-1408_6 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/NA12156-1408_6.fwd.pileup.methylFreq > NA12156-1408_6.fwd.pileup.methylFreq.hg19.BED.txt
- To generate methylation matrix I used Dinh'script (allBED2Matrix_DD.pl) to run since I got the wrong number of samples after getting ~81,332 line in the matrix. Dinh's script uses list of file instead of directory location.
- For exmple,
- GK0281-004 HAPMAP.BED/GK0281-004.fwd.pileup.methylFreq.hg19.BED.txt --> the first column is the display name, and the second column is the input file
- To run --> ./allBED2Matrix_DD.pl bed_list 40 1 0.1 (no need to put output file)