Noi/NOTES/2011-12-25

From ZhangLabWiki
Jump to navigation Jump to search

HL111 (NP-BSPP-Hapmap-Nov15) mapping[edit]

  • Note:
    • In sequencing sample spreadsheet, these libraries were recored in lane 1-3, but they were actually loaded in lane 5-7.
    • Libraries from two families (1362 and 1454) from HAPMAP plate, the 9 replicates for probe normalization and PGP1iPS treated with different bisulfite conversion kits (Zymoresearch and Imprint from Sigma). Also, one of UCLA sample (Indx94) was included in these libraries.
  • NP-BSPP-Hapmap-Nov15 library normalization: [[1]]
Indx Sample ID
Indx10 GM11995 (wellID G7)
Indx11 GM11993 (wellID F2)
Indx12 **GM10861 (wellID A9)
Indx13 GM11992 (wellID C2 )
Indx14 GM11994 (wellID G9 )
Indx15 GM10860 (wellID F10)
Indx16 GM12813 (wellID E8)
Indx17 GM12802 (wellID E7)
Indx18 GM12815 (wellID G12)
Indx19 GM12812 (wellID F11)
Indx20 GM12814 (wellID A7)
Indx21 GM12801 (wellID D9)
Indx22 NA12156 (wellID B3)-1
Indx23 NA12156 (wellID B3)-2
Indx24 NA12156 (wellID B3)-3
Indx25 NA12156 (wellID B3)-4
Indx26 NA12156 (wellID B3)-5
Indx27 NA12156 (wellID B3)-6
Indx28 NA12156 (wellID B3)-7
Indx29 NA12156 (wellID B3)-8
Indx30 NA12156 (wellID B3)-9
Indx41 **PGP1iPS
Indx42 PGP1iPS-IMPRINT
Indx94 UCLA-RO_Indx94
  • Sequencing data is in genome-miner: /home/kunzhang/FreshReads/111209_SN1001

Merge and trim 27bp from 5' end[edit]

  • Note: the last four digits in sample ID represent the family number

less s_5_1_Indx10.txt s_5_2_Indx10.txt s_6_1_Indx10.txt s_6_2_Indx10.txt s_7_1_Indx10.txt s_7_2_Indx10.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM11995-1362
less s_5_1_Indx11.txt s_5_2_Indx11.txt s_6_1_Indx11.txt s_6_2_Indx11.txt s_7_1_Indx11.txt s_7_2_Indx11.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM11993-1362
less s_5_1_Indx12.txt s_5_2_Indx12.txt s_6_1_Indx12.txt s_6_2_Indx12.txt s_7_1_Indx12.txt s_7_2_Indx12.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM10861-1362
less s_5_1_Indx13.txt s_5_2_Indx13.txt s_6_1_Indx13.txt s_6_2_Indx13.txt s_7_1_Indx13.txt s_7_2_Indx13.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM11992-1362
less s_5_1_Indx14.txt s_5_2_Indx14.txt s_6_1_Indx14.txt s_6_2_Indx14.txt s_7_1_Indx14.txt s_7_2_Indx14.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM11994-1362
less s_5_1_Indx15.txt s_5_2_Indx15.txt s_6_1_Indx15.txt s_6_2_Indx15.txt s_7_1_Indx15.txt s_7_2_Indx15.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM10860-1362
less s_5_1_Indx16.txt s_5_2_Indx16.txt s_6_1_Indx16.txt s_6_2_Indx16.txt s_7_1_Indx16.txt s_7_2_Indx16.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM12813-1454
less s_5_1_Indx17.txt s_5_2_Indx17.txt s_6_1_Indx17.txt s_6_2_Indx17.txt s_7_1_Indx17.txt s_7_2_Indx17.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM12802-1454
less s_5_1_Indx18.txt s_5_2_Indx18.txt s_6_1_Indx18.txt s_6_2_Indx18.txt s_7_1_Indx18.txt s_7_2_Indx18.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM12815-1454
less s_5_1_Indx19.txt s_5_2_Indx19.txt s_6_1_Indx19.txt s_6_2_Indx19.txt s_7_1_Indx19.txt s_7_2_Indx19.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM12812-1454
less s_5_1_Indx20.txt s_5_2_Indx20.txt s_6_1_Indx20.txt s_6_2_Indx20.txt s_7_1_Indx20.txt s_7_2_Indx20.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM12814-1454
less s_5_1_Indx21.txt s_5_2_Indx21.txt s_6_1_Indx21.txt s_6_2_Indx21.txt s_7_1_Indx21.txt s_7_2_Indx21.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GM12801-1454
less s_5_1_Indx22.txt s_5_2_Indx22.txt s_6_1_Indx22.txt s_6_2_Indx22.txt s_7_1_Indx22.txt s_7_2_Indx22.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/NA12156-1408_1
less s_5_1_Indx23.txt s_5_2_Indx23.txt s_6_1_Indx23.txt s_6_2_Indx23.txt s_7_1_Indx23.txt s_7_2_Indx23.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/NA12156-1408_2
less s_5_1_Indx24.txt s_5_2_Indx24.txt s_6_1_Indx24.txt s_6_2_Indx24.txt s_7_1_Indx24.txt s_7_2_Indx24.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/NA12156-1408_3
less s_5_1_Indx25.txt s_5_2_Indx25.txt s_6_1_Indx25.txt s_6_2_Indx25.txt s_7_1_Indx25.txt s_7_2_Indx25.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/NA12156-1408_4
less s_5_1_Indx26.txt s_5_2_Indx26.txt s_6_1_Indx26.txt s_6_2_Indx26.txt s_7_1_Indx26.txt s_7_2_Indx26.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/NA12156-1408_5
less s_5_1_Indx27.txt s_5_2_Indx27.txt s_6_1_Indx27.txt s_6_2_Indx27.txt s_7_1_Indx27.txt s_7_2_Indx27.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/NA12156-1408_6
less s_5_1_Indx28.txt s_5_2_Indx28.txt s_6_1_Indx28.txt s_6_2_Indx28.txt s_7_1_Indx28.txt s_7_2_Indx28.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/NA12156-1408_7
less s_5_1_Indx29.txt s_5_2_Indx29.txt s_6_1_Indx29.txt s_6_2_Indx29.txt s_7_1_Indx29.txt s_7_2_Indx29.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/NA12156-1408_8
less s_5_1_Indx30.txt s_5_2_Indx30.txt s_6_1_Indx30.txt s_6_2_Indx30.txt s_7_1_Indx30.txt s_7_2_Indx30.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/NA12156-1408_9
less s_5_1_Indx41.txt s_5_2_Indx41.txt s_6_1_Indx41.txt s_6_2_Indx41.txt s_7_1_Indx41.txt s_7_2_Indx41.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/PGP1iPS-ZYMO
less s_5_1_Indx42.txt s_5_2_Indx42.txt s_6_1_Indx42.txt s_6_2_Indx42.txt s_7_1_Indx42.txt s_7_2_Indx42.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/PGP1iPS-IMPRINT
less s_5_1_Indx94.txt s_5_2_Indx94.txt s_6_1_Indx94.txt s_6_2_Indx94.txt s_7_1_Indx94.txt s_7_2_Indx94.txt | /home/kunzhang/bin/trimFastq.pl 27 83 >/home/nplongth/Noi_scratch/HL111_BSPP-Hapmap_20111224/GK0281-004

Mapping to hg19 on Triton cluster[edit]

  • Note:
    • Since some files are very big, they failed to get the output files. I need to split those files (on Triton cluster) and do mapping again.
    • I used head and tail commands instead of split command.

head -111017824 GM10861-1362 > GM10861-1362-1
head -106854184 GM11992-1362 > GM11992-1362-1
head -122652416 GM11995-1362 > GM11995-1362-1
head -108824480 GM12802-1454 > GM12802-1454-1
head -109761668 GM12812-1454 > GM12812-1454-1
head -114040252 GM12813-1454 > GM12813-1454-1
head -109525956 GM12814-1454 > GM12814-1454-1
head -124399152 GM12815-1454 > GM12815-1454-1
head -113341284 NA12156-1408_1 > NA12156-1408_1-1
head -104618848 NA12156-1408_3 > NA12156-1408_3-1
head -109221100 NA12156-1408_4 > NA12156-1408_4-1
head -114618216 NA12156-1408_5 > NA12156-1408_5-1
head -111514412 NA12156-1408_6 > NA12156-1408_6-1
tail -111017824 GM10861-1362 > GM10861-1362-2
tail -106854184 GM11992-1362 > GM11992-1362-2
tail -122652416 GM11995-1362 > GM11995-1362-2
tail -108824480 GM12802-1454 > GM12802-1454-2
tail -109761668 GM12812-1454 > GM12812-1454-2
tail -114040252 GM12813-1454 > GM12813-1454-2
tail -109525956 GM12814-1454 > GM12814-1454-2
tail -124399152 GM12815-1454 > GM12815-1454-2
tail -113341284 NA12156-1408_1 > NA12156-1408_1-2
tail -104618848 NA12156-1408_3 > NA12156-1408_3-2
tail -109221100 NA12156-1408_4 > NA12156-1408_4-2
tail -114618216 NA12156-1408_5 > NA12156-1408_5-2
tail -111514412 NA12156-1408_6 > NA12156-1408_6-2

  • After mapping, the methylFreq files were concatenated (on genome-miner)

cat GM10861-1362-1.fwd.pileup.methylFreq GM10861-1362-2.fwd.pileup.methylFreq > GM10861-1362.fwd.pileup.methylFreq
cat GM11992-1362-1.fwd.pileup.methylFreq GM11992-1362-2.fwd.pileup.methylFreq > GM11992-1362.fwd.pileup.methylFreq
cat GM11995-1362-1.fwd.pileup.methylFreq GM11995-1362-2.fwd.pileup.methylFreq > GM11995-1362.fwd.pileup.methylFreq
cat GM12802-1454-1.fwd.pileup.methylFreq GM12802-1454-2.fwd.pileup.methylFreq > GM12802-1454.fwd.pileup.methylFreq
cat GM12812-1454-1.fwd.pileup.methylFreq GM12812-1454-2.fwd.pileup.methylFreq > GM12812-1454.fwd.pileup.methylFreq
cat GM12813-1454-1.fwd.pileup.methylFreq GM12813-1454-2.fwd.pileup.methylFreq > GM12813-1454.fwd.pileup.methylFreq
cat GM12814-1454-1.fwd.pileup.methylFreq GM12814-1454-2.fwd.pileup.methylFreq > GM12814-1454.fwd.pileup.methylFreq
cat GM12815-1454-1.fwd.pileup.methylFreq GM12815-1454-2.fwd.pileup.methylFreq > GM12815-1454.fwd.pileup.methylFreq
cat NA12156-1408_1-1.fwd.pileup.methylFreq NA12156-1408_1-2.fwd.pileup.methylFreq > NA12156-1408_1.fwd.pileup.methylFreq
cat NA12156-1408_3-1.fwd.pileup.methylFreq NA12156-1408_3-2.fwd.pileup.methylFreq > NA12156-1408_3.fwd.pileup.methylFreq
cat NA12156-1408_4-1.fwd.pileup.methylFreq NA12156-1408_4-2.fwd.pileup.methylFreq > NA12156-1408_4.fwd.pileup.methylFreq
cat NA12156-1408_5-1.fwd.pileup.methylFreq NA12156-1408_5-2.fwd.pileup.methylFreq > NA12156-1408_5.fwd.pileup.methylFreq
cat NA12156-1408_6-1.fwd.pileup.methylFreq NA12156-1408_6-2.fwd.pileup.methylFreq > NA12156-1408_6.fwd.pileup.methylFreq

  • Generated BED files by methylFreq2BED.pl script (on genome-miner).

perl /home/kunzhang/bin/methylFreq2BED.pl GM10861-1362 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/GM10861-1362.fwd.pileup.methylFreq > GM10861-1362.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl GM11992-1362 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/GM11992-1362.fwd.pileup.methylFreq > GM11992-1362.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl GM11995-1362 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/GM11995-1362.fwd.pileup.methylFreq > GM11995-1362.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl GM12802-1454 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/GM12802-1454.fwd.pileup.methylFreq > GM12802-1454.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl GM12812-1454 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/GM12812-1454.fwd.pileup.methylFreq > GM12812-1454.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl GM12813-1454 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/GM12813-1454.fwd.pileup.methylFreq > GM12813-1454.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl GM12814-1454 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/GM12814-1454.fwd.pileup.methylFreq > GM12814-1454.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl GM12815-1454 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/GM12815-1454.fwd.pileup.methylFreq > GM12815-1454.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl NA12156-1408_1 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/NA12156-1408_1.fwd.pileup.methylFreq > NA12156-1408_1.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl NA12156-1408_3 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/NA12156-1408_3.fwd.pileup.methylFreq > NA12156-1408_3.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl NA12156-1408_4 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/NA12156-1408_4.fwd.pileup.methylFreq > NA12156-1408_4.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl NA12156-1408_5 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/NA12156-1408_5.fwd.pileup.methylFreq > NA12156-1408_5.fwd.pileup.methylFreq.hg19.BED.txt
perl /home/kunzhang/bin/methylFreq2BED.pl NA12156-1408_6 10 <HAPMAP_mapping.data_hg19/HAPMAP.methylFreq/NA12156-1408_6.fwd.pileup.methylFreq > NA12156-1408_6.fwd.pileup.methylFreq.hg19.BED.txt

  • To generate methylation matrix I used Dinh'script (allBED2Matrix_DD.pl) to run since I got the wrong number of samples after getting ~81,332 line in the matrix. Dinh's script uses list of file instead of directory location.
  • For exmple,
    • GK0281-004 HAPMAP.BED/GK0281-004.fwd.pileup.methylFreq.hg19.BED.txt --> the first column is the display name, and the second column is the input file
    • To run --> ./allBED2Matrix_DD.pl bed_list 40 1 0.1 (no need to put output file)