Daniel:Notebook/Haplotyping/DataMap: Difference between revisions

From ZhangLabWiki
Jump to navigation Jump to search
>Djacobse
No edit summary
>Djacobse
 
(21 intermediate revisions by the same user not shown)
Line 4: Line 4:


This page contains the data map for the haplotyping project, specifically tables that outline locations of each of the data types.
This page contains the data map for the haplotyping project, specifically tables that outline locations of each of the data types.
==Reference Data==
The reference data is the data used to create the "true" VCFs, since HAPCUT requires a true vcf file as input which informs it where the relevant sites are.  The original PGP1 data is in genome miner in the directory /home/kunzhang/genomeSeq/Data/PGP1_CGI_WGS.  The data was from 3 different cell lines.
===Processing Reference Data===
The reference data information can be found [[Daniel:Notebook/Haplotyping/MergeVCF|here]]
===Data Update Dec. 2015===
It occurs to me this needs to be better updated, so I'm linking to a google doc.
{| class="wikitable" <hiddentext>generated with [[:de:Wikipedia:Helferlein/VBA-Macro for EXCEL tableconversion]] V1.8</hiddentext>
|- font-size:12pt;font-weight:bold"
| width="600" height="46" | Data Map
| width="600" | Notebook Link
|- style="font-size:12pt"  valign="bottom"
| [https://docs.google.com/spreadsheets/d/1gStTrST8MaWWlHeXMeWXVpjsNvRn8HUdeC4gSUbnPxQ/edit#gid=0 Hi-C Data Map]
|
|- style="font-size:12pt"  valign="bottom"
| [https://docs.google.com/spreadsheets/d/1gStTrST8MaWWlHeXMeWXVpjsNvRn8HUdeC4gSUbnPxQ/edit#gid=1124859722 BAC Data Map]
| [[Daniel:Notebook/Haplotyping/BACData|Processing BAC Data]]
|- style="font-size:12pt"  valign="bottom"
| [https://docs.google.com/spreadsheets/d/1gStTrST8MaWWlHeXMeWXVpjsNvRn8HUdeC4gSUbnPxQ/edit#gid=1059932227 LFR Data Map]
| [[Daniel:Notebook/Haplotyping/LFRData|Processing LFR Data]]
|- style="font-size:12pt"  valign="bottom"
| [https://docs.google.com/spreadsheets/d/1gStTrST8MaWWlHeXMeWXVpjsNvRn8HUdeC4gSUbnPxQ/edit#gid=11247187 BAC+Hi-C Data Map]
| [[Daniel:Notebook/Haplotyping/BAC-HiC|BAC and Hi-C Data Processing]]
|- style="font-size:12pt"  valign="bottom"
| [https://docs.google.com/spreadsheets/d/1gStTrST8MaWWlHeXMeWXVpjsNvRn8HUdeC4gSUbnPxQ/edit#gid=2147175569 LFR + Hi-C Data Map]
| [[Daniel:Notebook/Haplotyping/LFR-HiC|LFR and Hi-C Data Processing]]
|- style="font-size:12pt"  valign="bottom"
| [https://docs.google.com/spreadsheets/d/1gStTrST8MaWWlHeXMeWXVpjsNvRn8HUdeC4gSUbnPxQ/edit#gid=676014356 BAC+Hi-C+LFR Data Map]
| [[Daniel:Notebook/Haplotyping/LFR-BAC-HiC|LFR+BAC+Hi-C Data Processing]]
|- style="font-size:12pt"  valign="bottom"
| [https://docs.google.com/spreadsheets/d/1gStTrST8MaWWlHeXMeWXVpjsNvRn8HUdeC4gSUbnPxQ/edit#gid=1920656431 BEAGLE Data Map]
| [[Daniel:Notebook/Haplotyping/BEAGLE|BEAGLE Data Processing]]
|}


==Hi-C Data==
==Hi-C Data==
Line 89: Line 136:
  | PGP1_***BacPool_Indx**.fixed.bam
  | PGP1_***BacPool_Indx**.fixed.bam


|- style="font-size:12pt"  valign="bottom"
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | bam files-contigs
| height="15" | bam files-contigs
  | Genome Miner
  | Genome Miner
Line 95: Line 142:
  | Indx**.bacs.bam
  | Indx**.bacs.bam


|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
|- style="font-size:12pt"  valign="bottom"
| height="15" | bam files-by chrom
| height="15" | bam files-by chrom
  | Genome Miner
  | Genome Miner
Line 101: Line 148:
  | chr*/chr*.Indx**.bam
  | chr*/chr*.Indx**.bam


|- style="font-size:12pt"  valign="bottom"
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | HAIRS-raw
| height="15" | HAIRS-raw
  | Genome Miner
  | Genome Miner
Line 107: Line 154:
  | chr*/chr*.Indx**.bam
  | chr*/chr*.Indx**.bam


|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
|- style="font-size:12pt"  valign="bottom"
| height="15" | HAIRS-merge
| height="15" | HAIRS-merge
  | Genome Miner
  | Genome Miner
Line 113: Line 160:
  | chr*/fragments.chr*.bac.txt
  | chr*/fragments.chr*.bac.txt


|- style="font-size:12pt"  valign="bottom"
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | haplotypes
| height="15" | haplotypes
  | Genome Miner
  | Genome Miner
Line 131: Line 178:
===PGP1-21===
===PGP1-21===


{| class="wikitable" <hiddentext>generated with [[:de:Wikipedia:Helferlein/VBA-Macro for EXCEL tableconversion]] V1.8</hiddentext>
|- style="background-color:#D8E4BC;font-size:12pt;font-weight:bold"
| width="159" height="56" | File Type-PGP1_21
| width="545" | Path
| width="433" | File(s)
|- style="font-size:12pt"  valign="bottom"
| height="15" | VCF-by chrom
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/data_by_chamber
| chamber**/chamber**_pgp1_21.snps.raw.vcf
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | VCF-pgp1_21
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/data_by_chamber
| pgp1.22.SISSOR.vcf
|- style="font-size:12pt"  valign="bottom"
| height="15" | bam files-original
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/data_by_chamber
| chamber**_pgp1_21.BQSR.realigned.bam
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | bam files-contigs
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/contigs
| chamber**.contigs.bam


|- style="font-size:12pt"  valign="bottom"
| height="15" | bam files-by chrom
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/bams_by_chromosome
| chr*/chr*.chamber**.bam
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | HAIRS-raw
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/HAIRS
| chr*/fragments.chr*.chamber**.txt
|- style="font-size:12pt"  valign="bottom"
| height="15" | HAIRS-merge
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/HAIRS
| chr*/fragments.chr*.pgp1_21.txt
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | haplotypes
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/haplotypes
| haplotype.chr*.sissor.txt
|}


'''Notes:'''
'''Notes:'''
Line 140: Line 233:
===PGP1-22===
===PGP1-22===


{| class="wikitable" <hiddentext>generated with [[:de:Wikipedia:Helferlein/VBA-Macro for EXCEL tableconversion]] V1.8</hiddentext>
|- style="background-color:#D8E4BC;font-size:12pt;font-weight:bold"
| width="159" height="38" | File Type-PGP1_22
| width="545" | Path
| width="433" | File(s)
|- style="font-size:12pt"  valign="bottom"
| height="15" | VCF-by chrom
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/data_by_chamber
| chamber**/pgp22_chamber**.snps.raw.vcf
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | VCF-pgp1_22
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/data_by_chamber
| pgp1.22.SISSOR.vcf
|- style="font-size:12pt"  valign="bottom"
| height="15" | bam files-original
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/data_by_chamber
| chamber**/pgp22_chamber*.BQSR.realigned.bam
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | bam files-contigs
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/contigs
| chamber**.contigs.bam
|- style="font-size:12pt"  valign="bottom"
| height="15" | bam files-by chrom
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/bams_by_chromosome
| chr*/chr*.chamber*.bam
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | HAIRS-raw
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/HAIRS
| chr*/fragments.chr*.chamber*.txt
|- style="font-size:12pt"  valign="bottom"
| height="15" | HAIRS-merge
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/HAIRS
| chr*/fragments.chr*.pgp1_21.txt
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | haplotypes
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/haplotypes
| haplotype.chr*.sissor.txt


|}


'''Notes:'''
'''Notes:'''
Line 148: Line 287:


===Merge-SISSOR===
===Merge-SISSOR===
{| class="wikitable" <hiddentext>generated with [[:de:Wikipedia:Helferlein/VBA-Macro for EXCEL tableconversion]] V1.8</hiddentext>
|- style="background-color:#D8E4BC;font-size:12pt;font-weight:bold"
| width="159" height="36" | File Type-SISSOR (PGP1-21,22)
| width="545"  | Path
| width="433"  | File(s)
|- style="font-size:12pt"  valign="bottom"
| height="15" | VCF-PGP1
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp1_21and22
| pgp1_21and22.vcf
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | HAIRS-merge
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp1_21and22/HAIRS
| fragments.chr*.sissor.txt
|- style="font-size:12pt"  valign="bottom"
| height="15" | haplotypes
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp1_21and22/haplotypes
| haplotype.chr*.sissor.txt
|}
'''Notes:'''
* * chromosomes 1 to 22

Latest revision as of 18:11, 15 December 2015

Data Map[edit]

Back to Notebook

This page contains the data map for the haplotyping project, specifically tables that outline locations of each of the data types.

Reference Data[edit]

The reference data is the data used to create the "true" VCFs, since HAPCUT requires a true vcf file as input which informs it where the relevant sites are. The original PGP1 data is in genome miner in the directory /home/kunzhang/genomeSeq/Data/PGP1_CGI_WGS. The data was from 3 different cell lines.

Processing Reference Data[edit]

The reference data information can be found here

Data Update Dec. 2015[edit]

It occurs to me this needs to be better updated, so I'm linking to a google doc.

Data Map Notebook Link
Hi-C Data Map
BAC Data Map Processing BAC Data
LFR Data Map Processing LFR Data
BAC+Hi-C Data Map BAC and Hi-C Data Processing
LFR + Hi-C Data Map LFR and Hi-C Data Processing
BAC+Hi-C+LFR Data Map LFR+BAC+Hi-C Data Processing
BEAGLE Data Map BEAGLE Data Processing

Hi-C Data[edit]

File Type System Path File(s)
VCF TSCC /oasis/tscc/scratch/sselvaraj/human_tissues/htissues/KZPGP1/snps/ pgp1f_hg19_vcf_fixedCompleteGenomics_withHeaders_final-het-final.modifiedINFO3.vcf
VCF-by chromosome TSCC /oasis/tscc/scratch/sselvaraj/human_tissues/htissues/KZPGP1/snps/ chr*/sorted.chr*.pgp1f_hg19_vcf_fixedCompleteGenomics_withHeaders_final-het-final.modifiedINFO3_clean_final.vcf
bam files-original TSCC /oasis/tscc/scratch/sselvaraj/human_tissues/htissues/KZPGP1/fastq/hi-c PGP1.Rep1.rg.nodup.final_PE_sort.bam
bam files-by chrom TSCC /oasis/tscc/scratch/sselvaraj/human_tissues/htissues/KZPGP1/fastq/hi-c chr*/PGP1.final.chr*.corrected.bam
HAIRS-raw TSCC /oasis/tscc/scratch/djacobse/haplotypeSeqData/HAIRS fragments.chr*.hic.txt
HAIRS-merge XXXX XXXX XXXX
haplotypes TSCC /oasis/tscc/scratch/djacobse/haplotypeSeqData/hapcut_results haplotype.chr*.hic.txt

Notes:

  • * chromosomes 1 to 22

Bacterial Artificial Chromosome (BAC) Data[edit]

File Type System Path File(s)
VCF-All Genome Miner /media/LTS_33T/KZ_LTS33T/PGP1_BacPool/filtered_vcf pgp1_variants_bac_all.Indx73to96.vcf
VCF-by chromosome XXXX XXXX XXXX
bam files-original Genome Miner /media/LTS_33T/KZ_LTS33T/PGP1_BacPool/fixed.bam PGP1_***BacPool_Indx**.fixed.bam
bam files-contigs Genome Miner /media/LTS_33T/KZ_LTS33T/PGP1_BacPool/hapcut/bacContigs Indx**.bacs.bam
bam files-by chrom Genome Miner /media/LTS_33T/KZ_LTS33T/PGP1_BacPool/hapcut/bams_by_chromosome chr*/chr*.Indx**.bam
HAIRS-raw Genome Miner /media/LTS_33T/KZ_LTS33T/PGP1_BacPool/hapcut/HAIRS chr*/chr*.Indx**.bam
HAIRS-merge Genome Miner /media/LTS_33T/KZ_LTS33T/PGP1_BacPool/hapcut/HAIRS chr*/fragments.chr*.bac.txt
haplotypes Genome Miner /media/LTS_33T/KZ_LTS33T/PGP1_BacPool/hapcut/haplotypes haplotype.chr*.bac.txt

Notes:

  • * chromosomes 1 to 22
  • ** indexes 73 to 96
  • *** 3 or 6, some 6 pools (85 and 92) also have a **.2.fixed.bam notation

SISSOR DATA[edit]

PGP1-21[edit]

File Type-PGP1_21 Path File(s)
VCF-by chrom /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/data_by_chamber chamber**/chamber**_pgp1_21.snps.raw.vcf
VCF-pgp1_21 /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/data_by_chamber pgp1.22.SISSOR.vcf
bam files-original /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/data_by_chamber chamber**_pgp1_21.BQSR.realigned.bam
bam files-contigs /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/contigs chamber**.contigs.bam
bam files-by chrom /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/bams_by_chromosome chr*/chr*.chamber**.bam
HAIRS-raw /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/HAIRS chr*/fragments.chr*.chamber**.txt
HAIRS-merge /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/HAIRS chr*/fragments.chr*.pgp1_21.txt
haplotypes /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/haplotypes haplotype.chr*.sissor.txt

Notes:

  • * chromosomes 1 to 22
  • ** chambers 1 to 24

PGP1-22[edit]

File Type-PGP1_22 Path File(s)
VCF-by chrom /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/data_by_chamber chamber**/pgp22_chamber**.snps.raw.vcf
VCF-pgp1_22 /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/data_by_chamber pgp1.22.SISSOR.vcf
bam files-original /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/data_by_chamber chamber**/pgp22_chamber*.BQSR.realigned.bam
bam files-contigs /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/contigs chamber**.contigs.bam
bam files-by chrom /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/bams_by_chromosome chr*/chr*.chamber*.bam
HAIRS-raw /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/HAIRS chr*/fragments.chr*.chamber*.txt
HAIRS-merge /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/HAIRS chr*/fragments.chr*.pgp1_21.txt
haplotypes /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/haplotypes haplotype.chr*.sissor.txt

Notes:

  • * chromosomes 1 to 22
  • ** chambers 1 to 24

Merge-SISSOR[edit]

File Type-SISSOR (PGP1-21,22) Path File(s)
VCF-PGP1 /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp1_21and22 pgp1_21and22.vcf
HAIRS-merge /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp1_21and22/HAIRS fragments.chr*.sissor.txt
haplotypes /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp1_21and22/haplotypes haplotype.chr*.sissor.txt

Notes:

  • * chromosomes 1 to 22