Daniel:Notebook/Haplotyping/DataMap: Difference between revisions

From ZhangLabWiki
Jump to navigation Jump to search
>Djacobse
>Djacobse
 
(22 intermediate revisions by the same user not shown)
Line 4: Line 4:


This page contains the data map for the haplotyping project, specifically tables that outline locations of each of the data types.
This page contains the data map for the haplotyping project, specifically tables that outline locations of each of the data types.
==Reference Data==
The reference data is the data used to create the "true" VCFs, since HAPCUT requires a true vcf file as input which informs it where the relevant sites are.  The original PGP1 data is in genome miner in the directory /home/kunzhang/genomeSeq/Data/PGP1_CGI_WGS.  The data was from 3 different cell lines.
===Processing Reference Data===
The reference data information can be found [[Daniel:Notebook/Haplotyping/MergeVCF|here]]
===Data Update Dec. 2015===
It occurs to me this needs to be better updated, so I'm linking to a google doc.
{| class="wikitable" <hiddentext>generated with [[:de:Wikipedia:Helferlein/VBA-Macro for EXCEL tableconversion]] V1.8</hiddentext>
|- font-size:12pt;font-weight:bold"
| width="600" height="46" | Data Map
| width="600" | Notebook Link
|- style="font-size:12pt"  valign="bottom"
| [https://docs.google.com/spreadsheets/d/1gStTrST8MaWWlHeXMeWXVpjsNvRn8HUdeC4gSUbnPxQ/edit#gid=0 Hi-C Data Map]
|
|- style="font-size:12pt"  valign="bottom"
| [https://docs.google.com/spreadsheets/d/1gStTrST8MaWWlHeXMeWXVpjsNvRn8HUdeC4gSUbnPxQ/edit#gid=1124859722 BAC Data Map]
| [[Daniel:Notebook/Haplotyping/BACData|Processing BAC Data]]
|- style="font-size:12pt"  valign="bottom"
| [https://docs.google.com/spreadsheets/d/1gStTrST8MaWWlHeXMeWXVpjsNvRn8HUdeC4gSUbnPxQ/edit#gid=1059932227 LFR Data Map]
| [[Daniel:Notebook/Haplotyping/LFRData|Processing LFR Data]]
|- style="font-size:12pt"  valign="bottom"
| [https://docs.google.com/spreadsheets/d/1gStTrST8MaWWlHeXMeWXVpjsNvRn8HUdeC4gSUbnPxQ/edit#gid=11247187 BAC+Hi-C Data Map]
| [[Daniel:Notebook/Haplotyping/BAC-HiC|BAC and Hi-C Data Processing]]
|- style="font-size:12pt"  valign="bottom"
| [https://docs.google.com/spreadsheets/d/1gStTrST8MaWWlHeXMeWXVpjsNvRn8HUdeC4gSUbnPxQ/edit#gid=2147175569 LFR + Hi-C Data Map]
| [[Daniel:Notebook/Haplotyping/LFR-HiC|LFR and Hi-C Data Processing]]
|- style="font-size:12pt"  valign="bottom"
| [https://docs.google.com/spreadsheets/d/1gStTrST8MaWWlHeXMeWXVpjsNvRn8HUdeC4gSUbnPxQ/edit#gid=676014356 BAC+Hi-C+LFR Data Map]
| [[Daniel:Notebook/Haplotyping/LFR-BAC-HiC|LFR+BAC+Hi-C Data Processing]]
|- style="font-size:12pt"  valign="bottom"
| [https://docs.google.com/spreadsheets/d/1gStTrST8MaWWlHeXMeWXVpjsNvRn8HUdeC4gSUbnPxQ/edit#gid=1920656431 BEAGLE Data Map]
| [[Daniel:Notebook/Haplotyping/BEAGLE|BEAGLE Data Processing]]
|}


==Hi-C Data==
==Hi-C Data==
Line 57: Line 104:


|}
|}
'''Notes:'''
* * chromosomes 1 to 22


==Bacterial Artificial Chromosome (BAC) Data==
==Bacterial Artificial Chromosome (BAC) Data==
Line 85: Line 136:
  | PGP1_***BacPool_Indx**.fixed.bam
  | PGP1_***BacPool_Indx**.fixed.bam


|- style="font-size:12pt"  valign="bottom"
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | bam files-contigs
| height="15" | bam files-contigs
  | Genome Miner
  | Genome Miner
Line 91: Line 142:
  | Indx**.bacs.bam
  | Indx**.bacs.bam


|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
|- style="font-size:12pt"  valign="bottom"
| height="15" | bam files-by chrom
| height="15" | bam files-by chrom
  | Genome Miner
  | Genome Miner
Line 97: Line 148:
  | chr*/chr*.Indx**.bam
  | chr*/chr*.Indx**.bam


|- style="font-size:12pt"  valign="bottom"
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | HAIRS-raw
| height="15" | HAIRS-raw
  | Genome Miner
  | Genome Miner
Line 103: Line 154:
  | chr*/chr*.Indx**.bam
  | chr*/chr*.Indx**.bam


|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
|- style="font-size:12pt"  valign="bottom"
| height="15" | HAIRS-merge
| height="15" | HAIRS-merge
  | Genome Miner
  | Genome Miner
Line 109: Line 160:
  | chr*/fragments.chr*.bac.txt
  | chr*/fragments.chr*.bac.txt


|- style="font-size:12pt"  valign="bottom"
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | haplotypes
| height="15" | haplotypes
  | Genome Miner
  | Genome Miner
Line 122: Line 173:
* ** indexes 73 to 96
* ** indexes 73 to 96
* *** 3 or 6, some 6 pools (85 and 92) also have a **.2.fixed.bam notation
* *** 3 or 6, some 6 pools (85 and 92) also have a **.2.fixed.bam notation
==SISSOR DATA==
===PGP1-21===
{| class="wikitable" <hiddentext>generated with [[:de:Wikipedia:Helferlein/VBA-Macro for EXCEL tableconversion]] V1.8</hiddentext>
|- style="background-color:#D8E4BC;font-size:12pt;font-weight:bold"
| width="159" height="56" | File Type-PGP1_21
| width="545" | Path
| width="433" | File(s)
|- style="font-size:12pt"  valign="bottom"
| height="15" | VCF-by chrom
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/data_by_chamber
| chamber**/chamber**_pgp1_21.snps.raw.vcf
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | VCF-pgp1_21
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/data_by_chamber
| pgp1.22.SISSOR.vcf
|- style="font-size:12pt"  valign="bottom"
| height="15" | bam files-original
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/data_by_chamber
| chamber**_pgp1_21.BQSR.realigned.bam
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | bam files-contigs
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/contigs
| chamber**.contigs.bam
|- style="font-size:12pt"  valign="bottom"
| height="15" | bam files-by chrom
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/bams_by_chromosome
| chr*/chr*.chamber**.bam
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | HAIRS-raw
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/HAIRS
| chr*/fragments.chr*.chamber**.txt
|- style="font-size:12pt"  valign="bottom"
| height="15" | HAIRS-merge
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/HAIRS
| chr*/fragments.chr*.pgp1_21.txt
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | haplotypes
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/haplotypes
| haplotype.chr*.sissor.txt
|}
'''Notes:'''
* * chromosomes 1 to 22
* ** chambers 1 to 24
===PGP1-22===
{| class="wikitable" <hiddentext>generated with [[:de:Wikipedia:Helferlein/VBA-Macro for EXCEL tableconversion]] V1.8</hiddentext>
|- style="background-color:#D8E4BC;font-size:12pt;font-weight:bold"
| width="159" height="38" | File Type-PGP1_22
| width="545" | Path
| width="433" | File(s)
|- style="font-size:12pt"  valign="bottom"
| height="15" | VCF-by chrom
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/data_by_chamber
| chamber**/pgp22_chamber**.snps.raw.vcf
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | VCF-pgp1_22
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/data_by_chamber
| pgp1.22.SISSOR.vcf
|- style="font-size:12pt"  valign="bottom"
| height="15" | bam files-original
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/data_by_chamber
| chamber**/pgp22_chamber*.BQSR.realigned.bam
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | bam files-contigs
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/contigs
| chamber**.contigs.bam
|- style="font-size:12pt"  valign="bottom"
| height="15" | bam files-by chrom
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/bams_by_chromosome
| chr*/chr*.chamber*.bam
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | HAIRS-raw
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/HAIRS
| chr*/fragments.chr*.chamber*.txt
|- style="font-size:12pt"  valign="bottom"
| height="15" | HAIRS-merge
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/HAIRS
| chr*/fragments.chr*.pgp1_21.txt
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | haplotypes
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/haplotypes
| haplotype.chr*.sissor.txt
|}
'''Notes:'''
* * chromosomes 1 to 22
* ** chambers 1 to 24
===Merge-SISSOR===
{| class="wikitable" <hiddentext>generated with [[:de:Wikipedia:Helferlein/VBA-Macro for EXCEL tableconversion]] V1.8</hiddentext>
|- style="background-color:#D8E4BC;font-size:12pt;font-weight:bold"
| width="159" height="36" | File Type-SISSOR (PGP1-21,22)
| width="545"  | Path
| width="433"  | File(s)
|- style="font-size:12pt"  valign="bottom"
| height="15" | VCF-PGP1
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp1_21and22
| pgp1_21and22.vcf
|- style="background-color:#D9D9D9;font-size:12pt"  valign="bottom"
| height="15" | HAIRS-merge
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp1_21and22/HAIRS
| fragments.chr*.sissor.txt
|- style="font-size:12pt"  valign="bottom"
| height="15" | haplotypes
| /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp1_21and22/haplotypes
| haplotype.chr*.sissor.txt
|}
'''Notes:'''
* * chromosomes 1 to 22

Latest revision as of 18:11, 15 December 2015

Data Map[edit]

Back to Notebook

This page contains the data map for the haplotyping project, specifically tables that outline locations of each of the data types.

Reference Data[edit]

The reference data is the data used to create the "true" VCFs, since HAPCUT requires a true vcf file as input which informs it where the relevant sites are. The original PGP1 data is in genome miner in the directory /home/kunzhang/genomeSeq/Data/PGP1_CGI_WGS. The data was from 3 different cell lines.

Processing Reference Data[edit]

The reference data information can be found here

Data Update Dec. 2015[edit]

It occurs to me this needs to be better updated, so I'm linking to a google doc.

Data Map Notebook Link
Hi-C Data Map
BAC Data Map Processing BAC Data
LFR Data Map Processing LFR Data
BAC+Hi-C Data Map BAC and Hi-C Data Processing
LFR + Hi-C Data Map LFR and Hi-C Data Processing
BAC+Hi-C+LFR Data Map LFR+BAC+Hi-C Data Processing
BEAGLE Data Map BEAGLE Data Processing

Hi-C Data[edit]

File Type System Path File(s)
VCF TSCC /oasis/tscc/scratch/sselvaraj/human_tissues/htissues/KZPGP1/snps/ pgp1f_hg19_vcf_fixedCompleteGenomics_withHeaders_final-het-final.modifiedINFO3.vcf
VCF-by chromosome TSCC /oasis/tscc/scratch/sselvaraj/human_tissues/htissues/KZPGP1/snps/ chr*/sorted.chr*.pgp1f_hg19_vcf_fixedCompleteGenomics_withHeaders_final-het-final.modifiedINFO3_clean_final.vcf
bam files-original TSCC /oasis/tscc/scratch/sselvaraj/human_tissues/htissues/KZPGP1/fastq/hi-c PGP1.Rep1.rg.nodup.final_PE_sort.bam
bam files-by chrom TSCC /oasis/tscc/scratch/sselvaraj/human_tissues/htissues/KZPGP1/fastq/hi-c chr*/PGP1.final.chr*.corrected.bam
HAIRS-raw TSCC /oasis/tscc/scratch/djacobse/haplotypeSeqData/HAIRS fragments.chr*.hic.txt
HAIRS-merge XXXX XXXX XXXX
haplotypes TSCC /oasis/tscc/scratch/djacobse/haplotypeSeqData/hapcut_results haplotype.chr*.hic.txt

Notes:

  • * chromosomes 1 to 22

Bacterial Artificial Chromosome (BAC) Data[edit]

File Type System Path File(s)
VCF-All Genome Miner /media/LTS_33T/KZ_LTS33T/PGP1_BacPool/filtered_vcf pgp1_variants_bac_all.Indx73to96.vcf
VCF-by chromosome XXXX XXXX XXXX
bam files-original Genome Miner /media/LTS_33T/KZ_LTS33T/PGP1_BacPool/fixed.bam PGP1_***BacPool_Indx**.fixed.bam
bam files-contigs Genome Miner /media/LTS_33T/KZ_LTS33T/PGP1_BacPool/hapcut/bacContigs Indx**.bacs.bam
bam files-by chrom Genome Miner /media/LTS_33T/KZ_LTS33T/PGP1_BacPool/hapcut/bams_by_chromosome chr*/chr*.Indx**.bam
HAIRS-raw Genome Miner /media/LTS_33T/KZ_LTS33T/PGP1_BacPool/hapcut/HAIRS chr*/chr*.Indx**.bam
HAIRS-merge Genome Miner /media/LTS_33T/KZ_LTS33T/PGP1_BacPool/hapcut/HAIRS chr*/fragments.chr*.bac.txt
haplotypes Genome Miner /media/LTS_33T/KZ_LTS33T/PGP1_BacPool/hapcut/haplotypes haplotype.chr*.bac.txt

Notes:

  • * chromosomes 1 to 22
  • ** indexes 73 to 96
  • *** 3 or 6, some 6 pools (85 and 92) also have a **.2.fixed.bam notation

SISSOR DATA[edit]

PGP1-21[edit]

File Type-PGP1_21 Path File(s)
VCF-by chrom /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/data_by_chamber chamber**/chamber**_pgp1_21.snps.raw.vcf
VCF-pgp1_21 /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/data_by_chamber pgp1.22.SISSOR.vcf
bam files-original /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/data_by_chamber chamber**_pgp1_21.BQSR.realigned.bam
bam files-contigs /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/contigs chamber**.contigs.bam
bam files-by chrom /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/bams_by_chromosome chr*/chr*.chamber**.bam
HAIRS-raw /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/HAIRS chr*/fragments.chr*.chamber**.txt
HAIRS-merge /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/HAIRS chr*/fragments.chr*.pgp1_21.txt
haplotypes /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/haplotypes haplotype.chr*.sissor.txt

Notes:

  • * chromosomes 1 to 22
  • ** chambers 1 to 24

PGP1-22[edit]

File Type-PGP1_22 Path File(s)
VCF-by chrom /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/data_by_chamber chamber**/pgp22_chamber**.snps.raw.vcf
VCF-pgp1_22 /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/data_by_chamber pgp1.22.SISSOR.vcf
bam files-original /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/data_by_chamber chamber**/pgp22_chamber*.BQSR.realigned.bam
bam files-contigs /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp21/contigs chamber**.contigs.bam
bam files-by chrom /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/bams_by_chromosome chr*/chr*.chamber*.bam
HAIRS-raw /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/HAIRS chr*/fragments.chr*.chamber*.txt
HAIRS-merge /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/HAIRS chr*/fragments.chr*.pgp1_21.txt
haplotypes /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp22/haplotypes haplotype.chr*.sissor.txt

Notes:

  • * chromosomes 1 to 22
  • ** chambers 1 to 24

Merge-SISSOR[edit]

File Type-SISSOR (PGP1-21,22) Path File(s)
VCF-PGP1 /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp1_21and22 pgp1_21and22.vcf
HAIRS-merge /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp1_21and22/HAIRS fragments.chr*.sissor.txt
haplotypes /media/LTS_15T/DEJ_LTS/microfluidics_data/pgp1_21and22/haplotypes haplotype.chr*.sissor.txt

Notes:

  • * chromosomes 1 to 22