Chris:LabNotes/Microbiome/2013/2013-5-14: Difference between revisions
Jump to navigation
Jump to search
>Cjwei |
>Cjwei |
||
(4 intermediate revisions by the same user not shown) | |||
Line 15: | Line 15: | ||
| | | | ||
V | V | ||
Validation of hybrid assembly (assembly statistics, core genes search, assembly errors) | Validation of hybrid assembly (assembly statistics, core genes search, assembly errors) | ||
*A more detailed plan can be found in: [[File:Moleculo_Data_Analysis_Plan_5_14_2013.docx]] | *A more detailed plan can be found in: [[File:Moleculo_Data_Analysis_Plan_5_14_2013.docx]] | ||
==Procedure== | |||
===0_Illumina_Test_correction=== | |||
*Triton Folder: </projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Hybrid_Assembly_5_14_2013/0_Illumina_Test_correction/> | |||
*In order to first start with the test assembly of small Illumina dataset, we first want to perform correction of reads and test whether correction will help improve assembly | |||
*We are using the correction software used in Assemblathon2 by SOAPdenovo: (found in </home/cjwei/software/SOAPdenovo2-src-r223/SOAPdenovoPipeline4BirdInAssemblathon2/> | |||
**kmerfreq | |||
**correct_error | |||
*Steps for error correction: | |||
0) Need to gzip read files prior to error correction (can also use .gz files for assembly in SOAPdenovo) | |||
1)Create read list for correction <reads2cor.lst>, which is just the file location of reads. If have paired read files, must put each file consecutively. | |||
/projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Illumina_raw/HB_gDNA200b_121224L1-4_R1_filtered.fastq.gz | |||
/projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Illumina_raw/HB_gDNA200b_121224L1-4_R2_filtered.fastq.gz | |||
/projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Illumina_raw/HB_gDNA2kb_NoIndex_L005_R1_001_filtered.fastq.gz | |||
/projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Illumina_raw/HB_gDNA2kb_NoIndex_L005_R2_001_filtered.fastq.gz | |||
2) Run commands (need to run kmerfreq then correct_error scripts in order) <1_correction> | |||
*Parameters set based on those used in assemblathon2: | |||
**kmerfreq | |||
***k=k-mer size, 13-27 (used 27) | |||
***l=input read file list | |||
***p=output prefix | |||
***t=cores used (32 cores per node) | |||
***i=set initial size of hash table (used 400000000) | |||
***L=max read length (used 200 in order to ensure enough memory usuage set aside) | |||
**correct_error | |||
***k=k-mer size, must be same as kmerfreq (used 27) | |||
***l=low frequency cutoff (used 2) | |||
***a=set whether remain all the data or not (used 1) | |||
***e=set whether trim suspicious region at end of read according to Q-value or not (used 1) | |||
***w=set whether trim error bases with Q-value <=2 instead of correct (used 1, yes) | |||
***q=set quality threshold (35) | |||
***t=number of cores (used 32) | |||
***j=set whether convert read1 and read 2 corrected file into pair-end file (used 1, yes) | |||
/home/cjwei/software/SOAPdenovo2-src-r223/SOAPdenovoPipeline4BirdInAssemblathon2/kmerfreq -k 27 -l /projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Hybrid_Assembly_5_14_2013/1_Illumina_Test_Assembly/reads2cor.lst -p /projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Hybrid_Assembly_5_14_2013/1_Illumina_Test_Assembly/test_assembly -t 32 -i 400000000 -L 200 >/projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Hybrid_Assembly_5_14_2013/1_Illumina_Test_Assembly/kmerfreq.log | |||
/home/cjwei/software/SOAPdenovo2-src-r223/SOAPdenovoPipeline4BirdInAssemblathon2/correct_error -k 27 -l 2 -a 0 -e 1 -w 1 -q 35 -t 32 -j 1 /projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Hybrid_Assembly_5_14_2013/1_Illumina_Test_Assembly/test_assembly.freq.gz /projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Hybrid_Assembly_5_14_2013/1_Illumina_Test_Assembly/reads2cor.lst >/projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Hybrid_Assembly_5_14_2013/1_Illumina_Test_Assembly/correct.log | |||
3) Output can be found in original raw read folder as .cor.pair files because program combines the paired read files into one </projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Illumina_raw/> | |||
HB_gDNA200b_121224L1-4_R1_filtered.fastq.gz.cor.pair | |||
HB_gDNA2kb_NoIndex_L005_R1_001_filtered.fastq.gz.cor.pair | |||
===1a-c_Illumina_Test_Assembly=== | |||
*Continued on <http://genome-tech.ucsd.edu/LabNotes/index.php/Chris:LabNotes/Microbiome/2013/2013-5-15> |
Latest revision as of 19:00, 15 May 2013
Moleculo Data Analysis Part 1[edit]
Background[edit]
- We are integrating Illumina (short read) and Moleculo (long read) data to assemble the oncomelania genome
- The overall plan that we are using is:
Test assembly of small Illumina dataset | V Validation of assembly (using assembly statistics and core genes search) | V Assembly of whole Illumina dataset | V Integration of Moleculo with Illumina reads | V Validation of hybrid assembly (assembly statistics, core genes search, assembly errors)
- A more detailed plan can be found in: File:Moleculo Data Analysis Plan 5 14 2013.docx
Procedure[edit]
0_Illumina_Test_correction[edit]
- Triton Folder: </projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Hybrid_Assembly_5_14_2013/0_Illumina_Test_correction/>
- In order to first start with the test assembly of small Illumina dataset, we first want to perform correction of reads and test whether correction will help improve assembly
- We are using the correction software used in Assemblathon2 by SOAPdenovo: (found in </home/cjwei/software/SOAPdenovo2-src-r223/SOAPdenovoPipeline4BirdInAssemblathon2/>
- kmerfreq
- correct_error
- Steps for error correction:
0) Need to gzip read files prior to error correction (can also use .gz files for assembly in SOAPdenovo)
1)Create read list for correction <reads2cor.lst>, which is just the file location of reads. If have paired read files, must put each file consecutively.
/projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Illumina_raw/HB_gDNA200b_121224L1-4_R1_filtered.fastq.gz /projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Illumina_raw/HB_gDNA200b_121224L1-4_R2_filtered.fastq.gz /projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Illumina_raw/HB_gDNA2kb_NoIndex_L005_R1_001_filtered.fastq.gz /projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Illumina_raw/HB_gDNA2kb_NoIndex_L005_R2_001_filtered.fastq.gz
2) Run commands (need to run kmerfreq then correct_error scripts in order) <1_correction>
- Parameters set based on those used in assemblathon2:
- kmerfreq
- k=k-mer size, 13-27 (used 27)
- l=input read file list
- p=output prefix
- t=cores used (32 cores per node)
- i=set initial size of hash table (used 400000000)
- L=max read length (used 200 in order to ensure enough memory usuage set aside)
- correct_error
- k=k-mer size, must be same as kmerfreq (used 27)
- l=low frequency cutoff (used 2)
- a=set whether remain all the data or not (used 1)
- e=set whether trim suspicious region at end of read according to Q-value or not (used 1)
- w=set whether trim error bases with Q-value <=2 instead of correct (used 1, yes)
- q=set quality threshold (35)
- t=number of cores (used 32)
- j=set whether convert read1 and read 2 corrected file into pair-end file (used 1, yes)
- kmerfreq
/home/cjwei/software/SOAPdenovo2-src-r223/SOAPdenovoPipeline4BirdInAssemblathon2/kmerfreq -k 27 -l /projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Hybrid_Assembly_5_14_2013/1_Illumina_Test_Assembly/reads2cor.lst -p /projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Hybrid_Assembly_5_14_2013/1_Illumina_Test_Assembly/test_assembly -t 32 -i 400000000 -L 200 >/projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Hybrid_Assembly_5_14_2013/1_Illumina_Test_Assembly/kmerfreq.log
/home/cjwei/software/SOAPdenovo2-src-r223/SOAPdenovoPipeline4BirdInAssemblathon2/correct_error -k 27 -l 2 -a 0 -e 1 -w 1 -q 35 -t 32 -j 1 /projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Hybrid_Assembly_5_14_2013/1_Illumina_Test_Assembly/test_assembly.freq.gz /projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Hybrid_Assembly_5_14_2013/1_Illumina_Test_Assembly/reads2cor.lst >/projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Hybrid_Assembly_5_14_2013/1_Illumina_Test_Assembly/correct.log
3) Output can be found in original raw read folder as .cor.pair files because program combines the paired read files into one </projects/zhang-lab/cjwei/130419_Oncomelania_Ilumina_Moleculo/Illumina_raw/>
HB_gDNA200b_121224L1-4_R1_filtered.fastq.gz.cor.pair HB_gDNA2kb_NoIndex_L005_R1_001_filtered.fastq.gz.cor.pair