Editing
Sam:LabNotes/Microbiome-new/2011-5-8
Jump to navigation
Jump to search
Warning:
You are not logged in. Your IP address will be publicly visible if you make any edits. If you
log in
or
create an account
, your edits will be attributed to your username, along with other benefits.
Anti-spam check. Do
not
fill this in!
='''Repeat of 1,000 reads BLASTn taxonomic analysis from 04-25-'11 test'''= ==Background== *The taxonomic analysis on 4-25-'11 showed some inconsistent conclusions between blastn and bowtie mapping. *Although these could be due to many differences between two approaches, I am wondering if the sampling approach would affect the results too. *Dr. Zhang point out the concerns of -e setting in blastn, in which the default -e (10) setting might not be rigid enough. ==Objectives== *Test on proportional sampling size #Sampling 1000 reads for the lowest number group. #Sampling the proportional number of reads for other groups based on fold variance. *Test different blastn parameter settings: #Expectation value 10(default) => -e 0.01 #blastn DUST filter T(default) => F ===Test of -e parameters=== *No bad alignment structure was found using default -e 10 setting. *No difference on output results was observed by switching between -e 0.01 and -e 10. **It is probably due to that I used the blastn setting that only report the best hits. **I will switch to use -e 0.01 anyway for future blastn. ===Test of -F parameters=== *Turning off DUST-filter(FF) increase 10% hit compared to filter-on setting(FT) *However some data have problems to be run with FF setting. Error message: "Program failed, try executing the command manually.". *The problem sometime could be solved by changing the file name. *I need to do more tests to figure out why, but current I decide to keep using FT(default) for all of my blastn analysis. ==Real tests of ILL- and NX-library data== ===Decide sampling size=== [[File:050811_proportional_sampling.png|px300]] ===Shuffle sampling the fastq reads with proprotional sample size=== *Use the perl program Chris help me to modify to do sampling. The new program fastq2Tab2fas_shuf_sampling-cw3 need to input the '''number of total fastq read''' and '''desired sample-size'''. *Fastq-> Tabular -> Shuffling -> Downsampled -> Fasta -> (ready for blastn) Command example: samchiang@genemapster:~/Scripts$ perl fastq2Tab2fas_shuf_sampling-cw3 /media/disk-1/samchiang/Raw-Read-Complete/101123_HL083/HC-MDA-Ecoli-SAGs-Ind1to8-PE/s_4_1and2_ID1.txt 15836502 2984 & *In total 8 (ID1~ID8) x 2 (raw reads, failed-to-align reads) x 2 (ILL lib, NX lib) = 32 fastq data were re-sampled. The output ===blstn with -e 0.01 and FT settings against nt=== *blastn setting -K 1 -F T -m 0 -a 2 -v 1 -b 1 -e 0.01 *blastn setting -K 1 -F T -m 0 -a 2 -v 1 -b 1 -e 0.01 samchiang@genemapster:/media/disk-2/ncbi-blast-2.2.23+/bin$ ./legacy_blast.pl blastall -i /media/disk-1/samchiang/Sub_tests_folder_imp/6_10000reads_subset_analysis/2nd_analysis_with_propotional_sampling/2nd_test_new/NX_lib_failed/s_4_pe_ID8.failed.txt.shuf-1265reads.fa -d /media/disk-2/data/nt -K 1 -m 0 -a 2 -v 1 -b 1 -e 0.01 -o /media/disk-1/samchiang/Sub_tests_folder_imp/6_10000reads_subset_analysis/2nd_analysis_with_propotional_sampling/2nd_test_new/NX_lib_failed/s_4_pe_ID8.failed.txt.shuf-1265reads.fa.blastn.FT.txt -p /media/disk-2/ncbi-blast-2.2.23+/bin/blastn & ==Using MEGAN analysis== *Using MEGAN "compare" function to parse blastn taxonomic results. #Run MEGAN analysis on individual data #Import all of the .rma files (MEGAN output format) for comparison #Use normalized count setting (actually it won't matter if we chose to use % presentation later) #Select the most specific node on taxa rank for plotting. #Plot results in stacked bar chart and present in % ratio. ==Results== '''Previous bowtie mapping results summary''' [[File:050311_Ecoli_SAG_bowtie_mapping_summary.png|1100px]] [[File:Ecoli_SAG_raw_1000reads_up_ILL_lib-a.png|800px]] [[File:Ecoli_SAG_raw_1000reads_up_NX_lib-a.png|800px]] [[File:Ecoli_SAG_failed2align_1000reads_up_ILL_lib-a.png|800px]] [[File:Ecoli_SAG_failed2align_1000reads_up_NX_lib-a.png|800px]] ==Discussion== *Questions need to verify #Difference between fixed-sampling vs. proportional sampling #Difference between proportional sampling vs. bowtie mapping *Be careful that the colors represents E.coli and No hit are reversed in raw-reads chart and in failed-to-align reads charts. *I found that the libraries with higher E.coli abundance also showed higher E.coli abundance in the failed-to-align reads(bowtie mapping). **I am wondering why? But this phenomena implies that bowtie missed a lot of E.coli reads in all libraries evenly, so the missed showed same abundance of E.coli reads in failed-to-align pools. **Unfortunately, it's not working very well using blastn to estimate how many E.coli reads failed in bowtie since the blastn(using nt reference) can't be as specific as bowtie (using E.coli K-12 template). **However, the E.coli reads identified by blastn in failed-to-align pools showed about [[File:051211-Ecoli_abundance_ILL.PNG|500px]] [[File:051211-Ecoli_abundance_NX.PNG|500px]] ==Next== #Find out a way to improve genome coverage by decrease the failed-to-align E.coli reads: ##Increase error tolerance in bowtie. ##Test BWA alignment. Check genome coverage. Play with parameters settings. ##Redo genome coverage calculation using -m 10 bowtie #Is it necessary to do taxanomic analysis on unknown reads that fallen in no-hit category? ##I can blast them to customized database? #Test if the missed E.coli reads are critical on genome coverage. (Probably not. Since the genome coverage is associated with amplification bias. Libraries with higher E.coli reads don't necessary showed higher genome coverage). #Since the E.coli SAG still contains lots of non-specific contamination. ==Additional idea== *The E.coli SAG data could be a good dataset for testing binning accuracy. #Objective: demonstrate genome binning could be used and necessary to remove non-specific contamination in SAG samples.It will be great if genome binning could be done on raw-reads without de novo assembly. ##E.coli SOAPdenovo (K27M3) -> validation by mapping to E.coli-12 reference ##Contigs tetramer feq. count ##Clustering -> Treeview building ##blastn validation -> grow binned contigs ##Validation of clustered contigs by mapping to E.coli-K12 reference.
Summary:
Please note that all contributions to ZhangLabWiki may be edited, altered, or removed by other contributors. If you do not want your writing to be edited mercilessly, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource (see
ZhangLabWiki:Copyrights
for details).
Do not submit copyrighted work without permission!
Cancel
Editing help
(opens in new window)
Navigation menu
Personal tools
Not logged in
Talk
Contributions
Create account
Log in
Namespaces
Page
Discussion
English
Views
Read
Edit
View history
More
Search
Navigation
Main Page
Current events
Recent changes
Random page
Investigators
Matt Cai
Song Chen
Eric Chu
Dinh Diep
Elizabeth Duong
Shicheng Guo
Alan Fung
Daniel Jacobsen
Blue Lake
Huy Lam
Alice Li
Andrew Richards
Brandon Sos
Chris Wei
Yan Wu
Kun Zhang
Tools
What links here
Related changes
Special pages
Page information