anvi-profile-blitz

FAST profiling of BAM files to get gene-, contig-, or genome-level coverage and detection stats. Unlike anvi-profile, which is another anvi'o program that can profile BAM files, this program is designed to be very quick and only report long-format files for various read recruitment statistics per item. Plase also see the program anvi-script-get-coverage-from-bam for recovery of data from BAM files without an anvi'o contigs database.

🔙 To the main page of anvi’o programs and artifacts.

Authors

A. Murat Eren (Meren)

WebEmailTwitterGithub

Iva Veseli

EmailTwitterGithub

Requires

bam-file contigs-db

Can use

collection-txt genes-of-interest-txt

Provides

bam-stats-txt discov-stats

Usage

This program produces a bam-stats-txt from one or more bam-file given a contigs-db. It is designed to serve people who only need to process read recruitment data stored in a bam-file to recover coverage and detection statistics as well as the number of mapped reads (along with other statistics) for their genes, contigs, and/or genomes. It will report what’s going on nicely with memory usage information and estimated time of completion:

There are other programs in anvi’o software ecosystem that are similar to this one:

• anvi-profile also takes a bam-file and profiles it. They both require a contigs-db. But while anvi-profile produces a single-profile-db for downstream analyses in anvi’o, anvi-profile-blitz produces text files for downstream analyses by the user (via R, Python, or other solutions). In contrast to anvi-profile, anvi-profile-blitz is orders of magnitude faster with similar memory usage.

• anvi-script-get-coverage-from-bam also takes a bam-file and profiles it. They both produce text output files. But while anvi-script-get-coverage-from-bam does not require a contigs-db, anvi-profile-blitz requires one to work. They will both run very rapidly, anvi-script-get-coverage-from-bam will work with a much smaller amount of memory.

Output files

For output file formats, please see bam-stats-txt.

Running

You can use this program with one or more BAM files to recover minimal or extended statistics for contigs or genes in a contigs-db.

Since the program will not be able to ensure the contigs-db was generated from the same contigs-fasta that was used for read recruitment that resulted in bam-files for analysis, you can make serious mistakes if you mix up your workflow and start profiling BAM files that have nothing to do with a contigs-db. If you make a mistake like that, in the best case scenario you will get an empty output file because the program will skip all contigs with non-matching name. In the worst case scenario you will get a file if some names in contigs-db incorrectly matches to some names in the bam-file. While this warning may be confusing, you can avoid all these if you use the SAME FASTA FILE both as reference for read recruitment and as input for anvi-gen-contigs-database.

Contigs mode, default output

Profile contigs, produce a default output:

This example is with a single BAM file, but you can also have multiple BAM files as a parameter by using wildcards,

or by providing multiple paths:

Contigs mode, minimal output

Profile contigs, produce a minimal output. This is the fastest option:

Genome/bin mode via collections

If you have a collection-txt, you can summarize coverage/detection statistics per bin/genome instead of per contig.

Bin-level detection and coverage are computed by treating all contigs in a bin as a single genome (nucleotide arrays are concatenated under the hood). With --report-minimal, these statistics are streamed to keep memory usage close to contig mode even for large bins.

Genes mode, default output

Instead of contigs, profile genes, produce a default output:

--report-minimal will behave the same, and produce minimal output.

Genes mode, on a subset of genes

You don’t want to profile all genes in your contigs-db? Then you should tell anvi’o which specific genes you are interested in by either providing a comma-separated list of gene-caller IDs:

Or by providing a genes-of-interest-txt file:

Modifying DisCov parameters

For genomes or contigs, anvi-profile-blitz can compute the Distribution of Coverage (DisCov) score alongside other coverage statistics. DisCov combines a spread score S (proportion of windows with coverage) and an evenness score E (proportion of covered bases within a fold-range of the median nonzero coverage). See discov-stats for a full description of the metric and parameters.

The default parameters are designed to work well across typical metagenomics data, but you can adjust them:

Window sizing for S

The spread score (S) is computed by dividing the input sequence into non-overlapping windows. By default, contig-level stats use a window length equal to 1% of the contig length (minimum 300 bp). Genome/bin-level stats (when using --collection-txt) use a fixed 1,000 bp window. You can override these defaults with the following parameters:

• --window-length INT — use a fixed window size in bp for all sequences
• --window-length-as-percentage FLOAT — set window length as a percentage of each sequence’s length
• --min-window-length INT — set a minimum window length floor when using percentage mode

Fold-range for E

The evenness score (E) counts bases with coverage between a lower and upper fold-multiple of the median nonzero coverage. The default fold-range endpoints are 0.5x and 2.0x. To adjust:

• --foldrange-lower FLOAT — lower bound (default: 0.5)
• --foldrange-upper FLOAT — upper bound (default: 2.0)

Combining S and E

You can adjust how S and E are combined by changing either their weights or the overall DisCov formula:

• --alpha FLOAT — weight of S relative to E, in [0, 1] (default: 0.5)
• --discov-formula STRING — linear (DisCov = αS + (1-α)E) or geometric (DisCov = S^α × E^(1-α)) (default: linear)

Window-level output

To inspect the per-window values used to compute S (useful for debugging or visualization), add the --gen-window-level-output flag:

This produces an additional file named OUTPUT-WINDOWS.txt with per-window start/stop positions, coverage presence, and base counts within the fold-range.

Note that computing DisCov is not compatible with --gene-mode or --report-minimal.

Performance

The memory use will be correlated linaerly with the size of the contigs-db, but once everything is loaded, the memory usage will not increase substantially over time.

With the flag --report-minimal, anvi-profile-blitz profiled on a laptop computer 100,000 contigs that contained 1 billion nts in 6 minutes and used ~300 Mb memory. This contigs database had 1.5 million genes, and memory usage increased to 1.7 Gb when anvi-profile-blitz run in --gene-mode. The flag --gene-mode does not change time complexity dramatically.

Anvi’o has this program because Emile Faure presented us with a challenge: Emile had a ~140 Gb anvi’o contigs-db that contained nearly 70 million contig sequences from over 200 single-assembled metagenomes, and wanted to learn the coverages of each gene in the contigs database in 200 metagenomes individually. Yet the combination of anvi-profile and anvi-summarize jobs would take more than 40 days to complete. Since all Emile needed was to learn the coverages from BAM files, we implemented anvi-profile-blitz to skip the profiling step. The run took 8 hours to compute and report coverage values for 175 million genes in 70 million contigs, and the memory use remained below 200 Gb.

Edit this file to update this information.

Command line interface

The following is the list of parameters and their descriptions in the latest version of anvi-profile-blitz.

The code that generates this web page ensures that this information is the latest available in 9-dev (eunice). Which means, in some cases the information in this section (generated automatically) might be more comprehensive or up-to-date compared to the descriptions above (written by humans).

Command line usage

Parameters and their descriptions

Additional Resources

Are you aware of resources that may help users better understand the utility of this program? Please feel free to edit this file on GitHub. If you are not sure how to do that, find the __resources__ tag in this file to see an example.