discov-stats

TXT

A TXT-type anvi’o artifact. This artifact is typically generated, used, and/or exported by anvi’o (and not provided by the user)..

🔙 To the main page of anvi’o programs and artifacts.

Provided by

anvi-profile-blitz

Can be provided by

anvi-summarize

Required by

There are no anvi’o tools that require this artifact directly, which means it is most likely an end-product for the user.

Description

The distribution of coverage (discov) score is designed to capture the presence-absence of a contig or genome (‘bin’) in a metagenomic sample, as an alternative to using arbitrary detection thresholds for this purpose. Discov ranges from 0 to 1 and quantifies how well a sequence is covered by reads — not just how many bases have coverage, but whether that coverage is spread across the sequence as much as we would expect given the number of mapped reads and their total alignment length, and whether it is roughly even in depth. To achieve this, discov incorporates three metrics:

  • S (spread): the proportion of non-overlapping windows across the sequence that include bases with at least 1X coverage, normalized by the proportion we would expect if the same reads were placed randomly on the sequence.
  • E (evenness): how similar the mean coverage depths of the covered windows are to each other, computed from their coefficient of variation (CV).
  • D (detection): the proportion of bases with at least 1X coverage.

S and E are first combined into a spread-evenness score C, using either a geometric (the default) or a linear formula:

  • Geometric: C = S^α × E^(1-α)
  • Linear: C = αS + (1-α)E

where α is a value between 0 and 1 that controls how much weight is given to spread (S) relative to evenness (E). By default α = 0.5 (equal weighting).

Detection is then added on top of C, and the final score is capped at 1:

discov = min(1, C + D^2)

Squaring detection means that high detection gives a large boost, while low detection (which does not necessarily mean absence) contributes very little.

When discov values are reported – for instance, in bam-stats-txt – the number of windows, the observed and expected spread, the normalized spread, E, and C are individually reported in addition to the overall discov score.

Technical details & parameters

Here you’ll find a description on how discov is computed. You can modify several aspects of the discov calculation, and the parameters for each step are mentioned when relevant below.

Programs that produce these stats

anvi-profile-blitz computes discov scores as part of its default output (but not with --report-minimal or --gene-mode). anvi-summarize computes discov scores for each bin and each contig within each bin when the --report-discov flag is provided.

Input data

Discov is computed based on per-nucleotide coverage arrays for individual contigs and on the number of reads mapping to each contig. When computing scores at the genome-level, the arrays for all contigs within a given genome (or more generically, a given bin) are concatenated in alphabetical order of contig names, and the number of reads is summed across contigs. Note that the concatenation of per-contig coverage arrays means that the windows can span across contig boundaries within the genome.

anvi-profile-blitz obtains the coverage arrays and the number of reads directly from a bam-file. anvi-summarize obtains the coverage arrays from the auxiliary database file (AUXILIARY-DATA.db) produced by anvi-profile, and the number of reads from the profile-db.

Differences between anvi-profile-blitz and anvi-summarize The numbers reported by either strategy on the same dataset are usually the same, with the following exceptions:

  • when working with databases generated by anvi-profile with a nonzero --min-contig-length parameter, anvi-summarize will compute discov scores based on the filtered contigs. anvi-profile-blitz has no filtering options and will work with all contigs in the input bam-file.
  • when some contigs have extremely high coverage. The auxiliary database stores per-nucleotide coverage values up to a maximum of 65,535X, and higher values are capped. So for sequences with extremely high coverage, discov values from anvi-summarize can be slightly different from those reported by anvi-profile-blitz, which uses the exact coverage depths reported in the input BAM files. In practice, this mostly affects E, since capping the most highly-covered windows makes the window depths look slightly more even.

anvi-profile assigns each read to a single split (the one containing most of its aligned bases), and stores the number of reads per split in the profile-db. Profile databases generated with older versions of anvi’o do not include these counts, in which case anvi-summarize will tell you that it can not compute discov. If you have old profiles and you want to compute discov scores, you will need to find (or remake) the corresponding BAM files and then either use anvi-profile-blitz or re-run anvi-profile to fill in the read count data tables before running anvi-summarize.

Windows and window length

We divide each sequence (contig or genome) into non-overlapping windows to compute both S and E. The window size affects the sensitivity of these metrics. We want the windows to be long enough to capture meaningful clusters of mapped reads (hence, longer than typical read lengths), but short enough to effectively assess the distribution of these mapped reads along the sequence.

There are two strategies for setting the window length:

  • Fixed window length (--window-length): a constant window size in bp applied to all sequences.
  • Percentage-based window length (--window-length-as-percentage): the window size is set as a percentage of each sequence’s length, with a minimum length floor (--min-window-length, 300 bp by default) and a maximum length cap (--max-window-length, 10,000 bp by default). Set either of these to 0 to remove the corresponding limit.

If you don’t explicitly set any of these parameters, the default sizing strategy depends on the context. For genome-level stats, the default is a fixed 10,000 bp window. For contig-level stats, a percentage-based window (1% of contig length, with a minimum of 300 bp and a maximum of 10,000 bp) is used to better accommodate contigs of varying sizes. In anvi-summarize, if you set a window length parameter yourself, it applies to both the genome-level and the contig-level stats.

If a contig (or genome) is shorter than the window length, the sequence will not be divided; that is, the entire sequence will be a single window. In this case, the observed spread collapses to a binary value (does the sequence have any coverage, or not), and E will be 0 (see below). In most cases, anvi’o will warn you if sequences are shorter than the computed window length, and regardless you should be able to see the number of windows per sequence in the output files.

Discov scores computed using different window lengths are not comparable to each other. For instance, you cannot directly compare the discov value of a very short contig to that of a very long contig when using percentage-based window lengths. The min/max window length parameters can help to reduce the range of window sizes used in assemblies with a wide contig size distribution, but you should still use caution when interpreting scores across contigs. If you need the scores to be comparable, the best strategy is to set an explicit fixed window length that accommodates a majority of the useful contigs in your assembly.

Removal of very short ‘remainder’ windows

Since the windows are non-overlapping, the final window in the sequence is often shorter than the other windows (its length is the remainder when you divide the total sequence length by the window size). In general it is not a big problem to include one shorter window in the calculation, but it means that these final bases can sometimes have very disproportional representation. So when the remainder is very small (<10% of the window size), we exclude the final, shorter window entirely.

What happens if we are using a fixed window length and are dealing with very short contigs that by themselves are <10% of that length? We don’t throw anything away, and those contigs will have exactly 1 window.

S and the expected spread

The observed spread is the proportion of windows that contain at least one base with nonzero coverage:

S_obs = # of windows with coverage / # of windows

Schematic for Spread

But the proportion of covered windows depends heavily on how many reads map to the sequence. A handful of reads can only cover a handful of windows, even if the sequence is truly present in the sample. To account for this, we compare the observed spread to the spread we would expect if the same reads were placed randomly along the sequence:

S_exp = 1 − e^(−(B + N·w) / G)

where B is the total number of aligned bases (the sum of the coverage array), N is the number of reads mapping to the sequence, w is the window length, and G is the length of the sequence. The normalized spread is then

S = min(1, S_obs / S_exp)

so S is close to 1 when reads are spread across the sequence as much as (or more than) we would expect from random placement, and lower when reads are clustered in a few regions. If the sequence has no coverage at all, S is 0.

Read length has an impact on the expected spread and is implicitly taken into account by considering both the number of mapped reads (N) and the total alignment length (B): the same amount of coverage coming from a few long reads covers fewer windows than when it comes from many short reads. For example, 5 long reads with a mean length of 10,000 bp on a 200,000 bp sequence (with 1,000 bp windows) are expected to cover only about 24% of the windows, so observing 25% covered windows yields S = 1. The same coverage coming from 500 short reads of 100 bp would be expected to cover about 94% of the windows, so in that case S would be about 0.27.

In the output files, the observed spread is reported as actual_prop_windows_covered, the expected spread as expected_prop_windows_covered, and S as normalized_spread.

E and the coefficient of variation

Evenness is computed from the mean coverage depth of each window that has nonzero coverage, essentially asking the question: wherever there is coverage on the sequence, is the coverage depth even? (Uncovered windows are already accounted for in S, anyway). We compute the coefficient of variation (CV, the standard deviation divided by the mean) of these window means, and convert it into a score between 0 and 1:

E = 1 / (1 + CV)

Schematic for Evenness

When all covered windows have the same mean depth, CV = 0 and E = 1. The more the depths vary across windows, the closer E gets to 0.

If fewer than 3 windows have coverage, there is not enough information to estimate evenness, and E is set to 0 (maximum unevenness). This happens for sequences with very few windows (i.e., short sequences relative to the window length) and for sequences with very few mapped reads. With the default geometric formula, E = 0 means that C = 0, so discov will be equal to D^2 and generally very low.

Sequences with very few mapped reads

When only a handful of reads map to a sequence, there is not enough information to properly estimate either spread or evenness. Even so, both can look deceptively good: a few reads landing in separate windows are about as spread out as we expect from randomly-placed reads (so normalized S is high), and since each covered window holds about one read, the mean window coverage depths barely vary (so E is high). Without a safeguard, such sequences could get a discov score close to 1 from just a few reads.

To avoid this, anvi’o sets both normalized S and E to 0 for any sequence with fewer mapped reads than --discov-min-reads (20 by default). The spread-evenness score C is then 0, and discov collapses to D^2, which will be very low since a few reads cannot cover much of a sequence. The observed and expected proportions of covered windows are still reported for these sequences. You can set --discov-min-reads 0 to always compute normalized S and E. Note that reads are counted individually, so each mate of a read pair that maps counts as one read.

The same read threshold applies to genome-level and contig-level scores. The default of 20 reads was chosen based on tests with genome-level scores, and it may be too high for contig-level scores, where shorter sequences need fewer reads to be meaningfully covered. If you are working with contig-level scores (especially for short contigs), consider testing a lower --discov-min-reads value with contigs you know to be present or absent.

The weight parameter α

You can adjust the weights on S and E by setting the --alpha parameter. Increase α to increase the importance of spread, and decrease α to increase the importance of evenness. α should always be between 0 and 1 (inclusive. In case you really want C to be equivalent to one of S or E you can set α = 1 or α = 0, respectively).

Choosing the formula for combining S and E

The parameter --discov-formula controls which equation is used to combine S and E into the spread-evenness score C. If it’s geometric (the default), we compute a weighted geometric mean of S and E, and if it’s linear, we take a weighted average of the two.

The geometric formula requires a sequence to have both high S and high E to yield a high score, and C will be 0 if either of them is 0. The linear formula is more permissive: a high value of one metric can compensate for a low value of the other. That makes linear scores somewhat ambiguous, since a mid-level score could come from similar, moderate values of S and E or from one metric near 1 and the other near 0 (for instance, a sequence with only 1 or 2 covered windows will have E = 0, but with the linear formula it can still get a mid-level score from S alone).

Okay, but how do I use discov?

You can use discov to decide if a given sequence is present or absent in a metagenome sample by setting a threshold score above which it is considered present. Where you draw the line depends on how confident you want to be (or how much you want to avoid false positives).

To help you get an intuition on how to set the threshold, you can take a look at the distribution of discov scores in our test dataset. We manually-validated present or absent genomes across metagenomes that were randomly subsampled, and computed discov for each genome in each subsample using default parameters, or when changing the S-E formula to linear. Below, each dot represents one genome in one subsample (there are 25 total metagenomes each with 20 subsamples keeping 0.1% to 100% of reads, and each metagenome has ~5 present and ~5 absent genomes associated wth it). The dashed line indicates the discov threshold with the best precision and recall (highest F1 score) in each case.

Discov Distributions

Some rough guidelines from this data: discov thresholds for determining presence should always be set above 0.5. With all default parameters, a threshold around 0.6 is good for avoiding most false positives. The linear formula is more forgiving/ambiguous for mid-range spread and evenness values, and therefore the threshold should generally be set higher when using the linear version than when using the geometric formula. Note that these recommendations apply to genome-level metrics and specifically when using the default fixed window length of 10,000 bp; we haven’t tested contig-level metrics comprehensively enough to suggest thresholds for those yet.

Since the window lengths and alpha values used to compute discov can affect the range of discov scores you will get, it is worthwhile to run some tests with known present/absent genomes to help set a classification threshold that will work for the parameters / window sizing strategy you want to use.

More resources

  • The initial discov Pull Request explains some implementation decisions, shows results from the hyperparameter tuning used to set the default parameters, and also describes the outcome of various validation tests. However, please note that this is an earlier version of discov, when it exclusively consisted of the spread-evenness score (no detection in the formula). If you are interested in implementation decisions, you might also want to read the version 2 Pull Request to learn about why we changed certain things about the formula.
  • The pull request for --min-window-length includes some tests that show the effect of this parameter on the distribution of discov scores.
  • The pull request for --discov-min-reads includes some tests that support the need for this parameter and discusses how the default value was chosen.

Edit this file to update this information.