anvi-pan-genome

An anvi'o program to compute a pangenome from an anvi'o genome storage.

🔙 To the main page of anvi’o programs and artifacts.

Authors

Requires

genomes-storage-db

Can use

gene-clusters-txt

Provides

pan-db gene-clusters

Can provide

misc-data-items-order

Usage

This program implements pangenomics, and organizes genes found within a genomes-storage-db to create a pan-db.

Please first read the pangenomics tutorial to have a better understanding of the steps that lead to the generation of a pan-db.

Making sure your installation can do pangenomics

You can always test if your computer has all the dependencies for a successful pangenomics analysis by running,

anvi-self-test --suite pangenomics

If it runs without errors, you’re golden. If not, please consult with the most up-to-date installation instructions for anvi’o and get in touch with the anvi’o community for guidance.

A brief summary

The program anvi-pan-genome performs three major things for its user:

  1. Calculates the similarity between the all gene amino acid seqeunces found in genomes described in your genomes-storage-db using DIAMOND. You have some options. Although, (1) you can use the NCBI’s BLAST program blastp instead of DIAMOND using the --use-ncbi-blast flag, (2) instead of analyzing all genomes you can focus a subset using the --genome-names parameter, and (3) exclude genes that are partial from your analysis using the flag --exclude-partial-gene-calls if you think you must.

  2. Resolves gene clusters using the BLAST results via the MCL algorithm after discarding weak hits from the search results using the --minbit heuristic (inspired by the workflow implemented by ITEP (Benedict et al., 2014).

  3. Performs additional analyses of gene clusters for downstream analyses and visualization tasks. These analyses include,

    • Multiple sequnce alignment of amino acid sequences in each gene cluster,
    • Computation of functional and geometric homogeneity indices,
    • Computation of average amino-acid identity (AAI) within each gene cluster,
    • Hierarchical clustering analysis of gene clusters based on their distribution across genomes, and genomes based on their sharing of the gene pool.

The basic command line to run a pangenomic analysis that will do all the step above will look like the following:

anvi-pan-genome --genomes-storage genomes-storage-db \ --project-name PROJECT_NAME

But it is also possible for power users to initiate the anvi’o pangenomcs workflow with user defined gene clusters, which means it would be possible to visualize the pangenomic analyses performed by other tools in anvi’o. In this case, only the third step is performed with already established gene clusters:

anvi-pan-genome --genomes-storage genomes-storage-db \ --project-name PROJECT_NAME \ --gene-clusters-txt gene-clusters-txt

Controlling output file locations

By default, anvi-pan-genome places the resulting pan-db next to your genomes-storage-db with a name derived from your project name (e.g., PROJECT_NAME-PAN.db). If you want to put it somewhere specific, use --output-file:

anvi-pan-genome --genomes-storage genomes-storage-db \ --project-name PROJECT_NAME \ --output-file /path/to/PROJECT_NAME-PAN.db

The output file name must end with -PAN.db. No exceptions.

Intermediate data files generated during the analysis (e.g., DIAMOND search results) are stored in a temporary directory that anvi’o creates automatically and removes at the end of a successful run. If you want to keep those files — for instance, to re-run the analysis with different clustering parameters without repeating the expensive sequence search step — use --intermediate-data-dir:

anvi-pan-genome --genomes-storage genomes-storage-db \ --project-name PROJECT_NAME \ --intermediate-data-dir /path/to/intermediate/

On a subsequent run, you can then point anvi’o to that same directory to skip the sequence search entirely:

anvi-pan-genome --genomes-storage genomes-storage-db \ --project-name PROJECT_NAME \ --intermediate-data-dir /path/to/intermediate/ \ --minbit 0.6

If you run anvi-pan-genome with the --debug flag, anvi’o will keep the auto-generated intermediate directory and tell you its path at the end of the run, so you can inspect the files or reuse them with --intermediate-data-dir.

The ‘additional parameters’ mechanism for power users

At the core of the pangenomics workflow lies the reciprocal BLAST search that identifies sequence similarities within a pool of gene sequences. For this, anvi’o uses DIAMOND by default, but the user can change the search algorithm. Based on the algorithm used for this step, the matching anvi’o driver sets some default parameters for a successful run. Such as the proper parameter to explicitly define where the output files generated by DIAMOND should go, and so on. Apart from those mandatory parameters that are critical for a successful run, anvi’o allows the user to define a set of additional parameters to pass to the search algorithm.

This is done via the flag --additional-params-for-seq-search. For instance, the user could take a look at the parameters diamond offers by typing diamond help on their terminal, and may decide to use the --sensitive implemented by DIAMOND to enable a slower but more sensitive search, and use the parameter --id 98 to ask DIAMOND to not report any hits across genes that is lower than 98% sequence identity to limit gene clusters only those sequences that are extremely closely related while pushing everything else to be singletons (which can also be removed from the analysis with a separate --min-occurrence 2 flag anvi-pan-genome accepts). They can pass these parameters to DIAMOND by running their analysis the following way:

anvi-pan-genome --genomes-storage genomes-storage-db \ --project-name PROJECT_NAME \ --additional-params-for-seq-search “--masking 0 --sensitive --id 98”

The additional parameters used for the search will be stored in the resulting pan-db and can be viewed anytime using the program anvi-display-pan.

For DIAMOND, if no additional parameters is declared, anvi’o will include --masking 0 by default since we recently learned that not using that flag leads to the elmination of genes with many repeated elements (see #1955).

With the freedom of additional parameters for sequnce search, it is possible to make significant mistakes since anvi’o will have no opportunity to sanity-check user-defined additional parameters. If you are doing something experimental, please keep an eye on the output messages and error logs.

If the user choses to use NCBI’s BLAST program, in that case anvi’o will pass the value of the parameter --additional-params-for-seq-search to NCBI’s blastp.

Edit this file to update this information.

Command line interface

The following is the list of parameters and their descriptions in the latest version of anvi-pan-genome.

The code that generates this web page ensures that this information is the latest available in 9-dev (eunice). Which means, in some cases the information in this section (generated automatically) might be more comprehensive or up-to-date compared to the descriptions above (written by humans).


Command line usage

anvi-pan-genome [-h] -g GENOMES_STORAGE [-G GENOME_NAMES] [--exclude-partial-gene-calls] [-o FILE_PATH] [--intermediate-data-dir PATH] [-W] [--use-ncbi-blast] [--additional-params-for-seq-search CMD LINE PARAMS] [--minbit MINBIT] [--mcl-inflation INFLATION] [--min-percent-identity PERCENT] [--min-occurrence NUM_OCCURRENCE] [--gene-clusters-txt FILE PATH] [--skip-alignments] [--align-with ALIGNER] [--skip-homogeneity] [--quick-homogeneity] [-n PROJECT_NAME] [--description TEXT_FILE] [-T NUM_THREADS] [--skip-hierarchical-clustering] [--enforce-hierarchical-clustering] [--distance DISTANCE_METRIC] [--linkage LINKAGE_METHOD]

Parameters and their descriptions

INPUT DATA: GENOMES & GENES You first tell anvi'o where your genes are (which is stored in a fancy 'genomes storage' file you have ALREADY generated using the program `anvi-genomes-storage` (if this means nothing to you, see the anvi'o pangenomics tutorial)). They you tell anvi'o if you wish to limit your analysis to a smaller set of genomes than those anvi'o will find in that genomes-storage-db, or whether or not it should exclude partial gene calls.
-g, --genomes-storage GENOMES_STORAGE required
Anvi'o genomes storage file
-G, --genome-names GENOME_NAMES
Genome names to 'focus'. You can use this parameter to limit the genomes included in your analysis. You can provide these names as a comma-separated list of names, or you can put them in a file, where you have a single genome name in each line, and provide the file path.
--exclude-partial-gene-calls
By default, anvi'o includes all partial gene calls from the analysis, which, in some cases, may inflate the number of gene clusters identified and introduce extra heterogeneity within those gene clusters. Using this flag, you can request anvi'o to exclude partial gene calls from the analysis (whether a gene call is partial or not is an information that comes directly from the gene caller used to identify genes during the generation of the contigs database).
OUTPUT DATA: WHERE THINGS SHOULD APPEAR? If you completely skip this section, anvi'o will put things in reasonable places with reasonable names. But you can change that if you want to.
-o, --output-file FILE_PATH
File path to store results.
--intermediate-data-dir PATH
A directory path to store intermediate data files or to read them from. If you don't provide anything here, this program will come up with a temporary directory path by itself to store intermediate files, and most likely clean it later (unless you used the `--debug` flag).
-W, --overwrite-output-destinations
Overwrite if the output files and/or directories exist.
DEFAULT GENE CLUSTER FORMATION: RESOLVING NETWORK The second step in this workflow is to use the sequence search results to determine gene clusters. Which is a straightforward step thanks to the MCL algorithm. The following parameters are there to tweak this step, even though default choices will work for the vast majority of cases.
--minbit MINBIT
The minimum minbit value. The minbit heuristic provides a mean to set a to eliminate weak matches between two amino acid sequences. We learned it from ITEP (Benedict MN et al, doi:10.1186/1471-2164-15-8), which is a comprehensive analysis workflow for pangenomes, and decided to use it in the anvi'o pangenomic workflow, as well. Briefly, If you have two amino acid sequences, 'A' and 'B', the minbit is defined as 'BITSCORE(A, B) / MIN(BITSCORE(A, A), BITSCORE(B, B))'. So the minbit score between two sequences goes to 1 if they are very similar over the entire length of the 'shorter' amino acid sequence, and goes to 0 if (1) they match over a very short stretch compared even to the length of the shorter amino acid sequence or (2) the match between sequence identity is low. The default is 0.5. Default: 0.5
--mcl-inflation INFLATION
MCL inflation parameter, that defines the sensitivity of the algorithm during the identification of the gene clusters. More information on this parameter and it's effect on cluster granularity is here: (http://micans.org/mcl/man/mclfaq.html#faq7.2). The default is 2. Default: 2.0
--min-percent-identity PERCENT
Minimum percent identity between the two amino acid sequences for them to have an edge for MCL analysis. This value will be used to filter hits from Diamond search results. Because percent identity is not a predictor of a good match (since it does not communicate many other important factors such as the alignment length between the two sequences and its proportion to the entire length of those involved), we suggest you rely on 'minbit' parameter. But you know what? Maybe you shouldn't listen to anyone, and experiment on your own! The default is 0 percent. Default: 0.0
--min-occurrence NUM_OCCURRENCE
Do you not want singletons? You don't? Well, this parameter will help you get rid of them (along with doubletons, if you want). Anvi'o will remove gene clusters that occur less than the number you set using this parameter from the analysis. The default is 1, which means everything will be kept. If you want to remove singletons, set it to 2, if you want to remove doubletons as well, set it to 3, and so on. Default: 1
ALTERNATIVE GENE CLUSTER FORMATION: USER-PROVIDED GENE CLUSTERS As an alternative approach, you can provide anvi'o with your own gene cluster affiliations. In this case, anvi'o would not use the default way of computing gene clusters de novo, but take your word for which genes go together.
--gene-clusters-txt FILE PATH
A three-column TAB-delimited file that associates genes across genomes with one another as 'gene clusters' for DIY pangenomics 😬 See https://anvio.org/help/main/artifacts/gene-clusters-txt/ for details of the file format.
GENE CLUSTER CHARACTERIZATION Parameters that mostly impact how anvi'o characterizes your gene clusters once they are identified. Such as aligning within gene cluster sequences, computing homogeneity indices for them, etc. Important stuff.
--skip-alignments
By default, anvi'o attempts to align amino acid sequences in each gene cluster using multiple sequence alignment via muscle. You can use this flag to skip that step and be upset later.
--align-with ALIGNER
The multiple sequence alignment program to use when multiple sequence alignment is necessary. To see all available options, use the flag `--list-aligners`.
--skip-homogeneity
By default, anvi'o attempts to calculate homogeneity values for every gene cluster, given that they are aligned. You can use this flag to have anvi'o skip homogeneity calculations. Anvi'o will ignore this flag if you decide to skip alignments
--quick-homogeneity
By default, anvi'o will use a homogeneity algorithm that checks for horizontal and vertical geometric homogeneity (along with functional). With this flag, you can tell anvi'o to skip horizontal geometric homogeneity calculations. It will be less accurate but quicker. Anvi'o will ignore this flag if you skip homogeneity calculations or alignments all together.
OTHERS Sweet parameters of convenience.
-n, --project-name PROJECT_NAME
Name of the project. Please choose a short but descriptive name (so anvi'o can use it whenever she needs to name an output file, or add a new table in a database, or name her first born).
--description TEXT_FILE
A plain text file that contains some description about the project. You can use Markdown syntax. The description text will be rendered and shown in all relevant interfaces, including the anvi'o interactive interface, or anvi'o summary outputs.
-T, --num-threads NUM_THREADS
Maximum number of threads to use for multithreading whenever possible. Very conservatively, the default is 1, unless the environmental variable `ANVIO_THREADS` sets another positive integer. If you use this parameter, it will override `ANVIO_THREADS`. It is a good idea to not exceed the number of CPUs / cores on your system. Plus, please be careful with this option if you are running your commands on a SGE -- if you are clusterizing your runs, and asking for multiple threads to use, you may deplete your resources very fast. Default: 1
ORGANIZING GENE CLUSTERs These are stuff that will change the clustering dendrogram of your gene clusters.
--skip-hierarchical-clustering
Anvi'o attempts to generate a hierarchical clustering of your gene clusters once it identifies them so you can use `anvi-display-pan` to play with it. But if you want to skip this step, this is your flag.
--enforce-hierarchical-clustering
If you want anvi'o to try to generate a hierarchical clustering of your gene clusters even if the number of gene clusters exceeds its suggested limit for hierarchical clustering, you can use this flag to enforce it. Are you are a rebel of some sorts? Or did computers make you upset? Express your anger towards machine using this flag.
--distance DISTANCE_METRIC
The distance metric for the clustering of gene clusters. If you do not use this flag, the default distance metric will be used for each clustering configuration which is "euclidean".
--linkage LINKAGE_METHOD
The same story with the `--distance`, except, the system default for this one is ward.

Additional Resources

Are you aware of resources that may help users better understand the utility of this program? Please feel free to edit this file on GitHub. If you are not sure how to do that, find the __resources__ tag in this file to see an example.