anvi-get-sequences-for-gene-clusters

Authors
Requires
Provides
Can provide
Usage

Running on all gene clusters
Reporting a single representative sequence per gene cluster
Splitting gene clusters into their own files
Exporting only specific gene clusters

Part 1: Choosing gene clusters by collection, bin, or name
Part 2: Choosing gene clusters by their attributes

Fun with phylogenomics!

Command line interface

Command line usage
Parameters and their descriptions

Additional Resources

Do cool stuff with gene clusters in anvi'o pan genomes.

🔙 To the main page of anvi’o programs and artifacts.

Authors

A. Murat Eren (Meren)

Requires

pan-db genomes-storage-db

Provides

genes-fasta

Can provide

concatenated-gene-alignment-fasta misc-data-items

Usage

This aptly-named program gets the sequences for the gene clusters stored in a pan-db and returns them as either a genes-fasta or a concatenated-gene-alignment-fasta, which can directly go into the program anvi-gen-phylogenomic-tree for phylogenomics. This gives you advanced access to your gene clusters, so you can take them out of anvi’o and do whatever you please with them.

The program parameters also include a large collection of advanced filtering options. Using these options you can scrutinize your gene clusters in creative and precise ways. Using the combination of these filters you can focus on single-copy core gene clusters in a pangenome, or those occur only as singletons, or paralogs that contain more than a given number of sequences, and so on. Once you are satisfied with the output a given set of filters generate, you can add the matching gene clusters a misc-data-items with the flag --add-into-items-additional-data-table, which can be added to the interactive interface as additional layers when you visualize your pan-db using the program anvi-display-pan

By default, anvi-get-sequences-for-gene-clusters will generate a single output file. But you can ask the program to report every gene cluster that match to your filters as a separate FASTA file depending on your downstream analyses.

While the number of parameters this powerful program can utilize may seem daunting, many of the options just help you specify exactly from which gene clusters you want to get sequences.

Running on all gene clusters

Here is an example that shows the simplest possible run, which will export sequences for every single gene cluster found in the pan-db as amino acid sequences:

anvi-get-sequences-for-gene-clusters -g genomes-storage-db \ -p pan-db \ -o genes-fasta

The program will report the DNA sequences if the flag --report-DNA-sequences is used.

Reporting a single representative sequence per gene cluster

By default this program reports every gene sequence in each gene cluster. If instead you want a single representative amino acid sequence per gene cluster, use the flag --representative-sequences:

anvi-get-sequences-for-gene-clusters -g genomes-storage-db \ -p pan-db \ --representative-sequences \ -o genes-fasta

Anvi’o picks the representative using a medoid-based strategy: it discards length outliers, prefers complete (non-partial) gene calls, and selects the sequence that is most similar to the others in the gene cluster. The resulting fasta uses the gene cluster names as deflines and contains no gap characters.

This flag requires a pangenome for which gene alignments were computed (i.e., not created with --skip-alignments), and it cannot be combined with --concatenate-gene-clusters or --report-DNA-sequences. It honors all the same selection and filtering options described below, as well as --split-output-per-gene-cluster.

Splitting gene clusters into their own files

The command above will put all gene cluster sequences in a single output fasta file. If you would like to report each gene cluster in a separate FASTA file, it is also an option thanks to the flag --split-output-per-gene-cluster. This optional reporting throught this flag applies to all commands shown on this page. For instance, the following command will report every gene cluster as a separate FASTA file in your directory,

anvi-get-sequences-for-gene-clusters -g genomes-storage-db \ -p pan-db \ --split-output-per-gene-cluster

where the output files and paths will look like this in your work directory:

GC_00000001.fa
GC_00000002.fa
GC_00000003.fa
GC_00000004.fa
GC_00000005.fa
GC_00000006.fa
GC_00000007.fa
GC_00000008.fa
GC_00000009.fa
GC_00000010.fa
(...)

You can use the parameters --output-file-prefix to add file name prefixes to your output files. For instance, the following command,

anvi-get-sequences-for-gene-clusters -g genomes-storage-db \ -p pan-db \ --split-output-per-gene-cluster \ --output-file-prefix MY_PROJECT

will result in the following files in your work directory:

MY_PROJECT_GC_00000001.fa
MY_PROJECT_GC_00000002.fa
MY_PROJECT_GC_00000003.fa
MY_PROJECT_GC_00000004.fa
MY_PROJECT_GC_00000005.fa
MY_PROJECT_GC_00000006.fa
MY_PROJECT_GC_00000007.fa
MY_PROJECT_GC_00000008.fa
MY_PROJECT_GC_00000009.fa
MY_PROJECT_GC_00000010.fa
(...)

You can also use the parameter --output-file-prefix to store files in different directories. For instance, the following command (note the trailing / in the --output-file-prefix),

anvi-get-sequences-for-gene-clusters -g genomes-storage-db \ -p pan-db \ --split-output-per-gene-cluster \ --output-file-prefix A_TEST_DIRECTORY/

will result in the following files:

A_TEST_DIRECTORY/GC_00000001.fa
A_TEST_DIRECTORY/GC_00000002.fa
A_TEST_DIRECTORY/GC_00000003.fa
A_TEST_DIRECTORY/GC_00000004.fa
A_TEST_DIRECTORY/GC_00000005.fa
A_TEST_DIRECTORY/GC_00000006.fa
A_TEST_DIRECTORY/GC_00000007.fa
A_TEST_DIRECTORY/GC_00000008.fa
A_TEST_DIRECTORY/GC_00000009.fa
A_TEST_DIRECTORY/GC_00000010.fa
(...)

In contrast, the following command,

anvi-get-sequences-for-gene-clusters -g genomes-storage-db \ -p pan-db \ --split-output-per-gene-cluster \ --output-file-prefix A_TEST_DIRECTORY/A_NEW_PREFIX

will result in the following files:

A_TEST_DIRECTORY/A_NEW_PREFIX_GC_00000001.fa
A_TEST_DIRECTORY/A_NEW_PREFIX_GC_00000002.fa
A_TEST_DIRECTORY/A_NEW_PREFIX_GC_00000003.fa
A_TEST_DIRECTORY/A_NEW_PREFIX_GC_00000004.fa
A_TEST_DIRECTORY/A_NEW_PREFIX_GC_00000005.fa
A_TEST_DIRECTORY/A_NEW_PREFIX_GC_00000006.fa
A_TEST_DIRECTORY/A_NEW_PREFIX_GC_00000007.fa
A_TEST_DIRECTORY/A_NEW_PREFIX_GC_00000008.fa
A_TEST_DIRECTORY/A_NEW_PREFIX_GC_00000009.fa
A_TEST_DIRECTORY/A_NEW_PREFIX_GC_00000010.fa
(...)

Exporting only specific gene clusters

Part 1: Choosing gene clusters by collection, bin, or name

You can export only the sequences for a specific collection or bin with the parameters -C or -b respectively.

anvi-get-sequences-for-gene-clusters -g genomes-storage-db \ -p pan-db \ -o genes-fasta \ -C collection

You can always display the collections and bins available in your pan-db by adding --list-collections or --list-bins flags to your command.

Alternatively, you can export the specific gene clusters by name, either by providing a single gene cluster ID or a file with one gene cluster ID per line. For example:

anvi-get-sequences-for-gene-clusters -g genomes-storage-db \ -p pan-db \ -o genes-fasta \ --gene-cluster-ids-file gene_clusters.txt

where gene_clusters.txt contains the following:

GC_00000618
GC_00000643
GC_00000729

Part 2: Choosing gene clusters by their attributes

These parameters are used to exclude gene clusters that don’t reach certain thresholds and are applies on top of filters already applied (for example, you can use these to exclude clusters within a specific bin).

Here is a list of the different filters that you can use to exclude some subsection of your gene clusters:

min/max number of genomes that the gene cluster occurs in.
min/max number of genes from each genome. For example, you could exclude clusters that don’t appear in every genome 3 times, or get single-copy genes by setting max-num-genes-from-each-genome to 1.
min/max geometric homogenity index
min/max functional homogenity index
min/max combined homogenity index

For example, the following run on a genomes-storage-db that contains 50 genomes will report only the single-copy core genes with a functional homogenity index above 0.25:

anvi-get-sequences-for-gene-clusters -g genomes-storage-db \ -p pan-db \ -o genes-fasta \ --max-num-genes-from-each-genome 1 \ --min-num-genomes-gene-cluster-occurs 50 \ --min-functional-homogenity-index 0.25

You can also exclude genomes that are missing some number of the gene clusters that you’re working with by using the paramter --max-num-gene-clusters-missing-from-genome.

For each of these parameters, see the program’s help menu for more information.

Fun with phylogenomics!

To get a concatenated-gene-alignment-fasta (which you can use to run anvi-gen-phylogenomic-tree), use the parameter --concatenate-gene-clusters

anvi-get-sequences-for-gene-clusters -g genomes-storage-db \ -p pan-db \ -o genes-fasta \ --concatenate-gene-clusters

Here, you also have the option to specify a specific aligner (or list the available aligners), as well as provide a NEXUS formatted partition file, if you so choose.

Edit this file to update this information.

Command line interface

The following is the list of parameters and their descriptions in the latest version of anvi-get-sequences-for-gene-clusters.

The code that generates this web page ensures that this information is the latest available in 9-dev (eunice). Which means, in some cases the information in this section (generated automatically) might be more comprehensive or up-to-date compared to the descriptions above (written by humans).

Command line usage

anvi-get-sequences-for-gene-clusters [-h] -p PAN_DB [-g GENOMES_STORAGE] [-o FASTA] [--split-output-per-gene-cluster] [-O FILENAME_PREFIX] [--gene-cluster-id GENE_CLUSTER_ID] [--gene-cluster-ids-file FILE_PATH] [-C COLLECTION_NAME] [-b BIN_NAME] [--min-num-genomes-gene-cluster-occurs INTEGER] [--max-num-genomes-gene-cluster-occurs INTEGER] [--min-num-genes-from-each-genome INTEGER] [--max-num-genes-from-each-genome INTEGER] [--max-num-gene-clusters-missing-from-genome INTEGER] [--min-functional-homogeneity-index FLOAT] [--max-functional-homogeneity-index FLOAT] [--min-geometric-homogeneity-index FLOAT] [--max-geometric-homogeneity-index FLOAT] [--min-combined-homogeneity-index FLOAT] [--max-combined-homogeneity-index FLOAT] [--add-into-items-additional-data-table NAME] [--list-collections] [--list-bins] [--concatenate-gene-clusters] [--partition-file FILE_PATH] [--separator STRING] [--align-with ALIGNER] [--list-aligners] [--representative-sequences] [--report-DNA-sequences] [--just-do-it] [--dry-run]

Parameters and their descriptions

INPUT FILES Input files from the pangenome analysis.

-p, --pan-db PAN_DB required: Anvi'o pan database
-g, --genomes-storage GENOMES_STORAGE: Anvi'o genomes storage file

OUTPUT OPTION #1: SINGLE FILE Here you get to chose an output file name to report things. LUCKY YOU.

-o, --output-file FASTA: File path to store results.

OUTPUT OPTION #2: MULTIPLE FILES Here you can ask anvi'o to report multiple FASTA files, where each gene cluster goes into its own unique FASTA file. If you declare teh flag `--split-output-per-gene-cluster`, anvi'o will store each gene cluster into your work directory, using the gene cluster name as the output file name. You can use the `--output-file-prefix` parameter to store them in a different directory and/or with specific file name prefixes.

--split-output-per-gene-cluster: If/when there are more than one gene clusters to report, put each gene cluster into a separate FASTA file.
-O, --output-file-prefix FILENAME_PREFIX: A prefix to be used while naming the output files. No file type extensions please; just a prefix -- all ouptut file names and extensions will be appended to this prefix.

SELECTION Which gene clusters should be reported. You can ask for a single gene cluster, or multiple ones listed in a file, or you can use a collection and bin name to list gene clusters of interest. If you give nothing, this program will export alignments for every single gene cluster found in the profile database (and this is called 'customer service').

--gene-cluster-id GENE_CLUSTER_ID: Gene cluster ID you are interested in.
--gene-cluster-ids-file FILE_PATH: Text file for gene clusters (each line should contain be a unique gene cluster id).
-C, --collection-name COLLECTION_NAME: Collection name.
-b, --bin-id BIN_NAME: Bin name you are interested in.

ADVANCED FILTERS If you are here you must be looking for ways to specify exactly what you want from that database of gene clusters. These filters will be applied to what your previous selections reported.

--min-num-genomes-gene-cluster-occurs INTEGER: This filter will remove gene clusters from your report. Let's assume you have 100 genomes in your pan genome analysis. You can use this parameter if you want to work only with gene clusters that occur in at least X number of genomes. If you say '--min-num-genomes-gene-cluster-occurs 90', each gene cluster in the analysis will be required at least to appear in 90 genomes. If a gene occurs in less than that number of genomes, it simply will not be reported. This is especially useful for phylogenomic analyses, where you may want to only focus on gene clusters that are prevalent across the set of genomes you wish to analyze. Default: 0
--max-num-genomes-gene-cluster-occurs INTEGER: This filter will remove gene clusters from your report. Let's assume you have 100 genomes in your pan genome analysis. You can use this parameter if you want to work only with gene clusters that occur in at most X number of genomes. If you say '--max-num-genomes-gene-cluster-occurs 1', you will get gene clusters that are singletons. Combining this parameter with --min-num-genomes-gene-cluster-occurs can give you a very precise way to filter your gene clusters. Default: 9223372036854775807
--min-num-genes-from-each-genome INTEGER: This filter will remove gene clusters from your report. If you say '--min-num-genes-from-each-genome 2', this filter will remove every gene cluster, to which every genome in your analysis contributed less than 2 genes. This can be useful to find out gene clusters with many genes from many genomes (such as conserved multi-copy genes within a clade). Default: 0
--max-num-genes-from-each-genome INTEGER: This filter will remove gene clusters from your report. If you say '--max-num-genes-from-each-genome 1', every gene cluster that has more than one gene from any genome that contributes to it will be removed from your analysis. This could be useful to remove gene clusters with multi-copy genes from your report for appropriate phylogenomic analyses. For instance, using '--max-num-genes-from-each-genome 1' and 'min-num-genomes-gene-cluster-occurs X' where X is the total number of your genomes, would give you the single-copy gene clusters in your pan genome. Default: 9223372036854775807
--max-num-gene-clusters-missing-from-genome INTEGER: This filter will remove genomes from your report. If you have a list of gene cluster names, you can use this parameter to omit any genome from your report if it is missing more than a number of genes you desire. For instance, if you have 100 genomes in your pan genome, and you are interested in working only with genomes that have all 5 specific gene clusters of your choice, you can use '--max-num-gene-clusters-missing-from-genome 4' to remove remove the bins that are missing more than 4 of those 5 genes. This is especially useful for phylogenomic analyses. Parameter 0 will remove any genome that is missing any of the genes. Default: 0
--min-functional-homogeneity-index FLOAT: This filter will remove gene clusters from your report. If you say '--min-functional-homogeneity-index 0.3', every gene cluster with a functional homogeneity index less than 0.3 will be removed from your analysis. This can be useful if you only want to look at gene clusters that are highly conserved in resulting function Default: -1
--max-functional-homogeneity-index FLOAT: This filter will remove gene clusters from your report. If you say '--max-functional-homogeneity-index 0.5', every gene cluster with a functional homogeneity index greater than 0.5 will be removed from your analysis. This can be useful if you only want to look at gene clusters that don't seem to be functionally conserved Default: 1
--min-geometric-homogeneity-index FLOAT: This filter will remove gene clusters from your report. If you say '--min-geometric-homogeneity-index 0.3', every gene cluster with a geometric homogeneity index less than 0.3 will be removed from your analysis. This can be useful if you only want to look at gene clusters that are highly conserved in geometric configuration Default: -1
--max-geometric-homogeneity-index FLOAT: This filter will remove gene clusters from your report. If you say '--max-geometric-homogeneity-index 0.5', every gene cluster with a geometric homogeneity index greater than 0.5 will be removed from your analysis. This can be useful if you only want to look at gene clusters that have many not be as conserved as others Default: 1
--min-combined-homogeneity-index FLOAT: This filter will remove gene clusters from your report. If you say '--min-combined-homogeneity-index 0.3', every gene cluster with a combined homogeneity index less than 0.3 will be removed from your analysis. This can be useful if you only want to look at gene clusters that are highly conserved overall Default: -1
--max-combined-homogeneity-index FLOAT: This filter will remove gene clusters from your report. If you say '--max-combined-homogeneity-index 0.5', every gene cluster with a combined homogeneity index greater than 0.5 will be removed from your analysis. This can be useful if you only want to look at gene clusters that have many not be as conserved overall as others Default: 1
--add-into-items-additional-data-table NAME: If you use any of the filters, and would like to add the resulting item names into the items additional data table of your database, you can use this parameter. You will need to give a name for these results to be saved. If the given name is already in the items additional data table, its contents will be replaced with the new one. Then you can run anvi-interactive or anvi-display-pan to 'see' the results of your filters.

OTHER STUFF Yes. Stuff that are not like the ones above.

--list-collections: Show available collections and exit.
--list-bins: List available bins in a collection and exit.

PHYLOGENOMICS Get separately aligned and concatenated sequences for phylogenomics.

--concatenate-gene-clusters: Concatenate output gene clusters in the same order to create a multi-gene alignment output that is suitable for phylogenomic analyses.
--partition-file FILE_PATH: Some commonly used software for phylogenetic analyses (e.g., IQ-TREE, RAxML, etc) allow users to specify/test different substitution models for each gene of a concatenated multiple sequence alignments. For this, they use a special file format called a 'partition file', which indicates the site for each gene in the alignment. You can use this parameter to declare an output path for anvi'o to report a NEXUS format partition file in addition to your FASTA output (requested by Massimiliano Molari in #1333).
--separator STRING: Characters to separate things (the default is whatever is most suitable).
--align-with ALIGNER: The multiple sequence alignment program to use when multiple sequence alignment is necessary. To see all available options, use the flag `--list-aligners`.
--list-aligners: Show available software for multiple sequence alignment.

REPRESENTATIVE SEQUENCES Rather than reporting every gene in a gene cluster, report a single representative amino acid sequence per gene cluster.

--representative-sequences: Instead of reporting every gene sequence in each gene cluster, report a single representative amino acid sequence per gene cluster. Anvi'o picks the representative using a medoid-based strategy: it discards length outliers, prefers complete (non-partial) gene calls, and selects the sequence that is most similar to the others in the cluster. The resulting FASTA uses gene cluster names as deflines and contains no gap characters. This flag requires a pangenome for which gene alignments were computed (i.e., not created with `--skip-alignments`), and cannot be combined with `--concatenate-gene-clusters` or `--report-DNA-sequences`.

LIFE SAVERS Just when you need them.

--report-DNA-sequences: By default, this program reports amino acid sequences. Use this flag to report DNA sequences instead.
--just-do-it: Don't bother me with questions or warnings, just do it.
--dry-run: Don't do anything real. Test everything, and stop right before wherever the developer said 'well, this is enough testing', and decided to print out results.

Additional Resources

In action in the Anvi'o pangenomics tutorial

Are you aware of resources that may help users better understand the utility of this program? Please feel free to edit this file on GitHub. If you are not sure how to do that, find the __resources__ tag in this file to see an example.