anvi-profile
Table of Contents
- Authors
- Requires
- Provides
- Can provide
- Usage
- Basic Usage
- Inputs
- Checking your BAM file: always a good idea
- Profiling a subset of contigs
- Automatic pre-filtering of contigs without coverage
- Analysis Parameters
- Contig Specification
- Filter reads
- Ancient DNA friendly SNV profiling
- Hierarchical Clustering
- Variability
- Other Parameters
- Command line interface
- Additional Resources
The flagship anvi'o program to profile a BAM file. Running this program on a BAM file will quantify coverages per nucleotide position in read recruitment results and will average coverage and detection data per contig. It will also calculate single-nucleotide, single-codon, and single-amino acid variants, as well as structural variants, such as insertion and deletions, to eventually stores all data into a single anvi'o profile database. For very large projects, this program can demand a lot of time, memory, and storage resources. If all you want is to learn coverages of your nutleotides, genes, contigs, or your bins collections from BAM files very rapidly, and/or you do not need anvi'o single profile databases for your project, please see other anvi'o programs that profile BAM files, anvi-script-get-coverage-from-bam and anvi-profile-blitz.
🔙 To the main page of anvi’o programs and artifacts.
Authors
Requires
Provides
single-profile-db 
variability-profile 
Can provide
Usage
This program creates a single-profile-db from a bam-file and contigs-db.
Once you have a single-profile-db, you can run programs like anvi-cluster-contigs, anvi-estimate-metabolism, and anvi-gen-gene-level-stats-databases, as well as use the interactive interface with anvi-interactive. If you want to run these same contigs against multiple BAM files (because you have multiple samples), you’ll combine your single-profile-dbs into a profile-db after you’ve created them all using anvi-merge. See the pages for single-profile-db or profile-db for more you can do with these artifacts.
In short, this program runs various analyses on the contigs in your contigs-db and how they relate to the sample information stored in the bam-file you provided. It then stores this information into a single-profile-db. Specifically, this program calculates
- coverage per nucleotide position (if you’re unsure what coverage refers to, check out this page)
- single-nucleotide, single-codon, and single-amino acid variants (You can find all of those terms on the vocab page linked above, as well as a more detailed explaination here)
- structural variants such as insertions or deletions
Basic Usage
Inputs
This program takes in an indexed bam-file and a contigs-db. The BAM file contains the short reads from a single sample that will be used to create the profile database. Thus, here is a standard run with default parameters:
anvi-profile -i bam-file \ -c contigs-db
Alternatively, if you lack mapping data, you can add the flag --blank-profile so that you can still get the functionality of a profile database.
anvi-profile -c contigs-db \ --blank-profile
Checking your BAM file: always a good idea
If you want to first check your BAM file to see what contigs it contains, just use the flag --list-contigs to see a comprehensive list.
Profiling a subset of contigs
Note: This describes how to profile a named subset of contigs. To profile a subset of contigs based on their characterists (for example, only contigs of a certain length or that have a certain coverage), see the section below on “contig specifications”
By default, anvi’o will use every contig in your contigs-db. However, if you wish to focus specifically on a subset of these contigs, just provide a file that contains only the names of the contigs you want to analyze, one per line, using the tag --contigs-of-interest.
For example, you could run
anvi-profile -c Ross_sea_contigs.db \ --blank-profile \ --contigs-of-interest contigs_i_like.txt
Where contigs_i_like.txt looks like this:
SF15-RossSeacontig4922
SF15-RossSeacontig702
Automatic pre-filtering of contigs without coverage
Before profiling begins, anvi-profile automatically scans the BAM index to identify contigs that have at least one mapped read, and only loads data for those contigs. This is transparent and requires no user intervention.
This is particularly useful when profiling against a large reference database (such as a collection of MAGs) where most contigs in the contigs-db will have zero coverage from any individual metagenome. By skipping these contigs early, anvi’o avoids loading unnecessary data into memory. If you also provide a --contigs-of-interest file, anvi’o will take the intersection of contigs with coverage and your contigs of interest.
Analysis Parameters
Changing these will affect the way that your sequences are analyzed.
Keep in mind that if you plan to merge your resulting single-profile-db with others later in the project, you’ll want to keep these parameters consistent.
Contig Specification
To profile only contigs within a specific length, you can use the flags --min-contig-length and -max-contig-length. By default, the minimum length for analysis is 1000 and there is no maximum length.
But beyond these flags, you can specify which contigs you would like to profile much more explicitly using the flag --contigs-of-interest.
For instance, if you wish to work only with contigs that have more than a certain coverage across your samples, you can first run the program anvi-profile-blitz on all BAM files, then use the resulting output file bam-stats-txt to identify contigs of interest based on their coverages across samples, then put their names in a text file, and pass this file to anvi-profile using the flag --contigs-of-interest.
Filter reads
You can also ignore reads in your BAM file with a percent identity to the reference less than some threshold using the flag --min-percent-identity. By default, all reads are used.
For example, the following code will only look at contigs longer than 2000 nts and will ignore BAM file reads with less than 95 percent identity to the reference:
anvi-profile -c Ross_sea_contigs.db \ -i bam_file.bam \ --min-contig-length 2000 \ --min-percent-identity 95
By default, anvi’o fetches all reads from the bam file. With --fetch-filter you can determine which reads from a bam file will be used for profiling. The current filters are:
double-forwards: only paired-end reads with both R1 and R2 with a ‘forward’ orientation,double-reverses: only paired-end reads with both R1 and R2 with a ‘reverse’ orientation,inversions: only paired-end reads with both R1 and R2 either ‘forward’ or ‘reverse’ and a maximum insert size of 2000 nts,single-mapped-reads: only single mapped reads (mate is unmapped),distant-pairs-1K: only paired-end reads with a minimum 1000 nts insert size.
For example, the following code only considers ‘inversions’ reads:
anvi-profile -c Ross_sea_contigs.db \ -i bam_file.bam \ --fetch-filter inversions
Ancient DNA friendly SNV profiling
By default, anvi’o will report variable nucleotides and their allele frequencies from any nucleotide position in a given short read found in the BAM file. Although, there are some applications where the observed variation in short reads will depend on the location of the nucleotide positions in the read. For instance, in ancient DNA sequencing, the start and the end of short reads are often suffer from DNA damage, leading to an increased number of single-nucleotide variants that emerge from the edges of short reads when they are aligned to a reference. The program anvi-profile has an additional parameter to mitigate this: --skip-edges, which can be used like this:
anvi-profile -c contigs-db \ -i bam-file \ --skip-edges 5
This parameter offers a means to ameliorate the inflation of SNVs due to biases associated with short-read edges by enabling the user to ask anvi’o to ignore a few number of nucleotides from the beginning and the end of short reads as they’re being profiled. For instance, a parameter of 5 like shown above, will make sure that the mismatches of a given read to the reference sequence at the first and the last 5 nucleotides will not be reported in the variability table. Please note that the coverage data will not be impacted by the use of this parameter – only what is reported as variants will be impacted, decreasing the impact of noise in specific applications.
See https://github.com/merenlab/anvio/pull/2081 for more information.
Hierarchical Clustering
To cluster or not to cluster?
By default, anvi’o will not try to cluster your splits (since it takes quite a bit of runtime) unless you are using the tag --blank-profile. If you don’t want to run this, use the tag --skip-hierarchical-clustering.
If you’re planning to later merge this sample with others, it is better to perform clustering while running anvi-merge than at this stage.
However, if you want to bin this single sample or otherwise want clustering to happen, just use the tag --cluster-contigs.
If you do plan to cluster, you can set a custom distance metric or a custom linkage method.
Variability
Anvi-profile will throw away variability data below certain thresholds to reduce noise. After all, if you have a single C read at a position with a 1000X coverage where all other reads are T, this is probably not a variant position that you want to investigate further. By default, it will not analyze positions with coverage less than 10X, and it will further discard variants based on this criteria.
However, you can change the coverage threshold using the --min-coverage-for-variability flag. You can also report every variability position using the flag --report-variability-full.
For example, if you wanted to view every variant, you would profile with the following:
anvi-profile -c Ross_sea_contigs.db \ -i bam_file.bam \ --min-coverage-for-variability 1 \ --report-variability-full
Other Parameters
You should provide the sample name with the flag -S and can provide a description of your project using the --description tag followed by a text file. These will help anvi’o name output files and will show up in the anvi’o interfaces down the line.
You can characterize the codon frequencies of genes in your sample at the cost of some runtime. Despite time being money, codon frequency analysis can be helpful downstream. Simply add the tag --profile-SCVs and watch the magic happen.
If you have prior experience with --profile-SCVs being slow, you will be surprised how fast it is
since v6.2
Alternatively, you can choose not to store insertion and deletion data or single nucleotide variant data.
If you know the limits of your system, you can also multithread this program. See the program help menu for more information.
If you are profiling a large number of splits (e.g., more than 200,000) with many threads (e.g., more than 20), be aware that the memory requirement may outweigh the time gained by multithreading. If your job fails due to an out-of-memory (OOM) error, consider re-running with fewer threads.
Edit this file to update this information.
Command line interface
The following is the list of parameters and their descriptions in the latest version of anvi-profile.
The code that generates this web page ensures that this information is the latest available in 9-dev (eunice). Which means, in some cases the information in this section (generated automatically) might be more comprehensive or up-to-date compared to the descriptions above (written by humans).
Command line usage
Parameters and their descriptions
-i, --input-fileINPUT_BAM- Sorted and indexed BAM file to analyze. Takes a long time depending on the length of the file and parameters used for profiling.
-c, --contigs-dbCONTIGS_DB- Anvi'o contigs database generated by 'anvi-gen-contigs-database'
--blank-profile- If you only have contig sequences, but no mapping data (i.e., you found a genome and would like to take a look from it), this flag will become very handy. After creating a contigs database for your contigs, you can create a blank anvi'o profile database to use anvi'o interactive interface with that contigs database without any mapping data.
-S, --sample-nameNAME- It is important to set a sample name (using only ASCII letters and digits and without spaces) that is unique (considering all others). If you do not provide one, anvi'o will try to make up one for you based on other information (although, you should never let the software decide these things).
--descriptionTEXT_FILE- A plain text file that contains some description about the project. You can use Markdown syntax. The description text will be rendered and shown in all relevant interfaces, including the anvi'o interactive interface, or anvi'o summary outputs.
-M, --min-contig-lengthINT- Minimum length of contigs in a BAM file to analyze. The minimum length should be long enough for tetra-nucleotide frequency analysis to be meaningful. There is no way to define a golden number of minimum length that would be applicable to genomes found in all environments, but we chose the default to be 1000, and have been happy with it. You are welcome to experiment, but we advise to never go below 1,000. You also should remember that the lower you go, the more time it will take to analyze all contigs. You can use --list-contigs parameter to have an idea how many contigs would be discarded for a given M. Default:
1000 --max-contig-lengthINT- Just like the minimum contig length parameter, but to set a maximum. Basically this will remove any contig longer than a certain value. Why would anyone need this? Who knows. But if you ever do, it is here. Default:
0 --contigs-of-interestFILE- It is possible to focus on only a set of contigs. If you would like to do that and ignore the rest of the contigs in your contigs database, use this parameter with a flat file every line of which desribes a single contig name.
--list-contigs- When declared, the program will list contigs in the BAM file and exit gracefully without any further analysis.
--min-percent-identityPERCENT_IDENTITY- Ignore any reads with a percent identity to the reference less than this number, e.g. 95. If not provided, all reads in the BAM file will be used (and things will run faster).
--fetch-filterFILTER- By default, anvi'o fetches all reads from a BAM file. Once a read is 'fetched', some reads may be excluded if you have used parameters such as `--min-percent-identity`. But the `--fetch-filter` is different as it determines WHICH reads from a BAM file will be used for profiling at all. You can do a lot of fun things with this parameter. For details, please read the online documentation for `anvi-profile` using the URL you should see at the end of the `--help` output on your terminal. The known filters are the following: proper-pairs, double-forwards, double-reverses, inversions, distant-inversions, single-mapped-reads, distant-pairs-1K.
--report-variability-full- One of the things anvi-profile does is to store information about variable nucleotide positions (SNVs). Usually it does not report every variable position, since not every variable position is genuine variation. Say, if you have 1,000 coverage, and all nucleotides at that position are Ts and only one of them is a C, the confidence of that C being a real variation is quite low. anvi'o has a simple algorithm in place to reduce the impact of noise. However, using this flag you can disable it and ask profiler to report every single variation (which may result in very large output files and millions of reports, but you are the boss). Do not forget to take a look at '--min-coverage-for-variability' parameter. Also note that this flag controls indel reporting: normally '--min-coverage-for-variability' and internal anvi'o heuristics control whether or not indels should be reported, but with this flag all indels are reported.
-V, --min-coverage-for-variabilityINT- Minimum coverage of a nucleotide position to be subjected to SNV profiling. By default, anvi'o will not attempt to make sense of variation in a given nucleotide position if it is covered less than 10X. You can change that minimum using this parameter. This parameter also controls the minimum coverage for reporting indels. If an indel is observed at a position, yet the coverage of the position in the contig where the indel starts is less than this parameter, the indel will be discarded. Default:
10 --skip-SNV-profiling- By default, anvi'o characterizes single-nucleotide variation in each sample. The use of this flag will instruct profiler to skip that step. Please remember that parameters and flags must be identical between different profiles using the same contigs database for them to merge properly.
--skip-INDEL-profiling- The alignment of a read to a reference genome/sequence can be imperfect, such that the read exhibits insertions or deletions relative to the reference. Anvi'o normally stores this information in the profile database since the time taken and extra storage do not amount to much, but if you insist on not having this information, you can skip storing this information by providing this flag. Note: If --skip-SNV-profiling is provided, --skip-INDEL-profiling will automatically be enforced.
--profile-SCVs- Anvi'o can perform accurate characterization of codon frequencies in genes during profiling. While having codon frequencies opens doors to powerful evolutionary insights in downstream analyses, due to its computational complexity, this feature comes 'off' by default. Using this flag you can rise against the authority, as you always should, and make anvi'o profile codons.
--skip-edgesNUM NUCLEOTIDES- By default, anvi'o will report variable nucleotides and their allele frequencies from any nucleotide .position in a given short read found in the BAM file. Although, there are some applications where the observed variation in short reads will depend on the location of the nucleotide positions in the read. For instance, in ancient DNA sequencing, the start and the end of short reads are often suffer from DNA damage, leading to an increased number of single-nucleotide variants that emerge from the edges of short reads when they are aligned to a reference. This parameter offers a means to ameliorate the inflation of SNVs due to biases associated with short-read edges by enabling the user to ask anvi'o to ignore a few number of nucleotides from the beginning and the end of short reads as they're being profiled. For instance, a parameter of `5` will make sure that the mismatches of a given read to the reference sequence at the first and the last 5 nucleotides will not be reported in the variability table. Please note that the coverage data will not be impacted by the use of this parameter -- only what is reported as variants will be impacted, decreasing the impact of noise in specific applications. Choices:
01234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950Default:0
--cluster-contigs- Single profiles are rarely used for genome binning or visualization, and since clustering step increases the profiling runtime for no good reason, the default behavior is to not cluster contigs for individual runs. However, if you are planning to do binning on one sample, you must use this flag to tell anvi'o to run cluster configurations for single runs on your sample.
--skip-hierarchical-clustering- If you are not planning to use the interactive interface (or if you have other means to add a tree of contigs in the database) you may skip the step where hierarchical clustering of your items are preformed based on default clustering recipes matching to your database type.
--distanceDISTANCE_METRIC- The distance metric for the hierarchical clustering. Only relevant if you are using `--cluster-contigs` flag. The default is "euclidean". Default:
euclidean --linkageLINKAGE_METHOD- The linkage method for the hierarchical clustering. Just like the distance metric this is only relevant if you are using it with `--cluster-contigs` flag. The default is "ward". Default:
ward
-o, --output-dirDIR_PATH- Directory path for output files
-W, --overwrite-output-destinations- Overwrite if the output files and/or directories exist.
-T, --num-threadsNUM_THREADS- Maximum number of threads to use for multithreading whenever possible. Very conservatively, the default is 1, unless the environmental variable `ANVIO_THREADS` sets another positive integer. If you use this parameter, it will override `ANVIO_THREADS`. It is a good idea to not exceed the number of CPUs / cores on your system. Plus, please be careful with this option if you are running your commands on a SGE -- if you are clusterizing your runs, and asking for multiple threads to use, you may deplete your resources very fast. Default:
1 --queue-sizeINT- The queue size for worker threads to store data to communicate to the main thread. The default is set by the class based on the number of threads. If you have *any* hesitation about whether you know what you are doing, you should not change this value. Default:
0 --write-buffer-sizeINT- How many items should be kept in memory before they are written to the disk. The default is 5000. The larger the buffer size, the less frequently the program will access the disk, yet the more memory will be consumed since the processed items will be cleared off the memory only after they are written to the disk. The default buffer size will likely work for most cases, but if you feel you need to reduce it, we trust you. Please keep an eye on the memory usage output to make sure the memory use never exceeds the size of the physical memory. Default:
5000 --force-multi- This is not useful to non-developers. It forces the multi-process routine even when 1 thread is chosen.
Additional Resources
Are you aware of resources that may help users better understand the utility of this program? Please feel free to edit this file on GitHub. If you are not sure how to do that, find the __resources__ tag in this file to see an example.