anvi-run-workflow
Table of Contents
Execute, manage, parallelize, and troubleshoot entire 'omics workflows and chain together anvi'o and third party programs.
🔙 To the main page of anvi’o programs and artifacts.
Authors
Requires
Provides
Usage
This program allows you to run a workflow implemented by anvi’o developers for various commonly used set of steps to typically process your raw data (i.e., short reads or contigs from genomes, transcriptomes, metagenomes, metatranscriptomes, etc). Some aspects of this program is described in this tutorial.
For a list of currently available anvi’o workflows, please see the workflow artifact.
Before running the workflow
Each workflow requires a workflow-config: the file that details all of the parameters for the workflow. To get the workflow-config with the default parameters, just run
anvi-run-workflow -w workflow \ --get-default-config CONFIG.json
Before running a workflow, it is also a good idea to check the required dependencies by running
anvi-run-workflow -w workflow \ --list-dependencies
The main run
The main run of the workflow should look like this:
anvi-run-workflow -w workflow \ -c CONFIG.json --save-workflow-graph
The flag --save-workflow-graph creates a visual representation of the anvio programs that the workflow you’re running used.
You can also use the -A flag at the end of the parameter list to change other Snakemake parameters.
Logs and the workflow manifest
Every workflow run writes rule logs under 00_LOGS, within a subdirectory for the workflow or named run. Logs are then organized by rule name:
00_LOGS/<workflow-or-run-name>/<rule-name>/<job-specific-name>.log
For instance, a metagenomics profiling job may write to 00_LOGS/metagenomics/anvi_profile/G01-S01.log, while an SRA download checksum job may write to 00_LOGS/sra_download/check_md5sum/SRR5965623.log.
Each workflow run also creates a tab-delimited manifest in the same workflow-specific log directory:
00_LOGS/<workflow-or-run-name>/<workflow-name>-workflow-manifest.tsv
This file lists the status of each Snakemake job, the rule name, the group and read wildcards when they exist, the rule log path, and the Snakemake log path when Snakemake reports one. If a workflow stops because a rule failed, this manifest is often the quickest way to find the relevant rule log.
Edit this file to update this information.
Command line interface
The following is the list of parameters and their descriptions in the latest version of anvi-run-workflow.
The code that generates this web page ensures that this information is the latest available in 9-dev (eunice). Which means, in some cases the information in this section (generated automatically) might be more comprehensive or up-to-date compared to the descriptions above (written by humans).
Command line usage
Parameters and their descriptions
-w, --workflowWORKFLOW- You must specify a workflow name. To see a list of available workflows run --list-workflows.
--get-default-configOUTPUT_FILENAME- Store a json formatted config file with all the default settings of the workflow. This is a good draft you could use in order to write your own config file. This config file contains all parameters that could be configured for this workflow. NOTICE: the config file is provided with default values only for parameters that are set by us in the workflow. The values for the rest of the parameters are determined by the relevant program.
--list-workflows- Print a list of available snakemake workflows
--list-dependencies- Print a list of the dependencies of this workflow. You must provide a workflow name and a config file. snakemake will figure out which rules need to be run according to your config file, and according to the files available on your disk. According to the rules that need to be run, we will let you know which programs are going to be used, so that you can make sure you have all of them installed and loaded.
-c, --config-fileCONFIG_FILE- A JSON-formatted configuration file.
--dry-run- Don't do anything real. Test everything, and stop right before wherever the developer said 'well, this is enough testing', and decided to print out results.
--skip-dry-run- Don't do a dry run. Just start the workflow! Useful when your job is so big it takes hours to do a dry run.
--save-workflow-graph- Save a graph representation of the workflow. If you are using this flag and if your system is unable to generate such graph outputs, you will hear anvi'o complaining (still, totally worth trying).
-A, --additional-paramsADDITIONAL_PARAMS- Additional snakemake parameters to add when running snakemake. NOTICE: --additional-params HAS TO BE THE LAST ARGUMENT THAT IS PASSED TO anvi-run-workflow, ANYTHING THAT FOLLOWS WILL BE CONSIDERED AS PART OF THE ADDITIONAL PARAMETERS THAT ARE PASSED TO SNAKEMAKE. Any parameter that is accepted by snakemake should be fair game here, but it is your responsibility to make sure that whatever you added makes sense. To see what parameters are available please refer to the snakemake documentation. For example, you could use this to set up cluster submission using --additional-params --cluster 'YOUR-CLUSTER-SUBMISSION-CMD'.
Additional Resources
Are you aware of resources that may help users better understand the utility of this program? Please feel free to edit this file on GitHub. If you are not sure how to do that, find the __resources__ tag in this file to see an example.