How merenlab/anvio works
Main Components to Capture
• User Interfaces – Different modes are available:
– Command‐line interface: all the bin/ scripts (e.g., “anvi-analyze-synteny”, “anvi-compute-completeness”, etc.) that serve as entry points.
– Interactive visualization: found under the “anvio/data/interactive” folder with HTML, CSS, and JavaScript components for displaying charts, contigs, gene clusters, and workflow outputs.
• Core Python Library (located in the “anvio” directory):
– Core modules and utilities (e.g., clustering, gene calling, taxonomy operations, sequence operations, variability, etc.).
– Data submodules and drivers: drivers, parsers, and workflows (Snakefiles) that interface with external tools such as BLAST, HMMER, Prodigal, and others.
– Migration and database management: see “anvio/migrations”, “anvio/tables”, and “anvio/db.py”, which comprise data transformation, storage, and update logic.
• Workflow Components:
– The workflow files (Snakefiles) in “anvio/workflows” that orchestrate various analyses (metagenomics, pangenomics, phylogenomics, etc.).
– Each workflow also encapsulates rules and scripts that integrate with the core python modules.
• External Tools and Integration Layers:
– A suite of drivers (in “anvio/drivers”) wrap external tools, ensuring that anvi’o can call programs like Diamond, FastANI, Muscle, etc.
– Configuration files (like the YAML and INI files in sandbox/test directories) define how analyses run.
• Containerization and Deployment:
– Docker-related files (e.g., Dockerfiles in the “Dockerfiles” directory) indicate containerized environments for reproducible execution.
– CI/CD workflows in “.github/workflows” that run tests and migrations.
• Supporting Documentation and Community:
– Extensive documentation is maintained in “anvio/docs” (including detailed program and workflow instructions) and the README which directs users to tutorials and help pages.
Identifying Relationships and Data Flow
– Represent data flows between components such as:
• Input data (e.g., genomic sequences, contigs, alignment files) fed into pipelines.
• Data transformation modules (migrations, table creation, clustering, annotation) that process and store results.
• Visualization modules that output interactive HTML/JS pages.
• Configuration management (YAML/INI files) guiding workflow behavior.
– Show dependencies on external tools as separate blocks that are encapsulated by drivers/wrappers.
Architectural Patterns and Design Principles
– Modularity: The project is divided along functional lines (analysis, visualization, database/storage, and workflows).
– Extensibility: Drivers and parsers are clearly separated, allowing easy addition of new bioinformatics tools.
– Workflow orchestration: Use of Snakemake workflows indicates a pipeline pattern, where specific input data flows through rule-based processes.
– Containerization: Dockerfiles indicate that components can be deployed in isolated containers, ensuring reproducibility.