How tns-schrauwen/popqc works
PopQC is a Rust workspace organized as a pipeline: discover QC outputs, parse them into a common model, analyze cohort-level distributions/outliers, then render a standalone HTML report. The main executable is popqc-cli, and the rest of the crates are library boundaries that isolate discovery, parsing, analysis, and report generation.
1. Top-level orchestration
The CLI (crates/popqc-cli) is the only user-facing entrypoint. It accepts one or more input directories, optional metadata, and report settings, then coordinates the full run. The repo is built as a workspace, so the CLI depends on the other crates rather than embedding all logic directly.
2. Input sources
The primary input is QC output from genomics pipelines, especially multiqc_data directories and other tool-specific logs. The CLI also accepts sample metadata as TSV/CSV, where one column is treated as the sample ID and the remaining columns become report annotations. This metadata is a separate input stream from the QC metrics.
3. Discovery layer
crates/popqc-discovery is responsible for walking the provided paths and finding candidate QC files within a bounded directory depth. This is the boundary between arbitrary filesystem layout and the normalized parsing pipeline. It allows PopQC to work across different workflow conventions without the CLI needing to know file names up front.
4. Parser layer
crates/popqc-parsers converts tool-specific outputs into structured records. The repo shows explicit support for MultiQC table exports and STAR-specific outputs, and the parser registry suggests a plugin-like architecture for adding more formats. This layer is where heterogeneous QC files are mapped into a unified internal representation.
5. Core domain model
crates/popqc-core contains the central data model and analysis logic. The main submodules are model, metrics, config, normalize, outlier, error, and traits, which indicates that raw parsed values are normalized, scored/compared, and fed into outlier detection using shared abstractions. This crate is the main “business logic” boundary of the project.
6. Normalization and cohort analysis
The normalize and metrics modules suggest a cohort-wide processing step that makes different QC metrics comparable across samples and tools. The outlier module is likely where failing samples or unusual observations are detected based on distributional behavior across the cohort. This is the main analytical stage that differentiates PopQC from a pure QC viewer.
7. Report generation
crates/popqc-report turns the analyzed cohort data into an HTML artifact. The presence of html.rs and the README’s “self-contained interactive HTML report” imply that the report bundles the necessary presentation assets into a single file with no server dependency. This is the final sink of the pipeline.
8. Output boundary
The product of a run is a report file on disk, not a long-running service. That makes the architecture batch-oriented and offline: inputs are read from the filesystem, transformed in memory, and written once as HTML. There is no database or backend API in this repository.
9. Testing and fixtures
The repository includes mirrored fixtures under both crates/popqc-cli/tests and top-level tests, which likely exercise end-to-end parsing and report generation using real-looking multiqc_data and metadata files. These fixtures define the expected input contract and help validate interoperability with common genomics QC outputs.
10. Deployment and packaging
The project is packaged for CLI distribution via Cargo, Docker, and Bioconda, but those are delivery formats rather than runtime subsystems. Architecturally, they all wrap the same compiled Rust binary and do not introduce separate services.
11. Important conceptual flow for a diagram
A useful diagram would show: filesystem inputs and metadata → discovery → parser registry and format-specific parsers → normalized core model/metrics → outlier analysis → HTML report writer → single report file. The CLI sits above the whole flow as the coordinator and user interface.
12. Key architectural boundary
The strongest boundary in the repo is between “data ingestion/parsing” and “analysis/reporting.” Discovery and parsers are responsible for understanding external QC formats; core and report are responsible for PopQC’s cohort logic and presentation. That separation is what makes the tool extensible to additional genomics workflows without changing the report pipeline.
Main entry points: registry.rs Parser registry · main.rs popqc-cli · lib.rs Discovery · model.rs Core model
Components 17
Entry & inputs
User-facing command plus filesystem inputs
- popqc-cli · Main executable that coordinates the full run
crates/popqc-cli/src/main.rs - QC inputs · Directories containing QC outputs and logs
- Sample metadata · Optional sample annotations joined into the report
crates/popqc-cli/tests/fixtures/test_data/metadata.tsv
Ingestion & parsing
Discovery and format-specific QC parsing
- Discovery · Finds candidate QC files within bounded directory depth
crates/popqc-discovery/src/lib.rs - Parser registry · Dispatches discovered files to the appropriate parser
crates/popqc-parsers/src/registry.rs - MultiQC tables · Parses MultiQC table exports into structured records
crates/popqc-parsers/src/multiqc_table.rs - STAR parser · Parses STAR-specific QC outputs
crates/popqc-parsers/src/star.rs - Parsers crate · Shared parsing boundary for heterogeneous QC formats
crates/popqc-parsers/src/lib.rs
Core analysis
Normalized model, metrics, and outlier detection
- Core model · Normalized sample and metric representation
crates/popqc-core/src/model.rs - Metrics · Shared metric definitions used for cohort comparison
crates/popqc-core/src/metrics.rs - Normalize · Maps parsed values into comparable cohort-ready form
crates/popqc-core/src/normalize.rs - Outlier detection · Flags unusual samples or observations from cohort distributions
crates/popqc-core/src/outlier.rs - Config & traits · Shared analysis configuration and abstraction traits
crates/popqc-core/src/config.rs
Reporting
HTML report generation and output
- Report crate · Builds the final HTML artifact from analyzed cohort data
crates/popqc-report/src/lib.rs - HTML render · Generates a self-contained interactive report document
crates/popqc-report/src/html.rs - HTML report · Standalone report written to disk
docs/sample_report/test_popqc_report.html
Fixtures
Representative real-world test inputs
- Test data · End-to-end fixture corpus for discovery, parsing, and reporting
tests/fixtures/test_data/multiqc_data/multiqc_general_stats.txt
Connections 17
- QC inputs → Discovery: scan · Walk input directories and locate candidate QC files
- Sample metadata → popqc-cli: join · Optional sample annotations are provided to the CLI
- popqc-cli → Discovery: orchestrate · CLI coordinates discovery as the first pipeline stage
- Discovery → Parsers crate: handoff · Discovered files are handed to the parsing layer
- Parsers crate → Parser registry: dispatch · Parser library routes files by detected format
- Parser registry → MultiQC tables: parse · MultiQC exports are parsed into structured records
- Parser registry → STAR parser: parse · STAR outputs are parsed into structured records
- MultiQC tables → Core model: map · Format-specific records are converted into the common model
- STAR parser → Core model: map · Format-specific records are converted into the common model
- Core model → Normalize: prepare · Raw model values are normalized for cohort comparison
- Metrics → Normalize: measure · Metric definitions guide normalization and comparison
- Normalize → Outlier detection: analyze · Normalized cohort values feed outlier detection
- Config & traits → Outlier detection: configure · Analysis behavior is controlled by shared config and traits
- Outlier detection → Report crate: summarize · Analysis results are packaged for reporting
- Report crate → HTML render: render · Report crate renders the final standalone HTML artifact
- HTML render → HTML report: write · Rendered HTML is written as a single output file
- Test data → popqc-cli: validate · Representative fixture data exercises the end-to-end pipeline