How zamboni-lab/masster-dist works
1. What this repository is
- The repo is primarily a public distribution and documentation wrapper for MASSter, not a full source tree in this checkout.
- The visible files are license and notices; the README describes the product architecture and release model.
2. Overall shape: local, file-based LC–MS/MS analysis
- MASSter is a Python package for processing metabolomics LC–MS/MS studies on a single machine or workstation cluster.
- The central unit of work is a Study, which contains multiple Samples; processing produces persistent result files rather than relying on a server or database.
3. Primary user entry points
- The package is intended to be driven by scripts for reproducible batch processing.
- There is an integrated Wizard that automates a typical end-to-end run from raw files to a study workflow.
- For post-processing exploration, users open the saved study in Jupyter or Marimo, and there is also a browser dashboard for interactive review after processing.
4. Data ingestion boundary
- MASSter can read either vendor raw formats or open formats like mzML/mz5.
- Vendor-format access depends on .NET via pythonnet and is therefore limited to Windows in the full version.
- On Linux/macOS, the practical input boundary is mzML/mz5; the README recommends mz5 for large DIA data.
5. Core analytical pipeline
- The product covers the stages that follow raw ingestion: centroiding, feature detection, RT alignment, adduct/isotopomer detection, sample merging, gap filling, quantification, MS2 spectrum selection, visualization, and export.
- The README states feature detection still relies on OpenMS, while MASSter redesigns the rest of the pipeline around it.
6. Processing model and reuse of intermediate results
- Samples are processed individually and cached as reusable artifacts, then assembled into a study-level result.
- This two-level model suggests a boundary between sample processing and study assembly, which is important for both scalability and troubleshooting.
7. Performance and scaling architecture
- The implementation is optimized for large studies using Polars, NumPy vectorization, multiprocessing, and chunking.
- MASSter avoids loading all data at once; it reads only the relevant fractions of raw data, which is especially important for DIA/ZTScan files and large studies.
8. Persistence and portability
- Results are stored in MASSter’s own HDF5-based formats: .sample5 and .study5.
- This format is designed to be portable across operating systems and Python versions, making analysis outputs the durable handoff between processing and interactive review.
9. Outputs and downstream exchange
- The tool exports to standard metabolomics formats such as mzTab-M, MGF, and parquet.
- This makes MASSter both a processing engine and a conversion layer for downstream tools and reporting workflows.
10. Runtime and platform constraints
- The supported Python range is 3.11–3.13.
- The full version is Windows-only for proprietary raw access; the light version is cross-platform and focuses on reading, visualizing, filtering, and exporting existing MASSter results.
11. Distribution model
- The project publishes a PEP 503-compatible package index for installation via uv or pip.
- The repository therefore acts partly as a release channel, not just a code repository, and installation is expected to happen from the hosted package index.
12. System boundaries to draw in an architecture diagram
- A good diagram would show: raw/vendor or mzML/mz5 files in, sample-level processing, study assembly, persisted HDF5 result files, and then notebook/dashboard/export consumers.
- External boundaries to annotate: OpenMS for feature detection, .NET/pythonnet for vendor raw access, and the GitHub-hosted package index/wiki for distribution and documentation.
Main entry points: README.md README · LICENSE License · THIRD_PARTY_NOTICES.md Notices
Components 18
Distribution & docs
Public release wrapper, documentation, and notices in this checkout.
- masster-dist · This checkout is primarily a public distribution and documentation wrapper for MASSter.
- README · Explains the product architecture, release model, supported platforms, and workflows.
README.md - License · Repository license text.
LICENSE - Notices · Third-party licensing and attribution information.
THIRD_PARTY_NOTICES.md - Package index · PEP 503-compatible package index used for installation via pip or uv.
Processing runtime
Core analytical flow from raw input to persisted study results.
- Scripts · Reproducible script-driven runs for automated processing.
- Wizard · Automates a typical end-to-end run from raw files to a study workflow.
- Raw files · Vendor raw formats or open mzML/mz5 inputs.
- mzML/mz5 · Cross-platform input path for open formats; mz5 is recommended for large DIA data.
- Sample processing · Cached sample-level processing that extracts reusable intermediate artifacts.
- Study assembly · Combines processed samples into a study-level result and coordinates downstream analysis.
- HDF5 results · Portable .sample5 and .study5 outputs used as durable handoff files.
Review & exchange
Interactive and export consumers of persisted MASSter results.
- Notebooks · Jupyter or Marimo exploration of saved studies after processing.
- Dashboard · Interactive browser-based review of processed results.
- Exports · mzTab-M, MGF, and parquet export targets for downstream tools and reporting.
External dependencies
Key third-party and platform boundaries called out by the repository.
- Vendor access · Vendor-format access path that depends on .NET via pythonnet and is Windows-only in the full version.
- OpenMS · Third-party feature detection dependency used within the pipeline.
- Polars stack · Vectorized, chunked, multiprocessing-heavy execution model for large studies.
Connections 18
- masster-dist → README: documents · The wrapper repository is primarily explained through the README.
- masster-dist → License: includes · License is part of the public distribution.
- masster-dist → Notices: includes · Third-party attributions accompany the distribution.
- masster-dist → Package index: publishes · The repo acts partly as a release channel for installation.
- Scripts → Wizard: orchestrates · Scripted runs and the wizard are the main entry points.
- Wizard → Raw files: starts from · Wizard drives processing from raw input files.
- Raw files → Vendor access: via · Vendor raw access routes through the .NET/pythonnet boundary.
- Raw files → mzML/mz5: or · Open formats are the cross-platform input path.
- mzML/mz5 → Sample processing: feeds · Open input data enters sample-level processing.
- Vendor access → Sample processing: feeds · Vendor raw data enters the same processing pipeline when supported.
- Sample processing → OpenMS: uses · Feature detection relies on OpenMS.
- Sample processing → Polars stack: accelerated by · Processing is optimized with Polars, vectorization, chunking, and multiprocessing.
- Sample processing → HDF5 results: writes · Per-sample outputs are cached as reusable HDF5 artifacts.
- HDF5 results → Study assembly: assembled into · Cached samples are combined into a study-level result.
- Study assembly → HDF5 results: saved as · The study result is persisted as a durable .study5 artifact.
- Study assembly → Notebooks: opened in · Saved studies are explored interactively in notebooks.
- Study assembly → Dashboard: viewed in · Processed results can be reviewed in a browser dashboard.
- Study assembly → Exports: exported to · Study outputs are converted to standard exchange formats.