How umcugenetics/dims works
Project Type and Purpose
• Type: A bioinformatics data‐processing pipeline (Nextflow + R + HPC + containers).
• Purpose: Automate Direct Infusion Mass Spectrometry (DIMS) data processing end-to-end: raw data ingestion → preprocessing → feature extraction → reporting.
File Structure
• Top‐level “DIMS.nf” and “assets/workflow.nf”: the Nextflow workflow definitions.
• “CustomModules/”: repository of per‐step scripts (R, shell, etc.) invoked by Nextflow processes.
• “db/”: static reference data (HMDB .RData files, theoretical mass tables).
• “Dockerfile” + “.gitmodules”: containerization and dependency declarations.
• “run_nextflow_dims.sh”: CLI wrapper for launching the pipeline on HPC.
Extract README Insights
• Two “frontends” for launching: an R Shiny GUI and a CLI script.
• Execution environment: Docker (locally) or Singularity (HPC).
• External tools: ThermoRawFileParser for raw file conversion; R libraries (xcms, dplyr, etc.).
• HPC scheduler: Slurm (srun).
Main Components
a. User Interfaces
• R Shiny GUI (optional front‐end)
• CLI wrapper (run_nextflow_dims.sh)
b. Workflow Orchestrator
• Nextflow engine (DIMS.nf + assets/workflow.nf)
c. Task Modules
• CustomModules scripts (R, shell) – each Nextflow “process”
d. Data Stores
• Input storage: raw .raw files + sample sheet
• Reference DB: db/ RData files and lookup tables
• Output storage: processed data, reports
e. Containers & Execution Environment
• Docker image (local) → Singularity image (HPC)
• External tool container: ThermoRawFileParser under /tools
f. Compute Infrastructure
• HPC cluster managed by Slurm
• Worker nodes / temporary storage
Relationships and Data Flows
• User initiates via GUI or CLI → triggers Nextflow run.
• Nextflow engine pulls container image → schedules processes on HPC nodes.
• Each process reads raw inputs + reference DB → executes CustomModules code.
• ThermoRawFileParser invoked where needed.
• Intermediate files flow through Nextflow channels → finally aggregate into output directory.
• Email notification sent on completion.
Architectural Patterns and Principles
• Pipeline Orchestration (Nextflow’s dataflow model)
• Containerization for reproducibility (Docker/Singularity)
• Modular design: each “process” is a self-contained script under CustomModules
• Separation of concerns: UI vs. orchestration vs. compute vs. storage