How pbfslab/deepprep works
Project Type and Purpose
• DeepPrep is a neuroimaging preprocessing pipeline that uses deep‐learning modules (FastSurfer, FastCSR, SUGAR, SynthMorph) orchestrated by Nextflow.
• It ingests BIDS‐formatted MRI data, applies multiple deep‐learning inference steps, handles volumetric and surface spaces, and produces preprocessed outputs plus QC and runtime reports.
File Structure
• Top‐level “Docker” folder: Dockerfiles and Singularity definitions for CPU/GPU containers.
• “deepprep” folder:
– Subdirectories for each module: FastCSR, FastSurfer (wrapper around FreeSurfer), SUGAR, SynthMorph.
– Nextflow pipeline definitions under “nextflow/” (deepprep.nf, config files, cluster configs).
– QC scripts under “qc/” (qc.nf, QC Python bins).
– Web UI under “web/” (Python pages for interactive runs).
– Entry‐point shell script “deepprep.sh” and Python “setup.py” for CLI installation.
• “docs/”: Sphinx documentation sources.
Main Components
A. Data Ingestion
• BIDS-formatted MRI dataset (Anat & BOLD)
B. Workflow Manager
• Nextflow core (“deepprep.nf” + configs)
• Executor back‐ends: local, Docker, Singularity, SLURM
C. Deep‐Learning Modules (each in containers)
1. FastSurfer (cortical surface reconstruction)
2. FastCSR (cortical surface registration)
3. SUGAR (surface‐based registration multi‐level)
4. SynthMorph (volumetric normalization)
5. FreeSurfer licensing fallback
D. QC & Reporting
• QC Nextflow subworkflow (“qc.nf”) calling Python scripts → HTML/PDF reports
• Runtime profiling reports via Nextflow reports
E. Web Interface
• Streamlit‐style pages under “web/pages” for GUI runs
F. Container Layer
• Dockerfiles (project‐wide + per‐module)
• Singularity definitions
G. External Services & Config
• FreeSurfer license server/file
• HPC cluster scheduler (SLURM, etc.)
Relationships & Interactions
• Within Manager, fan out to each deep‐learning module in parallel (if resources permit).
• Modules read/write to a shared “work” directory or object store.
• Process outputs feed QC workflow and final “derivatives” directory.
• Web UI triggers Nextflow with user parameters and displays logs/reports.
• Container layer encloses Nextflow and module tasks.
• Scheduler (SLURM/Docker) sits under Nextflow for resource allocation.
Architectural Patterns & Design Principles
• Pipeline Orchestration Pattern: Nextflow defines directed acyclic graph (DAG) of processes.
• Modular Plugin Pattern: each algorithm is a self‐contained module with its own Dockerfile.
• Separation of Concerns: data ingestion, processing, QC, reporting, and UI are distinct layers.
• Infrastructure As Code: cluster and container configs versioned alongside pipeline code.
Technologies & Frameworks to Annotate
• Nextflow DSL (pipeline scripts .nf, .config)
• Python 3.x (modules, QC scripts, web UI)
• Docker & Singularity (containerization)
• BIDS standard (input/output structure)
• HPC schedulers (SLURM, local executors)
• Sphinx (documentation)
Essential Details to Capture
• The two major workflows: anatomical (T1w) and functional (BOLD) pipelines
• Interdependency among modules (e.g., FastSurfer output → FastCSR input)
• QC subworkflow triggered post‐processing
• Container orchestration by Nextflow executors
• Support for Docker vs. Singularity vs. local vs. HPC
• Web UI invoking the same Nextflow pipeline with custom parameters