# shalindasilva1/detached-eeg architecture

> Architecture diagram of the GitHub repository shalindasilva1/detached-eeg, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shalindasilva1/detached-eeg
- Repository: https://github.com/shalindasilva1/detached-eeg

## Overview

##### Read the file tree as architecture
Key structure signals:

- **Entrypoints**
- `src/pipeline.py` → orchestrates the full run (config load → data load → splits → pre-transform → train/eval → save JSON)
- `src/visualize_results.py` → consumes `results/*.json` → outputs figures

- **Core library / utilities**
- `src/utl/config.py` → configuration loading / normalization (likely YAML)
- `src/utl/bids.py` → BIDS dataset discovery/loading
- `src/utl/eeg.py` → EEG-specific preprocessing/format conversion
- `src/utl/splits.py` → CV and specifically LOSO split logic

- **Configs / environments**
- `config.yml` → main runtime config
- `environment.yml`, `requirements.txt` → dependency/environment definition
- `experiments/configs/*.yaml` → experiment variants (cnn vs rocket, photic/rest etc.)

- **Artifacts**
- `results/loso_results.json`, `results/cv_results.json` → metrics
- `results/figures/` → plots generated from results JSON

- **Verification**

##### pipeline.py → (preprocesses/reshapes) → via `utl/eeg.py`
5. **pipeline.py** → (creates folds) → via `utl/splits.py` → produces **LOSO train/test indices**
6. **pipeline.py** → (GPU pre-transform) → **detach_rocket feature extraction** → produces **transformed feature matrices**
##### pipeline.py → (trains) → ensemble models → (predicts) → fold metrics
8. **pipeline.py** → (aggregates) → **subject-level metrics** → (writes) → `results/loso_results.json`
##### visualize_results.py → (reads) → `results/*.json` → (writes) → `results/figures/*`

A **conditional GPU path**:

##### Architectural patterns to explicitly call out
This project is best represented as:

- **Batch pipeline / offline experiment runner**
- **Configuration-driven execution**
- `config.yml` + experiment YAMLs drive behavior; code is mostly orchestration + reusable utilities.
- **Separation of concerns**
- Orchestrator (`pipeline.py`) vs utilities (`src/utl/*`) vs reporting (`visualize_results.py`)
- **Precomputation optimization**
- “pre-transformation strategy” is a key architectural choice: expensive kernel extraction is done upfront (GPU), then reused across LOSO folds.

A callout annotation near “Detach-Rocket feature extraction”:
- “Extract 10,000+ kernels once on GPU upfront → reuse across LOSO folds (time optimization).”

- **Python 3.10**
- **PyTorch + CUDA** (for detach_rocket acceleration)
- **detach-rocket** (external algorithm implementation)
- **BIDS** (input data format contract)
- (Optionally) plotting stack for visualization (matplotlib/seaborn) if visible in requirements; otherwise just “Plotting”.

- Actors: Researcher/User
- Containers: Pipeline Runner, Visualization Runner, Dataset (BIDS), Results Store, GPU/CUDA (as infra)
- Purpose: show the big moving pieces and file-based interfaces.

- Expand Pipeline Runner into stages:
- Config Load → BIDS Load → EEG Preprocess → LOSO Splitter → Pre-transform (Rocket kernels) → Train Ensemble → Evaluate/Aggregate → Write JSON
- This is the “most accurate” depiction of this repository’s architecture.

##### Be “very detailed” by splitting into many components (without inventing services)
Because this is not microservices, detail comes from **pipeline stages** and **module boundaries**, not from adding fake servers.

##### Write Results → [`results/loso_results.json`]

Separate branch:
**[User]** → **[visualize_results.py]** → (reads) **[`results/*.json`]** → **[`results/figures/*`]**

- “Optimization: pre-transform once (GPU) then reuse across LOSO folds.”

## Diagram (Mermaid)

```mermaid
flowchart TD

%% =========================
%% Level 1: Context / Containers (C4-ish)
%% =========================
subgraph "Level 1 — Offline Experiment Context"
direction TB

U["Researcher / User"]:::actor

subgraph "Execution (Batch Runners)"
direction TB
P["Pipeline Orchestrator\n(src/pipeline.py)"]:::runner
V["Visualization Runner\n(src/visualize_results.py)"]:::runner
end

subgraph "Configuration (Documents)"
direction TB
CFG["Main Config\n(config.yml)\nkeys: data.resting, experiment.n_splits,\nmodel.params.num_models, model.params.num_kernels"]:::doc
ECFG["Experiment Variant Configs\n(experiments/configs/*.yaml)"]:::doc
ENV["Conda Environment\n(environment.yml)"]:::doc
REQ["pip Requirements\n(requirements.txt)"]:::doc
end

subgraph "Data Stores (Filesystem)"
direction TB
BIDS[("BIDS EEG/MEG Dataset\n(external folder path)")]:::store
RJSON[("Results Metrics JSON\nresults/loso_results.json\nresults/cv_results.json")]:::store
FIGS[("Generated Figures\nresults/figures/*")]:::store
end

subgraph "ML Compute / External Dependencies"
direction TB
DR["detach-rocket\n(Feature Extraction)"]:::external
GPU["PyTorch + CUDA GPU\n(optional / recommended)"]:::external
CPU["CPU Fallback\n(if CUDA unavailable)"]:::external
end

end

%% Context connections (control + data)
U -->|"executes"| P
U -->|"executes"| V

P -.->|"loads"| CFG
P -.->|"loads"| ECFG
P -.->|"runs_in"| ENV
P -.->|"installs"| REQ

P -->|"reads_BIDS"| BIDS
P -->|"writes_metrics"| RJSON

V -->|"reads_results"| RJSON
V -->|"writes_plots"| FIGS

P -->|"uses"| DR
P -->|"CUDA_available?"| GPU
P -->|"CUDA_unavailable"| CPU

%% =========================
%% Level 2: Pipeline Dataflow (code-traceable stages)
%% =========================
subgraph "Level 2 — Pipeline Dataflow (src/pipeline.py)"
direction TB

S0["Start Run"]:::stage
S1["Load + Normalize Config\n(via src/utl/config.py)"]:::stage
S2["Discover / Load BIDS Dataset\n(via src/utl/bids.py)"]:::stage
S3["EEG Preprocess / Reshape\n(via src/utl/eeg.py)"]:::stage
S4["Create CV Splits\nLOSO (via src/utl/splits.py)"]:::stage

subgraph "Pre-Transformation Strategy (Optimization)"
direction TB
S5["Detach-Rocket Pre-transform\nExtract 10,000+ kernels upfront\n(num_kernels from config)\nOutput: transformed feature matrices"]:::stage
NOTE["Note: Pre-transform once (GPU) then reuse\nacross all LOSO folds (time optimization)."]:::note
end

S6["Train Ensemble (per fold)\nnum_models = 10\nFit models on train subjects"]:::stage
S7["Evaluate + Aggregate Metrics\nfold_metrics -> subject-level accuracy\n(confusion, trial accuracy)"]:::stage
S8["Persist Results\nWrite results/*.json"]:::stage

%% Stage-to-stage flow
S0 --> S1
S1 --> S2
S2 -->|"raw_epochs + metadata"| S3
S3 -->|"X,y + subject_ids"| S4
S4 -->|"train/test_indices per subject"| S5
S5 -->|"features reused across folds"| S6
S6 -->|"predictions"| S7
S7 --> S8

%% External data/config linkages for Level 2
CFG -.-> S1
ECFG -.-> S1
BIDS --> S2
S8 --> RJSON

%% GPU/CPU conditional path for pre-transform
S5 -->|"accelerate_kernels"| GPU
S5 -->|"fallback_compute"| CPU

NOTE -.-> S5
end

%% =========================
%% Visualization dataflow
%% =========================
subgraph "Post-run Reporting (src/visualize_results.py)"
direction TB
VR0["Start Visualization"]:::stage
VR1["Load Results JSON\n(results/*.json)"]:::stage
VR2["Generate Plots\nConfusion Matrix + Trial Accuracy"]:::stage
VR3["Write Figures\n(results/figures/*)"]:::stage

VR0 --> VR1 --> VR2 --> VR3
RJSON --> VR1
VR3 --> FIGS
end

%% =========================
%% Verification (tests)
%% =========================
subgraph "Verification"
direction TB
T1["Pipeline Tests\n(tests/test_pipeline.py)\nValidates pipeline behavior"]:::test
end
T1 -.->|"exercises"| P

%% =========================
%% Click Events (from component_mapping)
%% =========================
click P "https://github.com/shalindasilva1/detached-eeg/blob/main/src/pipeline.py"
click V "https://github.com/shalindasilva1/detached-eeg/blob/main/src/visualize_results.py"

click S1 "https://github.com/shalindasilva1/detached-eeg/blob/main/src/utl/config.py"
click CFG "https://github.com/shalindasilva1/detached-eeg/blob/main/config.yml"
click ECFG "https://github.com/shalindasilva1/detached-eeg/tree/main/experiments/configs/"

click S2 "https://github.com/shalindasilva1/detached-eeg/blob/main/src/utl/bids.py"
click S3 "https://github.com/shalindasilva1/detached-eeg/blob/main/src/utl/eeg.py"
click S4 "https://github.com/shalindasilva1/detached-eeg/blob/main/src/utl/splits.py"

click RJSON "https://github.com/shalindasilva1/detached-eeg/blob/main/results/loso_results.json"
click RJSON "https://github.com/shalindasilva1/detached-eeg/blob/main/results/cv_results.json"
click FIGS "https://github.com/shalindasilva1/detached-eeg/tree/main/results/figures/"

click ENV "https://github.com/shalindasilva1/detached-eeg/blob/main/environment.yml"
click REQ "https://github.com/shalindasilva1/detached-eeg/blob/main/requirements.txt"

click T1 "https://github.com/shalindasilva1/detached-eeg/blob/main/tests/test_pipeline.py"

%% =========================
%% Styles / Legend
%% =========================
subgraph "Legend"
direction TB
L1["Blue = Compute Stage / Runner"]:::stage
L2[("Green = Data Store (Filesystem)")]:::store
L3["Orange = External Lib / Accelerator"]:::external
L4["Gray = Document / Config"]:::doc
L5["Pink = Test"]:::test
L6["Yellow = Note / Annotation"]:::note
end

classDef actor fill:#111827,stroke:#111827,color:#FFFFFF,stroke-width:2px
classDef runner fill:#1D4ED8,stroke:#1E3A8A,color:#FFFFFF,stroke-width:2px
classDef stage fill:#2563EB,stroke:#1E3A8A,color:#FFFFFF,stroke-width:2px
classDef store fill:#16A34A,stroke:#14532D,color:#FFFFFF,stroke-width:2px
classDef external fill:#F59E0B,stroke:#92400E,color:#111827,stroke-width:2px
classDef doc fill:#9CA3AF,stroke:#374151,color:#111827,stroke-width:2px
classDef test fill:#EC4899,stroke:#9D174D,color:#FFFFFF,stroke-width:2px
classDef note fill:#FDE68A,stroke:#B45309,color:#111827,stroke-width:2px
```
