# visserle/pain-measurement architecture

> Architecture diagram of the GitHub repository visserle/pain-measurement, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/visserle/pain-measurement
- Repository: https://github.com/visserle/pain-measurement

## Overview

##### Project type and purpose
- This is a research-oriented data pipeline combining experimental data collection (physiological signals + questionnaires), data ingestion/storage, feature engineering, ML model training/evaluation, and reporting/visualization.
- Key use cases: run calibration & measurement sessions (audio prompts, hardware integration, Imotions), collect questionnaire responses via a small web app, ingest & anonymize raw data into a database, extract multi-modal features, train and compare time-series and ensemble models, and generate plots/notebooks for analysis.

##### High-level architectural layers
A. Data Collection Layer
– Calibration (src/experiments/calibration): scripts, audio files, estimator.
– Measurement (src/experiments/measurement): stimulus generator, rate limiter, hardware interface (imotions.py, thermoino.py), audio prompts, config/scripts.
– Questionnaires (src/experiments/questionnaires): Flask-like app.py, Jinja2 templates, CSS/static assets, scoring_schemas.
B. Data Ingestion & Storage Layer
– raw data → src/data: Imotions importer, anonymize.py, data_processing.py, quality_checks.py
– database_manager.py & database_schema.py hold data in local/remote DB
– YAML/TOML config files drive all ingestion steps
C. Feature Engineering Layer
– src/features: per-modality modules (eda.py, eeg.py, hr.py, pupil.py, face.py) plus filtering, resampling, scaling, transforming, label generation
– exploratory scripts under src/features/exploratory for EDA
D. ML Pipeline Layer
– src/models:
- data_preparation.py & sample_creation.py prepare train/test splits
- data_loader.py & scalers.py standardize inputs
- architectures (src/models/architectures): EEGNet, PatchTST, TimesNet, LSTM, ensembles, transformer blocks
- training_loop.py, hyperparameter_tuning.py, model_selection.py
- evaluation.py records metrics → results JSON, picks best checkpoints
– main.py + main_config.py drive end-to-end training runs
E. Reporting & Visualization Layer
– src/plots: scripts to generate figures (model performance, correlations, calibration curves, stimulus plots)
– Jupyter notebooks for interactive EDA & result presentation (notebooks/*.ipynb)
– reports/figures for static assets to include in write-ups
F. Orchestration & Results Storage
– run_experiments.ps1, run_model_comparison.ps1: wrap and schedule experiments
– results/ folder: hierarchies per experiment with model checkpoints, results.json, summary.txt

##### Relationships & data flow
1. Participant session → Calibration & Measurement modules (audio, hardware, questionnaire app)
→ raw data files
2. Raw data files → Data Ingestion layer → Database
3. DB records → Feature Engineering layer → Feature datasets
4. Feature datasets → ML Pipeline → Trained models + metric outputs (results/)
5. Results + models → Reporting layer (plots & notebooks) → reports

##### Architectural patterns & design principles
- Layered decomposition (collection → ingestion → features → modeling → reporting)
- Modular subpackages (clear Python modules for each concern)
- Config-driven behavior (YAML/TOML + pyproject)
- Reusable components (utilities in src/experiments/utils.py, src/features/utils.py, src/models/utils.py)
- Experiment reproducibility (scripts + versioned result directories)

##### Final pointers
– Be explicit about storage locations (DB vs. results folder).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Collection Layer
    subgraph "Data Collection Layer" 
        direction TB
        calibration["Calibration<br/>(src/experiments/calibration)"]:::collection
        measurement["Measurement<br/>(src/experiments/measurement)"]:::collection
        questionnaires["Questionnaire Web App<br/>(src/experiments/questionnaires)"]:::collection
        thermoino["Thermoino Interface<br/>(src/experiments/thermoino.py)"]:::collection
        tts["TTS Utility<br/>(src/experiments/tts.py)"]:::collection
    end

    %% Data Ingestion & Storage Layer
    subgraph "Data Ingestion & Storage Layer"
        direction TB
        imotions["Imotions Importer & Config"]:::ingestion
        anonymize["Anonymization<br/>(anonymize.py)"]:::ingestion
        processing["Data Processing & Quality Checks"]:::ingestion
        dbManager["Database Manager & Schema"]:::storage
    end

    %% Feature Engineering Layer
    subgraph "Feature Engineering Layer"
        direction TB
        coreFeatures["Core Feature Modules<br/>(eda,eeg,hr,pupil,face)"]:::features
        transforms["Transformations & Utilities"]:::features
        exploratory["Exploratory Scripts<br/>(src/features/exploratory)"]:::features
    end

    %% ML Pipeline Layer
    subgraph "ML Pipeline Layer"
        direction TB
        dataPrep["Data Preparation & Sampling"]:::modeling
        dataLoad["Data Loading & Scaling"]:::modeling
        architectures["Model Architectures<br/>(src/models/architectures)"]:::modeling
        training["Training Loop & Tuning"]:::modeling
        entryPoint["Entry Point & Config"]:::modeling
    end

    %% Reporting & Visualization Layer
    subgraph "Reporting & Visualization Layer"
        direction TB
        plots["Plotting Scripts & Style<br/>(src/plots)"]:::reporting
        notebooks["Interactive Notebooks<br/>(notebooks/)"]:::reporting
        figures["Report Figures<br/>(reports/figures)"]:::reporting
    end

    %% Orchestration & Results
    subgraph "Orchestration & Results Storage"
        direction TB
        runners["Experiment Runner Scripts"]:::storage
        resultsDir["Results Directory<br/>(results/)"]:::storage
    end

    %% Storage Cylinder
    db["Database"]:::storage

    %% Connections
    calibration -->|raw files| imotions
    measurement -->|raw files| imotions
    questionnaires -->|responses| dbManager
    thermoino --> measurement
    tts --> measurement

    imotions --> anonymize
    anonymize --> processing
    processing --> dbManager
    dbManager --> db

    db --> coreFeatures
    coreFeatures --> transforms
    transforms --> exploratory

    exploratory --> dataPrep
    dataPrep --> dataLoad
    dataLoad --> architectures
    architectures --> training
    training --> entryPoint
    entryPoint --> resultsDir
    entryPoint --> figures

    resultsDir --> plots
    resultsDir --> notebooks
    resultsDir --> figures

    runners --> calibration
    runners --> measurement
    runners --> questionnaires
    runners --> imotions
    runners --> entryPoint
    runners --> resultsDir

    %% Labels on main flow
    calibration -->|raw files| imotions
    imotions -->|anonymized data| anonymize
    anonymize -->|clean data| processing
    processing -->|db records| dbManager
    dbManager -->|feature inputs| coreFeatures
    training -->|models & metrics.json| resultsDir
    resultsDir -->|plots & notebooks| plots

    %% Click Events
    click calibration "https://github.com/visserle/pain-measurement/tree/main/src/experiments/calibration"
    click measurement "https://github.com/visserle/pain-measurement/tree/main/src/experiments/measurement"
    click questionnaires "https://github.com/visserle/pain-measurement/tree/main/src/experiments/questionnaires"
    click thermoino "https://github.com/visserle/pain-measurement/blob/main/src/experiments/thermoino.py"
    click tts "https://github.com/visserle/pain-measurement/blob/main/src/experiments/tts.py"
    click imotions "https://github.com/visserle/pain-measurement/blob/main/src/data/imotions_data.py"
    click anonymize "https://github.com/visserle/pain-measurement/blob/main/src/data/anonymize.py"
    click processing "https://github.com/visserle/pain-measurement/blob/main/src/data/data_processing.py"
    click processing "https://github.com/visserle/pain-measurement/blob/main/src/data/quality_checks.py"
    click dbManager "https://github.com/visserle/pain-measurement/blob/main/src/data/database_manager.py"
    click dbManager "https://github.com/visserle/pain-measurement/blob/main/src/data/database_schema.py"
    click coreFeatures "https://github.com/visserle/pain-measurement/blob/main/src/features/eda.py"
    click coreFeatures "https://github.com/visserle/pain-measurement/blob/main/src/features/eeg.py"
    click coreFeatures "https://github.com/visserle/pain-measurement/blob/main/src/features/hr.py"
    click coreFeatures "https://github.com/visserle/pain-measurement/blob/main/src/features/pupil.py"
    click coreFeatures "https://github.com/visserle/pain-measurement/blob/main/src/features/face.py"
    click transforms "https://github.com/visserle/pain-measurement/blob/main/src/features/filtering.py"
    click transforms "https://github.com/visserle/pain-measurement/blob/main/src/features/resampling.py"
    click transforms "https://github.com/visserle/pain-measurement/blob/main/src/features/scaling.py"
    click transforms "https://github.com/visserle/pain-measurement/blob/main/src/features/transforming.py"
    click transforms "https://github.com/visserle/pain-measurement/blob/main/src/features/labels.py"
    click transforms "https://github.com/visserle/pain-measurement/blob/main/src/features/stimulus.py"
    click transforms "https://github.com/visserle/pain-measurement/blob/main/src/features/utils.py"
    click exploratory "https://github.com/visserle/pain-measurement/tree/main/src/features/exploratory"
    click dataPrep "https://github.com/visserle/pain-measurement/blob/main/src/models/data_preparation.py"
    click dataPrep "https://github.com/visserle/pain-measurement/blob/main/src/models/sample_creation.py"
    click dataLoad "https://github.com/visserle/pain-measurement/blob/main/src/models/data_loader.py"
    click dataLoad "https://github.com/visserle/pain-measurement/blob/main/src/models/scalers.py"
    click architectures "https://github.com/visserle/pain-measurement/tree/main/src/models/architectures"
    click training "https://github.com/visserle/pain-measurement/blob/main/src/models/training_loop.py"
    click training "https://github.com/visserle/pain-measurement/blob/main/src/models/hyperparameter_tuning.py"
    click training "https://github.com/visserle/pain-measurement/blob/main/src/models/model_selection.py"
    click training "https://github.com/visserle/pain-measurement/blob/main/src/models/evaluation.py"
    click entryPoint "https://github.com/visserle/pain-measurement/blob/main/src/models/main.py"
    click entryPoint "https://github.com/visserle/pain-measurement/blob/main/src/models/main_config.py"
    click entryPoint "https://github.com/visserle/pain-measurement/blob/main/src/models/models_config.py"
    click plots "https://github.com/visserle/pain-measurement/tree/main/src/plots"
    click notebooks "https://github.com/visserle/pain-measurement/tree/main/notebooks/"
    click figures "https://github.com/visserle/pain-measurement/tree/main/reports/figures"
    click runners "https://github.com/visserle/pain-measurement/blob/main/run_experiments.ps1"
    click runners "https://github.com/visserle/pain-measurement/blob/main/run_experiments_dev.ps1"
    click runners "https://github.com/visserle/pain-measurement/blob/main/run_model_comparison.ps1"
    click resultsDir "https://github.com/visserle/pain-measurement/tree/main/results/"

    %% Styles
    classDef collection fill:#e0f7fa,stroke:#0288d1,color:#000
    classDef ingestion fill:#b3e5fc,stroke:#0288d1,color:#000
    classDef features fill:#c8e6c9,stroke:#388e3c,color:#000
    classDef modeling fill:#ffe0b2,stroke:#f57c00,color:#000
    classDef reporting fill:#e1bee7,stroke:#8e24aa,color:#000
    classDef storage fill:#eeeeee,stroke:#424242,color:#000
```
