# newcastleuniversity/dispel architecture

> Architecture diagram of the GitHub repository newcastleuniversity/dispel, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/newcastleuniversity/dispel
- Repository: https://github.com/newcastleuniversity/dispel

## Overview

##### Project Type and Purpose
- DISPEL is an open-source Python library for Digital Signal Processing (DSP).
- It standardizes extraction of sensor-derived measures (SDMs) from wearables or smartphone data via a modular, object-oriented pipeline.

##### High-Level Architecture
DISPEL is organized as a layered, plugin-based toolkit rather than a classic front-end/backend web app. It consists of:
A. Data Providers (plugins for different device/data formats)
B. IO Layer (loading, exporting, utility routines)
C. Core Data Model (in-memory representations: collections, epochs, flags, measures, traces)
D. Processing Pipeline (assertions, filtering, epoching, transform, feature extraction)
E. Signal Processing Primitives (filter, DTW, orientation, vector math)
F. Statistical Modules (learning, reliability)
G. Documentation Extensions (Sphinx directives)
H. Utilities & Registry

##### Main Components & Relationships
1. Data Sources (External)
- Smartphone, wearables, CSV, HDF5, JSON, proprietary formats (ADS, APDM, BDH, sensorlog, etc.)
- Supplied by “dispel/providers/*” modules
2. IO Layer (“dispel/io”)
- Core I/O classes handle raw data import and standardized export
- Utilities handle format detection & conversion
3. Registry (“dispel/providers/registry.py”)
- Dynamically discovers registered providers & tasks
4. Core Data Model (“dispel/data”)
- Central domain objects: DataCollection, DataSet, DataTrace, Epoch, Flag, Measure, Validators
5. Processing Pipeline (“dispel/processing”)
- Pipeline orchestrator calls modules in sequence: assertions → extract → transform → level & flagging → epoch segmentation → measure computation
6. Signal Processing (“dispel/signal”)
- Low-level DSP building blocks (filters, interpolators, orientation estimation, DTW); consumed by processing modules
7. Statistics (“dispel/stats”)
- Post-processing analyses: learning algorithms, reliability metrics
8. Documentation & CLI
- Sphinx extension for measure docs (“dispel/sphinxext”)
- Package entry points (setup.cfg) for console scripts (if any)

Data Flow:
[Providers] → [IO Layer] → [Core Data Model] → [Processing Pipeline] → [Stats Module] → [Export/Visualization]

##### Architectural Patterns & Design Principles
- Plugin/Registry Pattern: “dispel/providers” auto‐registered tasks for extensibility
- Layered Architecture: clear separation I/O → model → processing → analysis
- Single Responsibility: each module focuses on one stage (e.g., raw import, flagging, transform)
- Dependency Injection: pipeline orchestrator accepts configurable steps & parameters

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Data Sources
    subgraph "External Data Sources"
        DS1["Smartphone & Wearables"]:::external
        DS2["CSV/JSON/HDF5 Files"]:::external
        DS3["Proprietary Formats"]:::external
    end

    %% Plugin Providers
    subgraph "Plugin Providers"
        PR["Data Providers"]:::provider
        REG["Provider Registry"]:::provider
    end

    %% IO Layer
    subgraph "IO Layer"
        IO_core["Core I/O"]:::core
        IO_raw["Raw Import"]:::core
        IO_export["Export Routines"]:::core
        IO_flags["Flags I/O"]:::core
        IO_utils["Format Utils"]:::core
    end

    %% Core Data Model
    subgraph "Core Data Model"
        DM_core["Domain Objects"]:::core
        DM_collections["Collections"]:::core
        DM_epochs["Epochs"]:::core
        DM_flags["Flags"]:::core
        DM_levels["Levels"]:::core
        DM_measures["Measures"]:::core
        DM_raw["Raw Wrapper"]:::core
        DM_validators["Validators"]:::core
        DM_values["Values"]:::core
        DM_devices["Device Metadata"]:::core
    end

    %% Processing Pipeline
    subgraph "Processing Pipeline"
        PP_core["Pipeline Orchestrator"]:::core
        PP_assert["Assertions"]:::core
        PP_extract["Extraction"]:::core
        PP_transform["Transformation"]:::core
        PP_epochs["Epoch Segmentation"]:::core
        PP_flags["Flagging"]:::core
        PP_level["Level Computation"]:::core
        PP_filters["Level Filters"]:::core
        PP_dataset["DataSet Abstraction"]:::core
        PP_datatrace["DataTrace Abstraction"]:::core
        PP_trace["Trace Processing"]:::core
        PP_modalities["Modality Logic"]:::core
        PP_utils["Pipeline Utils"]:::core
    end

    %% Signal Processing Primitives
    subgraph "Signal Processing"
        SIG_core["Signal Core"]:::core
        SIG_filter["Filtering"]:::core
        SIG_dtw["DTW"]:::core
        SIG_accel["Accelerometer Routines"]:::core
        SIG_interp["Interpolators"]:::core
        SIG_orient["Orientation Estimation"]:::core
        SIG_geom["Geometric Transforms"]:::core
        SIG_vector["Vector Math"]:::core
        SIG_sensor["Sensor Utils"]:::core
    end

    %% Statistical Modules
    subgraph "Statistical Modules"
        ST_core["Stats Core"]:::core
        ST_learning["Learning Algorithms"]:::core
        ST_reliability["Reliability Metrics"]:::core
    end

    %% Documentation & Utilities
    subgraph "Docs & Utilities"
        DOC_sphinx["Sphinx Extensions"]:::util
        UTIL_general["General Utils"]:::util
        CLI["CLI Entry Points"]:::util
    end

    %% Testing Harness
    subgraph "Testing Harness"
        TEST_data["tests/data"]:::util
        TEST_io["tests/io"]:::util
        TEST_processing["tests/processing"]:::util
        TEST_providers["tests/providers"]:::util
        TEST_signal["tests/signal"]:::util
        TEST_stats["tests/stats"]:::util
    end

    %% Connections
    DS1 --> PR
    DS2 --> PR
    DS3 --> PR
    PR --> IO_core
    PR --> IO_raw
    PR --> IO_export
    PR --> IO_flags
    PR --> IO_utils
    PR -.-> REG
    IO_core --> DM_core
    IO_raw --> DM_raw
    IO_export --> DM_core
    IO_flags --> DM_flags
    IO_utils --> DM_core
    DM_core --> PP_core
    DM_collections --> PP_core
    DM_epochs --> PP_epochs
    DM_flags --> PP_flags
    DM_levels --> PP_level
    DM_measures --> PP_core
    DM_raw --> PP_extract
    DM_validators --> PP_assert
    DM_values --> PP_core
    DM_devices --> PP_core
    PP_core --> PP_assert
    PP_assert --> PP_extract
    PP_extract --> PP_transform
    PP_transform --> PP_filters
    PP_filters --> PP_epochs
    PP_epochs --> PP_level
    PP_level --> PP_flags
    PP_flags --> PP_trace
    PP_trace --> PP_dataset
    PP_dataset --> PP_datatrace
    PP_datatrace --> PP_modalities
    PP_modalities --> ST_core
    PP_utils -.-> PP_core
    ST_core --> ST_learning
    ST_core --> ST_reliability
    ST_learning --> IO_export
    ST_reliability --> IO_export
    SIG_core --> PP_core
    SIG_filter --> PP_transform
    SIG_dtw --> PP_transform
    SIG_accel --> PP_extract
    SIG_interp --> PP_extract
    SIG_orient --> PP_transform
    SIG_geom --> PP_transform
    SIG_vector --> PP_transform
    SIG_sensor --> PP_extract
    DOC_sphinx --> UTIL_general
    UTIL_general --> PP_core
    CLI --> PP_core
    TEST_data --> DM_core
    TEST_io --> IO_core
    TEST_processing --> PP_core
    TEST_providers --> PR
    TEST_signal --> SIG_core
    TEST_stats --> ST_core

    %% Click Events for Component Mapping
    click PR "https://github.com/newcastleuniversity/dispel/tree/main/dispel/providers/"
    click REG "https://github.com/newcastleuniversity/dispel/blob/main/dispel/providers/registry.py"
    click IO_core "https://github.com/newcastleuniversity/dispel/blob/main/dispel/io/core.py"
    click IO_raw "https://github.com/newcastleuniversity/dispel/blob/main/dispel/io/raw.py"
    click IO_export "https://github.com/newcastleuniversity/dispel/blob/main/dispel/io/export.py"
    click IO_flags "https://github.com/newcastleuniversity/dispel/blob/main/dispel/io/flags.py"
    click IO_utils "https://github.com/newcastleuniversity/dispel/blob/main/dispel/io/utils.py"
    click DM_core "https://github.com/newcastleuniversity/dispel/blob/main/dispel/data/core.py"
    click DM_collections "https://github.com/newcastleuniversity/dispel/blob/main/dispel/data/collections.py"
    click DM_epochs "https://github.com/newcastleuniversity/dispel/blob/main/dispel/data/epochs.py"
    click DM_flags "https://github.com/newcastleuniversity/dispel/blob/main/dispel/data/flags.py"
    click DM_levels "https://github.com/newcastleuniversity/dispel/blob/main/dispel/data/levels.py"
    click DM_measures "https://github.com/newcastleuniversity/dispel/blob/main/dispel/data/measures.py"
    click DM_raw "https://github.com/newcastleuniversity/dispel/blob/main/dispel/data/raw.py"
    click DM_validators "https://github.com/newcastleuniversity/dispel/blob/main/dispel/data/validators.py"
    click DM_values "https://github.com/newcastleuniversity/dispel/blob/main/dispel/data/values.py"
    click DM_devices "https://github.com/newcastleuniversity/dispel/blob/main/dispel/data/devices.py"
    click PP_core "https://github.com/newcastleuniversity/dispel/blob/main/dispel/processing/core.py"
    click PP_assert "https://github.com/newcastleuniversity/dispel/blob/main/dispel/processing/assertions.py"
    click PP_extract "https://github.com/newcastleuniversity/dispel/blob/main/dispel/processing/extract.py"
    click PP_transform "https://github.com/newcastleuniversity/dispel/blob/main/dispel/processing/transform.py"
    click PP_epochs "https://github.com/newcastleuniversity/dispel/blob/main/dispel/processing/epochs.py"
    click PP_flags "https://github.com/newcastleuniversity/dispel/blob/main/dispel/processing/flags.py"
    click PP_level "https://github.com/newcastleuniversity/dispel/blob/main/dispel/processing/level.py"
    click PP_filters "https://github.com/newcastleuniversity/dispel/blob/main/dispel/processing/level_filters.py"
    click PP_dataset "https://github.com/newcastleuniversity/dispel/blob/main/dispel/processing/data_set.py"
    click PP_datatrace "https://github.com/newcastleuniversity/dispel/blob/main/dispel/processing/data_trace.py"
    click PP_trace "https://github.com/newcastleuniversity/dispel/blob/main/dispel/processing/trace.py"
    click PP_modalities "https://github.com/newcastleuniversity/dispel/blob/main/dispel/processing/modalities.py"
    click PP_utils "https://github.com/newcastleuniversity/dispel/blob/main/dispel/processing/utils.py"
    click SIG_core "https://github.com/newcastleuniversity/dispel/blob/main/dispel/signal/core.py"
    click SIG_filter "https://github.com/newcastleuniversity/dispel/blob/main/dispel/signal/filter.py"
    click SIG_dtw "https://github.com/newcastleuniversity/dispel/blob/main/dispel/signal/dtw.py"
    click SIG_accel "https://github.com/newcastleuniversity/dispel/blob/main/dispel/signal/accelerometer.py"
    click SIG_interp "https://github.com/newcastleuniversity/dispel/blob/main/dispel/signal/interpolators.py"
    click SIG_orient "https://github.com/newcastleuniversity/dispel/blob/main/dispel/signal/orientation.py"
    click SIG_geom "https://github.com/newcastleuniversity/dispel/blob/main/dispel/signal/geometric.py"
    click SIG_vector "https://github.com/newcastleuniversity/dispel/blob/main/dispel/signal/vectorial.py"
    click SIG_sensor "https://github.com/newcastleuniversity/dispel/blob/main/dispel/signal/sensor.py"
    click ST_core "https://github.com/newcastleuniversity/dispel/blob/main/dispel/stats/core.py"
    click ST_learning "https://github.com/newcastleuniversity/dispel/blob/main/dispel/stats/learning.py"
    click ST_reliability "https://github.com/newcastleuniversity/dispel/blob/main/dispel/stats/reliability.py"
    click DOC_sphinx "https://github.com/newcastleuniversity/dispel/blob/main/dispel/sphinxext/dispel_directive.py"
    click UTIL_general "https://github.com/newcastleuniversity/dispel/blob/main/dispel/utils.py"

    %% Styles
    classDef external fill:#aee,stroke:#333,stroke-width:1px
    classDef provider fill:#f96,stroke:#333,stroke-width:1px
    classDef core fill:#9f9,stroke:#333,stroke-width:1px
    classDef util fill:#ccc,stroke:#333,stroke-width:1px
```
