# sleep-revolution/esadafileconsumer architecture

> Architecture diagram of the GitHub repository sleep-revolution/esadafileconsumer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sleep-revolution/esadafileconsumer
- Repository: https://github.com/sleep-revolution/esadafileconsumer

## Overview

##### Main Components

a. Data Input Layer:
- Entry point through `RunOnOneFile.py` and `Processor.py`
- File input handling capabilities

b. Core Processing Pipeline:
Into three main processing blocks:

##### Preprocessing Module
- Multiple preprocessing filters as subcomponents:
- Bandpass filter (bpfilter.py)
- Chebyshev high-pass filter (chebyhpfilter.py)
- Signal harmonisation (harmonisation.py)
- IQR standardization (iqrstd.py)
- Resampling (resample.py)
- Reshaping (reshape.py)
- Spectrogram generation (spectrograms.py)

##### Model Module
- Mixture models component (MixtureModels.py)
- Main model implementation (model.py)
- API integration (noxsasapi.py)

##### Data Processing Module
- Data generation (datagenerator.py)
- Part signal processing (partsignal.py)
- Prediction handling (prediction.py)
- Predictors management (predictors.py)

c. Utility Layer:
- Configuration management (yamlutils.py)
- Experiment saving (save_xp.py)
- System introspection (introspection.py)

**[Input Layer]**
- ↓
**[Preprocessing Layer]**
- ↓
**[Model Layer]**
- ↓
**[Data Processing Layer]**
- ↓
**[Output/Results]**

##### Additional Elements

- Version/dependency information from requirements.txt
- Docker container boundaries
- Entry points and exit points clearly

##### Implementation Notes

- Maintain consistent spacing and alignment
- Clear labels for all components
- Brief descriptions for complex interactions

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Styles and classes
    classDef input fill:#a8d1f0,stroke:#000
    classDef preprocessing fill:#90EE90,stroke:#000
    classDef model fill:#FFB347,stroke:#000
    classDef data fill:#DDA0DD,stroke:#000
    classDef utility fill:#D3D3D3,stroke:#000
    classDef config fill:#F0E68C,stroke:#000
    classDef container fill:#E6E6FA,stroke:#000

    %% Input Layer
    subgraph Input
        RunFile["RunOnOneFile.py"]
        Processor["Processor.py"]
    end
    class RunFile,Processor input

    %% Preprocessing Layer
    subgraph Preprocessing
        direction TB
        BPFilter["Bandpass Filter"]
        ChebyFilter["Chebyshev Filter"]
        Harmonisation["Signal Harmonisation"]
        IQRStd["IQR Standardization"]
        Resample["Resampling"]
        Reshape["Reshaping"]
        Spectrograms["Spectrogram Generation"]
    end
    class BPFilter,ChebyFilter,Harmonisation,IQRStd,Resample,Reshape,Spectrograms preprocessing

    %% Model Layer
    subgraph ModelLayer
        direction TB
        MixMod["Mixture Models"]
        CoreModel["Core Model"]
        API["NOXSAS API"]
        TrainedModel[("MM10scorer_aSAGA.pickle")]
    end
    class MixMod,CoreModel,API,TrainedModel model

    %% Data Processing Layer
    subgraph DataProcessing
        direction TB
        DataGen["Data Generator"]
        SignalProc["Signal Processing"]
        PredHandler["Prediction Handler"]
        PredMgr["Predictors Manager"]
    end
    class DataGen,SignalProc,PredHandler,PredMgr data

    %% Utility Layer
    subgraph Utilities
        direction TB
        YAMLUtils["YAML Utils"]
        SaveXP["Experiment Saver"]
        Introspect["System Introspector"]
        PreProConf["PreProConf.yaml"]
    end
    class YAMLUtils,SaveXP,Introspect,PreProConf utility

    %% Container Configuration
    subgraph Container
        Docker["Dockerfile"]
        Reqs["requirements.txt"]
    end
    class Docker,Reqs container

    %% Connections
    RunFile --> Processor
    Processor --> Preprocessing
    Preprocessing --> ModelLayer
    ModelLayer --> DataProcessing
    YAMLUtils --> Preprocessing
    YAMLUtils --> ModelLayer
    YAMLUtils --> DataProcessing

    %% Click events
    click RunFile "https://github.com/Sleep-Revolution/EsadaFileConsumer/blob/main/RunOnOneFile.py"
    click Processor "https://github.com/Sleep-Revolution/EsadaFileConsumer/blob/main/Processor.py"
    click BPFilter "https://github.com/Sleep-Revolution/EsadaFileConsumer/blob/main/src/preprocessing/bpfilter.py"
    click ChebyFilter "https://github.com/Sleep-Revolution/EsadaFileConsumer/blob/main/src/preprocessing/chebyhpfilter.py"
    click Harmonisation "https://github.com/Sleep-Revolution/EsadaFileConsumer/blob/main/src/preprocessing/harmonisation.py"
    click IQRStd "https://github.com/Sleep-Revolution/EsadaFileConsumer/blob/main/src/preprocessing/iqrstd.py"
    click Resample "https://github.com/Sleep-Revolution/EsadaFileConsumer/blob/main/src/preprocessing/resample.py"
    click Reshape "https://github.com/Sleep-Revolution/EsadaFileConsumer/blob/main/src/preprocessing/reshape.py"
    click Spectrograms "https://github.com/Sleep-Revolution/EsadaFileConsumer/blob/main/src/preprocessing/spectrograms.py"
    click MixMod "https://github.com/Sleep-Revolution/EsadaFileConsumer/blob/main/src/model/MixtureModels.py"
    click CoreModel "https://github.com/Sleep-Revolution/EsadaFileConsumer/blob/main/src/model/model.py"
    click API "https://github.com/Sleep-Revolution/EsadaFileConsumer/blob/main/src/model/noxsasapi.py"
    click TrainedModel "https://github.com/Sleep-Revolution/EsadaFileConsumer/blob/main/src/run/MM10scorer_aSAGA.pickle"
    click DataGen "https://github.com/Sleep-Revolution/EsadaFileConsumer/blob/main/src/data/datagenerator.py"
    click SignalProc "https://github.com/Sleep-Revolution/EsadaFileConsumer/blob/main/src/data/partsignal.py"
    click PredHandler "https://github.com/Sleep-Revolution/EsadaFileConsumer/blob/main/src/data/prediction.py"
    click PredMgr "https://github.com/Sleep-Revolution/EsadaFileConsumer/blob/main/src/data/predictors.py"
    click YAMLUtils "https://github.com/Sleep-Revolution/EsadaFileConsumer/blob/main/src/utils/yamlutils.py"
    click SaveXP "https://github.com/Sleep-Revolution/EsadaFileConsumer/blob/main/src/utils/save_xp.py"
    click Introspect "https://github.com/Sleep-Revolution/EsadaFileConsumer/blob/main/src/utils/introspection.py"
    click PreProConf "https://github.com/Sleep-Revolution/EsadaFileConsumer/blob/main/src/run/PreProConf.yaml"
    click Docker "https://github.com/Sleep-Revolution/EsadaFileConsumer/tree/main/Dockerfile"
    click Reqs "https://github.com/Sleep-Revolution/EsadaFileConsumer/blob/main/requirements.txt"
```
