# shalinvachheta017/masterarbeit_mlops- architecture

> Architecture diagram of the GitHub repository shalinvachheta017/masterarbeit_mlops-, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shalinvachheta017/masterarbeit_mlops-
- Repository: https://github.com/shalinvachheta017/masterarbeit_mlops-

## Overview

##### Project Type and Purpose
- This is an end-to-end MLOps pipeline for continuous mental health monitoring via wearable sensors.
- It ingests raw accelerometer/gyroscope data, preprocesses and fuses it, analyzes data quality and model architecture, then (once labels are available) trains, evaluates, and deploys a 1D-CNN-BiLSTM classifier via an inference API, with CI/CD, monitoring, and model registry.

##### High-Level Component Breakdown
a. Data Layer
- Raw sensor files (Excel) in 01_data/raw
- Processed CSVs (50 Hz fused) in 01_data/processed
- Sample/test data in 01_data/samples

b. Preprocessing
- sensor_data_pipeline.py orchestrates fusion, resampling, sliding windows (02_src/preprocessing)
- Path configuration and example usage scripts

c. Analysis
- Data quality and exploratory analyses (02_src/analysis + 04_notebooks/exploration)
- Model inspection (inspect_model.py + model_info.json)

d. Training Pipeline (planned)
- Scripts under 02_src/training (to be built)
- MLflow tracking & model registry

e. Model Artifacts
- Pretrained model metadata in 03_models/pretrained
- Future trained models in 03_models/trained

f. Evaluation & Reporting
- Metrics, confusion matrices, evaluation logs in forthcoming modules
- Output reports in 05_outputs/analysis and 05_outputs/reports

g. Serving & MLOps Infrastructure (planned)
- FastAPI inference service
- Docker containerization
- GitHub Actions CI/CD
- Monitoring & drift detection

h. Configuration & Documentation
- Requirements, linting (08_config)
- Detailed docs and mentor communication (07_docs)
- Notebook experiments (04_notebooks/experiments)

i. Logging
- Preprocessing, training, evaluation logs (06_logs)

##### Interactions and Data Flow
1. Raw sensor data → Preprocessing pipeline → Processed CSVs
2. Processed data → Data analysis scripts & notebooks → Quality reports
3. (Once labels arrive) Processed data + labels → Training pipeline → MLflow model registry
4. Trained model → Evaluation module → Evaluation reports & logs
5. Registered model → FastAPI service → External client requests
6. CI/CD pipelines (GitHub Actions) trigger builds, tests, deployments
7. Monitoring service consumes logs/metrics for drift detection

##### Architectural Patterns & Principles
- Modular, layered pipeline (separate preprocessing, analysis, training, serving)
- Clear separation of concerns: data ingestion, transformation, model ops, serving
- MLOps best practices: experiment tracking, model registry, reproducibility
- Use of configuration management (path_config, requirements.txt)
- Notebook-driven exploration vs. production scripts

##### Final Checklist
- External actors (wearable devices → data lake; API consumers)
- Call out blocking dependencies (mentor-provided labels) with annotation

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Layer
    subgraph "Data Layer"
        direction TB
        Wearables["Wearable Devices"]:::external
        Raw["Raw Sensor Files (Excel)"]:::data
        ProcCSV["Processed CSVs (50 Hz fused)"]:::data
        Samples["Sample/Test Data"]:::data
    end
    click ProcCSV "https://github.com/shalinvachheta017/masterarbeit_mlops-/tree/main/01_data/processed"
    click Samples "https://github.com/shalinvachheta017/masterarbeit_mlops-/tree/main/01_data/samples"

    %% Preprocessing Layer
    subgraph "Preprocessing"  
        direction TB
        SensorPipeline["sensor_data_pipeline.py\n(Python, Pandas, NumPy)"]:::compute
        ExampleUsage["example_usage.py"]:::compute
        PathConfig["path_config.py"]:::compute
    end
    click SensorPipeline "https://github.com/shalinvachheta017/masterarbeit_mlops-/blob/main/02_src/preprocessing/sensor_data_pipeline.py"
    click ExampleUsage "https://github.com/shalinvachheta017/masterarbeit_mlops-/blob/main/02_src/preprocessing/example_usage.py"
    click PathConfig "https://github.com/shalinvachheta017/masterarbeit_mlops-/blob/main/02_src/path_config.py"

    %% Analysis Layer
    subgraph "Analysis"
        direction TB
        DataAnalysis["analyze_data.py\n(Data Quality)"]:::compute
        ModelInspection["inspect_model.py\n(Model Info)"]:::compute
        ExplorationNotebooks["Exploration Notebooks"]:::compute
    end
    click DataAnalysis "https://github.com/shalinvachheta017/masterarbeit_mlops-/blob/main/02_src/analysis/analyze_data.py"
    click ModelInspection "https://github.com/shalinvachheta017/masterarbeit_mlops-/blob/main/02_src/analysis/inspect_model.py"
    click ExplorationNotebooks "https://github.com/shalinvachheta017/masterarbeit_mlops-/tree/main/04_notebooks/exploration"

    %% Experimental Notebooks
    subgraph "Experimental Notebooks"
        direction TB
        ExpNotebooks["Experiments Notebooks"]:::compute
    end
    click ExpNotebooks "https://github.com/shalinvachheta017/masterarbeit_mlops-/tree/main/04_notebooks/experiments"

    %% Training Pipeline (Planned)
    subgraph "Training Pipeline (awaiting labels)" 
        direction TB
        TrainingPlaceholder["Training Scripts\n(TensorFlow/Keras, MLflow)"]:::compute
    end

    %% Model Registry & Serving (Planned)
    subgraph "Model Registry & Serving"
        direction TB
        Pretrained["pretrained model_info.json"]:::registry
        Trained["Trained Models\n(MLflow Registry)"]:::registry
        FastAPI["FastAPI Inference Service\n(Docker)"]:::registry
    end
    click Pretrained "https://github.com/shalinvachheta017/masterarbeit_mlops-/blob/main/03_models/pretrained/model_info.json"

    %% Outputs & Reporting
    subgraph "Evaluation & Reporting"
        direction TB
        Reports["Analysis Reports\n& Metrics"]:::registry
    end
    click Reports "https://github.com/shalinvachheta017/masterarbeit_mlops-/tree/main/05_outputs/analysis"

    %% Infrastructure
    subgraph "Infrastructure & Docs"
        direction TB
        CICD["GitHub Actions CI/CD"]:::infra
        ConfigPylint[".pylintrc"]:::infra
        ConfigReqs["requirements.txt"]:::infra
        Docs["Project README"]:::infra
        ProjectInfo["README.md"]:::infra
        MentorComm["EMAIL_TO_MENTOR.md"]:::infra
    end
    click ConfigPylint "https://github.com/shalinvachheta017/masterarbeit_mlops-/blob/main/08_config/.pylintrc"
    click ConfigReqs "https://github.com/shalinvachheta017/masterarbeit_mlops-/blob/main/08_config/requirements.txt"
    click ProjectInfo "https://github.com/shalinvachheta017/masterarbeit_mlops-/blob/main/07_docs/project_info/README.md"
    click MentorComm "https://github.com/shalinvachheta017/masterarbeit_mlops-/blob/main/07_docs/mentor_communication/EMAIL_TO_MENTOR.md"
    click Docs "https://github.com/shalinvachheta017/masterarbeit_mlops-/blob/main/README.md"

    %% Monitoring & Logs
    Monitoring["Monitoring & Drift Detection"]:::infra
    Logs["Preprocessing & Training Logs"]:::infra

    %% External Client
    Client["API Consumers"]:::external

    %% Connections
    Wearables -->|"Uploads Excel"| Raw
    Raw -->|"Pandas→CSV"| SensorPipeline
    SensorPipeline -->|"Writes CSV"| ProcCSV
    SensorPipeline --> ExampleUsage
    ExampleUsage --> PathConfig
    ProcCSV -->|"Feeds Processed Data"| DataAnalysis
    ProcCSV --> ExplorationNotebooks
    ProcCSV --> ExpNotebooks
    DataAnalysis -->|"Generates Reports"| Reports
    ProcCSV -->|"Await labels"| TrainingPlaceholder
    TrainingPlaceholder -->|"MLflow API"| Trained
    Trained --> Pretrained
    Trained --> FastAPI
    FastAPI -->|"HTTP/JSON"| Client
    CICD --> SensorPipeline
    CICD --> TrainingPlaceholder
    CICD --> FastAPI
    SensorPipeline --> Logs
    TrainingPlaceholder --> Logs
    FastAPI --> Monitoring
    Logs --> Monitoring

    %% Styles
    classDef data fill:#cce5ff,stroke:#3399ff
    classDef compute fill:#d4edda,stroke:#28a745
    classDef registry fill:#ffe5b4,stroke:#ff9900
    classDef infra fill:#e2e3e5,stroke:#6c757d
    classDef external fill:#f8d7da,stroke:#dc3545,stroke-dasharray: 5 5
```
