How shalinvachheta017/masterarbeit_mlops- works
Project Type and Purpose
• This is an end-to-end MLOps pipeline for continuous mental health monitoring via wearable sensors.
• It ingests raw accelerometer/gyroscope data, preprocesses and fuses it, analyzes data quality and model architecture, then (once labels are available) trains, evaluates, and deploys a 1D-CNN-BiLSTM classifier via an inference API, with CI/CD, monitoring, and model registry.
High-Level Component Breakdown
a. Data Layer
• Raw sensor files (Excel) in 01_data/raw
• Processed CSVs (50 Hz fused) in 01_data/processed
• Sample/test data in 01_data/samples
b. Preprocessing
• sensor_data_pipeline.py orchestrates fusion, resampling, sliding windows (02_src/preprocessing)
• Path configuration and example usage scripts
c. Analysis
• Data quality and exploratory analyses (02_src/analysis + 04_notebooks/exploration)
• Model inspection (inspect_model.py + model_info.json)
d. Training Pipeline (planned)
• Scripts under 02_src/training (to be built)
• MLflow tracking & model registry
e. Model Artifacts
• Pretrained model metadata in 03_models/pretrained
• Future trained models in 03_models/trained
f. Evaluation & Reporting
• Metrics, confusion matrices, evaluation logs in forthcoming modules
• Output reports in 05_outputs/analysis and 05_outputs/reports
g. Serving & MLOps Infrastructure (planned)
• FastAPI inference service
• Docker containerization
• GitHub Actions CI/CD
• Monitoring & drift detection
h. Configuration & Documentation
• Requirements, linting (08_config)
• Detailed docs and mentor communication (07_docs)
• Notebook experiments (04_notebooks/experiments)
i. Logging
• Preprocessing, training, evaluation logs (06_logs)
Interactions and Data Flow
1. Raw sensor data → Preprocessing pipeline → Processed CSVs
2. Processed data → Data analysis scripts & notebooks → Quality reports
3. (Once labels arrive) Processed data + labels → Training pipeline → MLflow model registry
4. Trained model → Evaluation module → Evaluation reports & logs
5. Registered model → FastAPI service → External client requests
6. CI/CD pipelines (GitHub Actions) trigger builds, tests, deployments
7. Monitoring service consumes logs/metrics for drift detection
Architectural Patterns & Principles
• Modular, layered pipeline (separate preprocessing, analysis, training, serving)
• Clear separation of concerns: data ingestion, transformation, model ops, serving
• MLOps best practices: experiment tracking, model registry, reproducibility
• Use of configuration management (path_config, requirements.txt)
• Notebook-driven exploration vs. production scripts
Final Checklist
• External actors (wearable devices → data lake; API consumers)
• Call out blocking dependencies (mentor-provided labels) with annotation