# shifts-project/shifts architecture

> Architecture diagram of the GitHub repository shifts-project/shifts, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shifts-project/shifts
- Repository: https://github.com/shifts-project/shifts

## Overview

##### Main Components

a) Dataset Components (Core):
- Multiple Sclerosis Lesions Segmentation (Medical Imaging)
- Vessel Power Estimation (Time Series)
- Weather Prediction (Tabular)
- Machine Translation (NLP)
- Motion Prediction (Autonomous Driving)

b) Framework Components:
- Data Loading & Processing
- Model Training
- Evaluation & Metrics
- Uncertainty Quantification
- Visualization Tools

**┌─ Data Sources Layer ─┐    ┌─ Processing Layer ─┐    ┌─ Analysis Layer ─┐**
- MS Lesions        │ -> │ - Data Loading     │ -> │ - Evaluation     │
- Vessel Power      │    │ - Preprocessing    │    │ - Metrics        │
- Weather           │    │ - Transformations  │    │ - Uncertainty    │
- Translation       │    └──────────────────┘    │ - Visualization  │
- Motion Prediction │                            └─────────────────┘
- ┘

##### Component Details

- Raw data storage
- Processed datasets
- Evaluation sets

b) Processing Components (Rectangles):
- Data loaders (PyTorch/TensorFlow)
- Preprocessing pipelines
- Feature extractors

c) Analysis Components (Hexagons):
- Metrics calculation
- Uncertainty estimation
- Visualization tools

##### Specific Implementation Details

Project-specific elements:
**┌─ Task-Specific Modules ─┐**
- mswml/ - MS lesions    │
- vpower/ - Vessel power │
- weather/ - Weather     │
- translation/ - MT      │
- sdc/ - Motion pred.    │
- ┘

##### Integration Points

Connections between:
- Dataset APIs (ysdc_dataset_api/)
- Evaluation metrics (metrics.py in each module)
- Uncertainty quantification (uncertainty.py)

##### Important Notes

- Dataset licenses and sources
- Evaluation points and metrics
- Uncertainty quantification methods
- Integration with external tools (WandB, CSV logging)

##### Standardized metrics across tasks

This architecture should capture both the high-level organization and the detailed implementation structure while maintaining clarity and usefulness for development and documentation purposes.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Style Definitions
    classDef data fill:#b3d9ff,stroke:#333
    classDef processing fill:#90EE90,stroke:#333
    classDef analysis fill:#FFB366,stroke:#333
    classDef uncertainty fill:#DDA0DD,stroke:#333
    classDef utils fill:#D3D3D3,stroke:#333

    %% Data Sources Layer
    subgraph DataSources
        MS["MS Lesions"]:::data
        VP["Vessel Power"]:::data
        WP["Weather Prediction"]:::data
        MT["Machine Translation"]:::data
        MP["Motion Prediction"]:::data
    end

    %% Processing Layer
    subgraph Processing
        DL["Data Loading"]:::processing
        PP["Preprocessing"]:::processing
        TR["Transformations"]:::processing
    end

    %% Training Layer
    subgraph Training
        MST["MS Training"]:::processing
        SDCT["SDC Training"]:::processing
        MTT["Translation Training"]:::processing
    end

    %% Analysis Layer
    subgraph Analysis
        EM["Evaluation & Metrics"]:::analysis
        UQ["Uncertainty Quantification"]:::uncertainty
        VT["Visualization Tools"]:::utils
    end

    %% Logging Layer
    subgraph Logging
        WB["WandB Logger"]:::utils
        CSV["CSV Logger"]:::utils
    end

    %% Connections
    DataSources --> Processing
    Processing --> Training
    Training --> Analysis
    Analysis --> Logging

    %% Click Events
    click MS "https://github.com/Shifts-Project/shifts/tree/main/mswml/"
    click VP "https://github.com/Shifts-Project/shifts/tree/main/vpower/"
    click WP "https://github.com/Shifts-Project/shifts/tree/main/weather/"
    click MT "https://github.com/Shifts-Project/shifts/tree/main/translation/"
    click MP "https://github.com/Shifts-Project/shifts/tree/main/sdc/"
    click MST "https://github.com/Shifts-Project/shifts/blob/main/mswml/train.py"
    click SDCT "https://github.com/Shifts-Project/shifts/blob/main/sdc/sdc/trainer.py"
    click MTT "https://github.com/Shifts-Project/shifts/blob/main/translation/assessment/eval_ensemble.sh"
    click DL "https://github.com/Shifts-Project/shifts/blob/main/sdc/ysdc_dataset_api/dataset/dataset.py"
    click TR "https://github.com/Shifts-Project/shifts/blob/main/sdc/ysdc_dataset_api/utils/transform.py"
    click EM "https://github.com/Shifts-Project/shifts/blob/main/mswml/metrics.py"
    click UQ "https://github.com/Shifts-Project/shifts/blob/main/mswml/uncertainty.py"
    click VT "https://github.com/Shifts-Project/shifts/blob/main/vpower/src/utils/plot_utils.py"
    click WB "https://github.com/Shifts-Project/shifts/blob/main/sdc/sdc/oatomobile/utils/loggers/wandb.py"
    click CSV "https://github.com/Shifts-Project/shifts/blob/main/sdc/sdc/oatomobile/utils/loggers/csv.py"

    %% Legend
    subgraph Legend
        L1["Data Sources"]:::data
        L2["Processing"]:::processing
        L3["Analysis"]:::analysis
        L4["Uncertainty"]:::uncertainty
        L5["Utilities"]:::utils
    end
```
