# s3n-w6i/drino architecture

> Architecture diagram of the GitHub repository s3n-w6i/drino, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/s3n-w6i/drino
- Repository: https://github.com/s3n-w6i/drino

## Overview

Key Components to Include in the System Design:

##### Core System Components

a) Data Processing Pipeline:
- Step 1: Data Fetching (GTFS)
- Step 2: Data Import
- Step 3: Data Validation
- Step 4: Data Merging
- Step 5: Simplification

b) Routing Engine Components:
- RAPTOR Algorithm Implementation
- STP (Scalable Transfer Patterns)
- Transfer Pattern Processing
As interconnected modules within the routing component.

c) Visualization Layer:
- A separate section for the visualization stack:
- Backend API (Rust-based)
- Frontend UI (React-based)

##### Architectural Patterns to Highlight

- Data Layer (bottom)
- Processing Layer (middle)
- Presentation Layer (top)

- Data processing components (e.g., blue)
- Algorithmic components (e.g., green)
- API/UI components (e.g., purple)

##### Important Details

a) Preprocessing Pipeline:
- Clustering algorithms (DBSCAN, GMM, K-means, OPTICS)
- Preprocessing requirements (RAM < 2GB, Time < 3h)

b) Query Processing:
- Query flow through the system
- Performance constraints (< 50ms, < 500MB RAM)

##### Technical Implementation Details

- Rust backend components
- React frontend components
- Data storage/processing components
- API endpoints

b) Data structures:
- Transfer Pattern Graph
- Transfer Pattern Table
- Direct Connections

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Legend
        L1[Backend Component]:::backend
        L2[Frontend Component]:::frontend
        L3[Data Processing]:::data
        L4[Algorithm]:::algorithm
        L5[("Database")]:::database
    end

    subgraph "Data Processing Pipeline"
        DF["Data Fetching"]:::data
        DI["Data Import"]:::data
        DV["Data Validation"]:::data
        DM["Data Merging"]:::data
        DS["Data Simplification"]:::data
        DF --> DI --> DV --> DM --> DS
    end

    subgraph "Routing Engine Core"
        RAPTOR["RAPTOR Algorithm"]:::algorithm
        STP["Scalable Transfer Patterns"]:::algorithm
        TPP["Transfer Pattern Processing"]:::algorithm
        RAPTOR <--> STP
        STP <--> TPP
    end

    subgraph "Clustering Algorithms"
        DBSCAN["DBSCAN"]:::algorithm
        GMM["GMM"]:::algorithm
        KMEANS["K-means"]:::algorithm
        OPTICS["OPTICS"]:::algorithm
    end

    subgraph "Data Structures"
        TPG["Transfer Pattern Graph"]:::data
        TPT["Transfer Pattern Table"]:::data
        DC["Direct Connections"]:::data
    end

    subgraph "Visualization Layer"
        API["Backend API"]:::backend
        UI["Frontend UI"]:::frontend
        UICOMP["UI Components"]:::frontend
        ROUTES["Frontend Routes"]:::frontend
        API --> UI
        UI --> UICOMP
        UI --> ROUTES
    end

    subgraph "Common Utilities"
        CONFIG["Configuration"]:::backend
        DATASET["Dataset Types"]:::backend
        ERRORS["Error Handling"]:::backend
        UTIL["Utility Functions"]:::backend
    end

    subgraph "Performance Requirements"
        PREPROC["Preprocessing: RAM < 2GB, Time < 3h"]:::requirement
        QUERY["Query: < 50ms, < 500MB RAM"]:::requirement
    end

    DS --> RAPTOR
    DBSCAN & GMM & KMEANS & OPTICS --> STP
    TPP --> TPG
    TPP --> TPT
    TPP --> DC
    RAPTOR --> API
    STP --> API
    TPP --> API

    %% Click events for component mapping
    click DF "https://github.com/s3n-w6i/drino/blob/master/data-harvester/src/step1_fetch_data.rs"
    click DI "https://github.com/s3n-w6i/drino/tree/master/data-harvester/src/step2_import_data/"
    click DV "https://github.com/s3n-w6i/drino/tree/master/data-harvester/src/step3_validate_data/"
    click DM "https://github.com/s3n-w6i/drino/blob/master/data-harvester/src/step4_merge_data.rs"
    click DS "https://github.com/s3n-w6i/drino/blob/master/data-harvester/src/step5_simplify.rs"
    click RAPTOR "https://github.com/s3n-w6i/drino/tree/master/routing/src/raptor/"
    click STP "https://github.com/s3n-w6i/drino/tree/master/routing/src/stp/"
    click TPP "https://github.com/s3n-w6i/drino/tree/master/routing/src/tp/"
    click DBSCAN "https://github.com/s3n-w6i/drino/blob/master/routing/src/stp/preprocessing/clustering/dbscan.rs"
    click GMM "https://github.com/s3n-w6i/drino/blob/master/routing/src/stp/preprocessing/clustering/gmm.rs"
    click KMEANS "https://github.com/s3n-w6i/drino/blob/master/routing/src/stp/preprocessing/clustering/k_means.rs"
    click OPTICS "https://github.com/s3n-w6i/drino/blob/master/routing/src/stp/preprocessing/clustering/optics_geo.rs"
    click TPG "https://github.com/s3n-w6i/drino/blob/master/routing/src/tp/transfer_pattern_ds/graph.rs"
    click TPT "https://github.com/s3n-w6i/drino/blob/master/routing/src/tp/transfer_pattern_ds/table.rs"
    click DC "https://github.com/s3n-w6i/drino/blob/master/routing/src/direct_connections.rs"
    click API "https://github.com/s3n-w6i/drino/tree/master/visualization/src/api/"
    click UI "https://github.com/s3n-w6i/drino/tree/master/visualization/ui/app/"
    click UICOMP "https://github.com/s3n-w6i/drino/tree/master/visualization/ui/app/components/"
    click ROUTES "https://github.com/s3n-w6i/drino/tree/master/visualization/ui/app/routes/"
    click CONFIG "https://github.com/s3n-w6i/drino/blob/master/common/src/types/config.rs"
    click DATASET "https://github.com/s3n-w6i/drino/blob/master/common/src/types/dataset.rs"
    click ERRORS "https://github.com/s3n-w6i/drino/blob/master/common/src/types/errors.rs"
    click UTIL "https://github.com/s3n-w6i/drino/tree/master/common/src/util/"

    classDef backend fill:#6c8ebf,stroke:#000
    classDef frontend fill:#d4b6db,stroke:#000
    classDef data fill:#a8d08d,stroke:#000
    classDef algorithm fill:#ffe699,stroke:#000
    classDef database fill:#f8cecc,stroke:#000
    classDef requirement fill:#fff2cc,stroke:#000
```
