# kobinabrandon/hourly-divvy-trip-predictor architecture

> Architecture diagram of the GitHub repository kobinabrandon/hourly-divvy-trip-predictor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kobinabrandon/hourly-divvy-trip-predictor
- Repository: https://github.com/kobinabrandon/hourly-divvy-trip-predictor

## Overview

##### Main Components Overview

a) Feature Pipeline
b) Training Pipeline
c) Inference Pipeline

##### Detailed Component Breakdown

Feature Pipeline Section:
- Preprocessing components showing:
- Data sourcing (from data_sourcing.py)
- Preprocessing (from preprocessing.py)
- Feature engineering (from feature_engineering.py)
- Indexing components (mixed_indexer.py and rounding_indexer.py)
- Data flow to Hopsworks Feature Store

Training Pipeline Section:
- Components for:
- Model architecture selection (models.py)
- Hyperparameter tuning component (hyperparameter_tuning.py)
- Training process (training.py)
- Connection to CometML for model registry
- Model versioning and tracking

Inference Pipeline Section:
- Split into Backend and Frontend subsections:
Backend:
- Feature store API interaction
- Model registry API interaction
- Inference engine
- Backfill mechanism
Frontend:
- Main dashboard (main.py)
- Predictions view
- Monitoring dashboard
- Data visualization components

##### Integration Points and Data Flow

Integration points with external services:
- Hopsworks Feature Store
- CometML Model Registry
- GitHub Actions for automated backfilling

##### Specific Design Elements

Use the following visual elements:

**Data Sources → Feature Pipeline → Feature Store → Training Pipeline → Model Registry → Inference Pipeline → Frontend**

##### Monitoring and Logging

A cross-cutting concern layer showing:
- Monitoring components (monitoring.py)
- Performance tracking
- Error logging

##### Key Technologies to Highlight

- Python (core language)
- Streamlit (frontend)
- Hopsworks (feature store)
- CometML (model registry)
- Docker (containerization)
- GitHub Actions (automation)

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph Legend
        direction TB
        L1[Process]:::process
        L2[(Database)]:::database
        L3([External Service]):::external
        L4[[API Component]]:::api
        L5[/Frontend Component/]:::frontend
    end

    subgraph "Feature Pipeline"
        direction TB
        DataSource[Data Sourcing]:::process
        Preprocess[Preprocessing]:::process
        FeatEng[Feature Engineering]:::process
        MixedIndex[Mixed Indexer]:::process
        RoundIndex[Rounding Indexer]:::process
        GeoData[(Geographical Data)]:::database
        
        DataSource --> Preprocess
        Preprocess --> FeatEng
        FeatEng --> MixedIndex
        FeatEng --> RoundIndex
        MixedIndex --> GeoData
    end

    subgraph "Training Pipeline"
        direction TB
        ModelArch[Model Architecture]:::process
        HyperTune[Hyperparameter Tuning]:::process
        Training[Training Process]:::process
        
        ModelArch --> HyperTune
        HyperTune --> Training
    end

    subgraph "Inference Pipeline"
        direction TB
        subgraph "Backend"
            direction TB
            FeatureAPI[[Feature Store API]]:::api
            ModelAPI[[Model Registry API]]:::api
            InferenceEng[Inference Engine]:::process
            Backfill[Backfill Mechanism]:::process
        end

        subgraph "Frontend"
            direction TB
            MainDash[/Main Dashboard/]:::frontend
            PredView[/Predictions View/]:::frontend
            MonitorDash[/Monitoring Dashboard/]:::frontend
            DataViz[/Data Visualization/]:::frontend
            DataView[/Data View/]:::frontend
        end
    end

    subgraph "External Services"
        direction TB
        Hopsworks([Hopsworks]):::external
        CometML([CometML]):::external
        GHActions([GitHub Actions]):::external
    end

    subgraph "Monitoring"
        direction TB
        Monitor[Monitoring System]:::process
        Config[Configuration]:::process
        Paths[Path Configuration]:::process
    end

    %% Connections
    DataSource --> Hopsworks
    Training --> CometML
    Backfill --> GHActions
    FeatureAPI --> MainDash
    ModelAPI --> PredView
    InferenceEng --> DataViz

    %% Styles
    classDef process fill:#a8d1ff,stroke:#000
    classDef database fill:#ffb366,stroke:#000
    classDef external fill:#90EE90,stroke:#000
    classDef api fill:#FF69B4,stroke:#000
    classDef frontend fill:#DDA0DD,stroke:#000

    %% Click Events
    click DataSource "https://github.com/kobinabrandon/Hourly-Divvy-Trip-Predictor/blob/main/src/feature_pipeline/data_sourcing.py"
    click Preprocess "https://github.com/kobinabrandon/Hourly-Divvy-Trip-Predictor/blob/main/src/feature_pipeline/preprocessing.py"
    click FeatEng "https://github.com/kobinabrandon/Hourly-Divvy-Trip-Predictor/blob/main/src/feature_pipeline/feature_engineering.py"
    click MixedIndex "https://github.com/kobinabrandon/Hourly-Divvy-Trip-Predictor/blob/main/src/feature_pipeline/mixed_indexer.py"
    click RoundIndex "https://github.com/kobinabrandon/Hourly-Divvy-Trip-Predictor/blob/main/src/feature_pipeline/rounding_indexer.py"
    click GeoData "https://github.com/kobinabrandon/Hourly-Divvy-Trip-Predictor/tree/main/data/geographical/mixed_indexer/"
    click ModelArch "https://github.com/kobinabrandon/Hourly-Divvy-Trip-Predictor/blob/main/src/training_pipeline/models.py"
    click HyperTune "https://github.com/kobinabrandon/Hourly-Divvy-Trip-Predictor/blob/main/src/training_pipeline/hyperparameter_tuning.py"
    click Training "https://github.com/kobinabrandon/Hourly-Divvy-Trip-Predictor/blob/main/src/training_pipeline/training.py"
    click FeatureAPI "https://github.com/kobinabrandon/Hourly-Divvy-Trip-Predictor/blob/main/src/inference_pipeline/backend/feature_store_api.py"
    click ModelAPI "https://github.com/kobinabrandon/Hourly-Divvy-Trip-Predictor/blob/main/src/inference_pipeline/backend/model_registry_api.py"
    click InferenceEng "https://github.com/kobinabrandon/Hourly-Divvy-Trip-Predictor/blob/main/src/inference_pipeline/backend/inference.py"
    click Backfill "https://github.com/kobinabrandon/Hourly-Divvy-Trip-Predictor/blob/main/src/inference_pipeline/backend/backfill_feature_store.py"
    click MainDash "https://github.com/kobinabrandon/Hourly-Divvy-Trip-Predictor/blob/main/src/inference_pipeline/frontend/main.py"
    click PredView "https://github.com/kobinabrandon/Hourly-Divvy-Trip-Predictor/blob/main/src/inference_pipeline/frontend/predictions.py"
    click MonitorDash "https://github.com/kobinabrandon/Hourly-Divvy-Trip-Predictor/blob/main/src/inference_pipeline/frontend/monitoring.py"
    click DataViz "https://github.com/kobinabrandon/Hourly-Divvy-Trip-Predictor/blob/main/src/inference_pipeline/frontend/plots.py"
    click DataView "https://github.com/kobinabrandon/Hourly-Divvy-Trip-Predictor/blob/main/src/inference_pipeline/frontend/data.py"
    click Monitor "https://github.com/kobinabrandon/Hourly-Divvy-Trip-Predictor/blob/main/src/monitoring.py"
    click Config "https://github.com/kobinabrandon/Hourly-Divvy-Trip-Predictor/blob/main/src/setup/config.py"
    click Paths "https://github.com/kobinabrandon/Hourly-Divvy-Trip-Predictor/blob/main/src/setup/paths.py"
```
