# busig-the-sql/busig architecture

> Architecture diagram of the GitHub repository busig-the-sql/busig, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/busig-the-sql/busig
- Repository: https://github.com/busig-the-sql/busig

## Overview

##### Project Type & Purpose
- This is a containerized, microservice-based full-stack application (“Busig”) for fetching, processing, and predicting city bus arrival times.
- It consists of:
- A React/Expo front-end (mobile & web)
- A Flask-based API Gateway
- Three backend services (training_data_collection, inference, bus_model)
- Persistent data stores (PostgreSQL, MongoDB)
- Docker Compose orchestration

##### File-Structure Analysis
- Top-level directories = microservices (training_data_collection, inference, bus_model, gateway), front-end, ORM (Django), benchmarking, inference/training code, docker-compose.yml.
- Each service has its own Dockerfile, requirements.txt, a Flask `app.py`, and supporting modules (e.g. GTFS fetchers, preprocessing pipelines, ML models).
- `orm/` holds a Django project for data models and migrations (PostgreSQL schema).
- `front-end/` holds an Expo React app with Redux slices, config, assets, screens, and hooks.
- `docker-compose.yml` brings up four Flask containers + a Postgres container + a Mongo container.

##### Key README Insights
- Four Flask services listening on ports:
– training_data_collection:5000
– inference:5001
– bus_model:5002
– gateway:5004
- .env supplies GTFS API keys
- `training_data_collection` builds Mongo collections from GTFS Realtime & Static
- `bus_model` builds/static Postgres GTFS database, and exposes route & schedule APIs
- `inference` loads pre-trained PyTorch models (.pth) to predict arrival times
- `gateway` routes incoming client calls to the above services

##### Main Components & Interactions
A. Front-end (React/Expo)
- Consumes REST endpoints exclusively on gateway: HTTP calls to http://localhost:5004/<endpoint>
- Dispatches user events (search stops/routes, view map) via Redux slices

B. API Gateway (Flask on 5004)
- Aggregator/orchestrator for all backend services
- Routes to:
– bus_model for static schedule & live status
– inference for predictive arrival times
– training_data_collection for raw data inspection (if exposed)

C. Bus Model Service (Flask on 5002)
- Builds & queries Postgres GTFS static DB
- Provides endpoints for: route list, stop arrivals (real-time + static)
- Runs update_GTFS_Static.sh to rebuild DB

D. Training Data Collection Service (Flask on 5000)
- Pulls GTFS Realtime & Static feeds
- Stores JSON in MongoDB
- Supplies raw data for model retraining & debugging

E. Inference Service (Flask on 5001)
- Preprocesses incoming JSON on-the-fly
- Loads PyTorch models (bus_time_prediction_model*.pth)
- Returns predicted arrival times

F. Databases & External Feeds
- PostgreSQL container → GTFS Static tables (built by bus_model & ORM migrations)
- MongoDB container → Collections of GTFS Realtime JSON (populated by training_data_collection)
- External GTFS APIs → polled by training_data_collection and bus_model scripts

##### Architectural Patterns & Principles
- Microservices: each concern in its own container + Flask app
- ETL pipeline: training_data_collection → preprocess → persist → model training
- ML inference service separate from data pipeline
- API Gateway: single ingress point for front-end, decouples UI from services
- Containerization for environment parity, orchestrated via docker-compose

##### Essential Elements Checklist
- Ports and protocols on connections (e.g., 5002, HTTPS/REST)
- Notes on triggers (e.g., “cron-like update_GTFS_Static.sh”)

## Diagram (Mermaid)

```mermaid
graph TB
    %% UI Layer
    UI["React/Expo App"]:::ui

    %% API Layer
    API["API Gateway\nFlask (port 5004)"]:::gateway

    %% Compute Layer
    subgraph "Microservices (Compute Layer)"
        direction TB
        BM["Bus Model Service\nFlask (port 5002)"]:::micro
        TDC["Training Data Collection Service\nFlask (port 5000)"]:::micro
        INF["Inference Service\nFlask (port 5001)"]:::micro
        ORM["ORM (Django for Postgres)\nMigrations & Models"]:::micro
    end

    %% Datastores
    PG["PostgreSQL"]:::db
    MG["MongoDB"]:::db

    %% External Systems
    EXT["External GTFS APIs"]:::external

    %% Orchestration
    DC["docker-compose.yml"]:::external

    %% Data Flows
    UI -->|"HTTP REST\n(port 5004)"| API
    API -->|"REST API\n(port 5002)"| BM
    API -->|"REST API\n(port 5000)"| TDC
    API -->|"REST API\n(port 5001)"| INF

    BM -->|"SQL Queries"| PG
    TDC -->|"MongoDB Driver"| MG

    BM -->|"HTTP Fetch"| EXT
    TDC -->|"HTTP Fetch"| EXT

    ORM -->|"Migrations"| PG

    %% Orchestration Connections
    DC -.-> API
    DC -.-> BM
    DC -.-> TDC
    DC -.-> INF
    DC -.-> PG
    DC -.-> MG

    %% Click Events
    click UI "https://github.com/busig-the-sql/busig/tree/main/front-end/"
    click API "https://github.com/busig-the-sql/busig/blob/main/gateway/app.py"
    click BM "https://github.com/busig-the-sql/busig/blob/main/bus_model/app.py"
    click TDC "https://github.com/busig-the-sql/busig/blob/main/training_data_collection/app.py"
    click INF "https://github.com/busig-the-sql/busig/blob/main/inference/app.py"
    click ORM "https://github.com/busig-the-sql/busig/blob/main/orm/manage.py"
    click DC "https://github.com/busig-the-sql/busig/blob/main/docker-compose.yml"

    %% Styles
    classDef ui fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef gateway fill:#FFA500,stroke:#333,stroke-width:1px
    classDef micro fill:#90EE90,stroke:#333,stroke-width:1px
    classDef db fill:#DDA0DD,stroke:#333,stroke-width:1px
    classDef external fill:#D3D3D3,stroke:#333,stroke-width:1px
```
