# realronaldrump/everystreet-new architecture

> Architecture diagram of the GitHub repository realronaldrump/everystreet-new, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/realronaldrump/everystreet-new
- Repository: https://github.com/realronaldrump/everystreet-new

## Overview

##### Main Components
- Frontend:

- Backend:
- The backend is built using Python (with FastAPI as the web framework). Place “app.py” as the central entry point.
- Supporting modules (e.g., “bouncie_trip_fetcher.py”, “db.py”, “map_matching.py”, “trip_processing.py”, “tasks.py”, “preprocess_streets.py”, “street_coverage_calculation.py”, etc.) to represent functionalities like data fetching, processing, and map matching.

- Database:
- MongoDB as the primary data store. Note that stored data includes trip details, map matching results, street coverage info, custom places, and task management details.
- Mention that asynchronous access is handled via Motor.

- External APIs & Services:
- Integration with external services:
· Bouncie API — provides vehicle trip data.
· Mapbox Map Matching API — used to snap raw GPS data to road networks.
· Nominatim and Overpass API — used for geocoding and retrieving OpenStreetMap (OSM) data.

- Containerization & Deployment:
- Use the Dockerfile to represent containerization for deployment, indicating how the entire environment can be encapsulated.

##### Relationships and Data Flow
- Between the Frontend and Backend (e.g., via REST APIs or WebSockets for live trip tracking).
- Between the Backend and MongoDB for data storage and retrieval.
- From the Backend to External APIs, indicating data fetching and map matching operations.
- From background tasks (triggered via APScheduler) to the MongoDB and to external APIs for periodic processing.
- How user interactions on the frontend (e.g., trip editing, data upload) initiate requests processed by the backend application.

##### Architectural Patterns and Design Principles
- Separation of Concerns: The project cleanly separates frontend templates/static files from backend logic.
- Asynchronous Processing: Emphasize usage of asynchronous calls (aiohttp, Motor) and background task scheduling (APScheduler) to handle heavy geospatial processing and data fetching without blocking.
- Modularization: Note that various functionalities (map matching, trip fetching, street coverage, preprocessing, etc.) are encapsulated in separate Python modules.
- Containerization: The Dockerfile indicates that container deployment is supported, simplifying environment replication.

##### Instructing the Principal Software Engineer
- The user interface (frontend) and the backend API.
- MongoDB and the external APIs as separate entities that the backend communicates with.
- Drill down further if needed by clustering related backend functionalities (e.g., grouping modules for data manipulation, geospatial processing, and API integrations).
- Component-specific notes for technologies:
- Mention FastAPI for the backend, APScheduler for background tasks, and the set of JavaScript libraries used in the frontend.

## Diagram (Mermaid)

```mermaid
graph TD
    %% Frontend Layer
    subgraph "Frontend Layer"
        A["HTML Templates"]:::frontend
        B["Static Assets"]:::frontend
    end

    %% Backend Layer
    subgraph "Backend Layer"
        C["FastAPI Backend (app.py)"]:::backend
        subgraph "Data Processing Modules"
            D["Trip Processing (trip_processing.py)"]:::backend
            E["Map Matching (map_matching.py)"]:::backend
            F["Preprocess Streets (preprocess_streets.py)"]:::backend
            G["Street Coverage Calculation (street_coverage_calculation.py)"]:::backend
            H["Update Geo Points (update_geo_points.py)"]:::backend
        end
        I["Bouncie Trip Fetcher (bouncie_trip_fetcher.py)"]:::backend
    end

    %% Background Task Manager
    J["Background Tasks Manager (tasks.py)"]:::tasks

    %% Database
    K["MongoDB Database"]:::database

    %% External APIs & Services
    subgraph "External APIs & Services"
        L["Bouncie API"]:::external
        M["Mapbox Map Matching API"]:::external
        N["Nominatim/Overpass API"]:::external
    end

    %% Containerization & Deployment
    O["Docker Container (Dockerfile)"]:::docker

    %% Connections between Frontend and Backend
    A -->|"sends_request"| C
    B -->|"sends_request"| C

    %% Backend internal processing and external API calls
    C -->|"processes"| D
    C -->|"processes"| E
    C -->|"processes"| F
    C -->|"processes"| G
    C -->|"processes"| H
    C -->|"invokes"| I
    C -->|"stores_data"| K
    C -->|"calls"| L
    C -->|"calls"| M
    C -->|"calls"| N

    %% Bouncie API Integration from Backend module
    I -->|"fetches"| L

    %% Background tasks scheduling and API calls
    J -->|"schedules"| D
    J -->|"schedules"| E
    J -->|"schedules"| F
    J -->|"schedules"| G
    J -->|"updates"| K
    J -->|"calls"| L
    J -->|"calls"| M
    J -->|"calls"| N

    %% Deployment container relationships
    O -->|"deploys"| C
    O -->|"deploys"| J
    O -->|"hosts"| K

    %% Class Styles
    class A,B frontend
    class C,D,E,F,G,H,I backend
    class J tasks
    class K database
    class L,M,N external
    class O docker

    %% Class Definitions for Colors
    classDef frontend fill:#cce5ff,stroke:#004085,stroke-width:2px;
    classDef backend fill:#d4edda,stroke:#155724,stroke-width:2px;
    classDef tasks fill:#fff3cd,stroke:#856404,stroke-width:2px;
    classDef database fill:#f8d7da,stroke:#721c24,stroke-width:2px;
    classDef external fill:#e2e3e5,stroke:#6c757d,stroke-width:2px;
    classDef docker fill:#d1ecf1,stroke:#0c5460,stroke-width:2px;

click A "https://github.com/realronaldrump/everystreet-new/tree/main/templates/"
click B "https://github.com/realronaldrump/everystreet-new/tree/main/static/"
click C "https://github.com/realronaldrump/everystreet-new/blob/main/app.py"
click D "https://github.com/realronaldrump/everystreet-new/blob/main/trip_processing.py"
click E "https://github.com/realronaldrump/everystreet-new/blob/main/map_matching.py"
click F "https://github.com/realronaldrump/everystreet-new/blob/main/preprocess_streets.py"
click G "https://github.com/realronaldrump/everystreet-new/blob/main/street_coverage_calculation.py"
click H "https://github.com/realronaldrump/everystreet-new/blob/main/update_geo_points.py"
click I "https://github.com/realronaldrump/everystreet-new/blob/main/bouncie_trip_fetcher.py"
click J "https://github.com/realronaldrump/everystreet-new/blob/main/tasks.py"
click K "https://github.com/realronaldrump/everystreet-new/blob/main/db.py"
click O "https://github.com/realronaldrump/everystreet-new/tree/main/Dockerfile"
```
