# imaddde867/navicast architecture

> Architecture diagram of the GitHub repository imaddde867/navicast, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/imaddde867/navicast
- Repository: https://github.com/imaddde867/navicast

## Overview

##### Main Components
- Data Ingestion & Streaming:
- AIS Data Feed: External Digitraffic AIS feed, which streams vessel positional data.
- MQTT Client (mqtt_client.py): Connects and subscribes to the AIS feed.
- ais_streaming Service (referenced in the README): Likely similar to the MQTT client, ensuring continuous data flow.
- Data Storage:
- PostgreSQL Database: Stores raw AIS data (as defined in the provided schema) and prediction results.
- Backup/Alternate Storage: There is also an anaconda_projects/db folder (with an SQLite file) that may assist in development/testing.
- Machine Learning (Prediction) Module:
- Prediction Service (prediction_service.py): Runs the XGBoost model (stored in vessel_prediction_model.pkl) to generate vessel position predictions.
- ML Model Documentation: Files like ML_Model.ipynb and ML_MODEL.pdf detail model creation and feature engineering.
- Backend API:
- API Server (api_server.py): Built using FastAPI; serves HTTP endpoints (e.g., /vessels) for the frontend to fetch real-time tracking data.
- Frontend:
- Static Files (static/ folder): Contains HTML, CSS, JavaScript (with Leaflet.js) for the interactive visualization of vessels and predictions.
- Assets & Branding: Includes logos and images (e.g., NAVICAST-logo) for a consistent UI presentation.

##### Relationships and Interactions
- The AIS Data Feed streams external data, which is received by the MQTT Client.
- The MQTT Client processes this data and forwards or stores it into the PostgreSQL Database.
- The Prediction Service processes historical and real-time data from the database and uses the tuned XGBoost model to predict future positions. The output is then written back to the database.
- The API Server polls the database to serve the current and predicted vessel positions via REST endpoints.
- The Frontend queries the FastAPI service to render an interactive map using Leaflet.js, displaying both current and predicted positions.
- Optionally, the system includes batch or real-time data processing pipelines (as hinted by the documentation and notebooks).

##### Architectural Patterns & Design Principles
- Separation of Concerns:
- Clearly distinct layers for data ingestion, storage, processing, and presentation.
- Real-Time Data Processing:
- Use of MQTT for low-latency streaming and FastAPI for serving fresh data.
- Machine Learning Integration:
- Dedicated ML modules (with model tuning via Optuna) that operate on organized data from the database.
- Modularity and Scalability:
- Components can be developed, tested, and scaled independently (e.g., swapping out data sources or ML models, scaling the API server).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Data Source
    EXT["Digitraffic AIS Feed"]:::external

    %% Data Ingestion Layer
    subgraph "Data Ingestion Layer"
        MQTT["MQTT Client (mqtt_client.py)"]:::ingestion
        AIS["ais_streaming Service"]:::ingestion
    end

    %% Data Storage Layer
    subgraph "Data Storage"
        DB["PostgreSQL Database"]:::database
        DEV["Development DB (SQLite)"]:::database
    end

    %% Machine Learning Module
    subgraph "Machine Learning Module"
        PRED["Prediction Service (prediction_service.py)"]:::ml
        XGB["XGBoost Model"]:::ml
    end

    %% Backend API Layer
    API["API Server (FastAPI)"]:::backend

    %% Frontend Layer
    subgraph "Frontend"
        HTML["Frontend (static/index.html)"]:::frontend
        LOGO["NAVICAST-logo Assets"]:::frontend
    end

    %% ML Documentation
    subgraph "ML Documentation"
        NOTEBOOK["ML_Model.ipynb"]:::documentation
        MLPDF["ML_Model.pdf"]:::documentation
    end

    %% Data Flow Connections
    EXT -->|"data_in"| MQTT
    MQTT -->|"forwards"| DB
    MQTT -->|"cooperates"| AIS
    DEV ---|"dev_backup"| DB

    DB -->|"historical_data"| PRED
    PRED -->|"writes_preds"| DB
    PRED -->|"uses"| XGB

    DB -->|"fetch_data"| API
    PRED -->|"provides_preds"| API

    API -->|"api_call"| HTML
    API -->|"api_call"| LOGO

    %% Styles
    classDef external fill:#ffcccc,stroke:#990000,stroke-width:2px;
    classDef ingestion fill:#ccffcc,stroke:#339900,stroke-width:2px;
    classDef database fill:#ffe680,stroke:#cc9900,stroke-width:2px;
    classDef ml fill:#ccccff,stroke:#333399,stroke-width:2px;
    classDef backend fill:#d9ccff,stroke:#6600cc,stroke-width:2px;
    classDef frontend fill:#cceeff,stroke:#006699,stroke-width:2px;
    classDef documentation fill:#efccff,stroke:#a020f0,stroke-width:2px;

    %% Click Events
    click MQTT "https://github.com/imaddde867/navicast/blob/main/mqtt_client.py"
    click DB "https://github.com/imaddde867/navicast/blob/main/anaconda_projects/db/project_filebrowser.db"
    click PRED "https://github.com/imaddde867/navicast/blob/main/prediction_service.py"
    click XGB "https://github.com/imaddde867/navicast/blob/main/vessel_prediction_model.pkl"
    click NOTEBOOK "https://github.com/imaddde867/navicast/blob/main/ML_Model.ipynb"
    click MLPDF "https://github.com/imaddde867/navicast/blob/main/ML_Model.pdf"
    click API "https://github.com/imaddde867/navicast/blob/main/api_server.py"
    click HTML "https://github.com/imaddde867/navicast/blob/main/static/index.html"
    click LOGO "https://github.com/imaddde867/navicast/tree/main/static/NAVICAST-logo"
```
