How imaddde867/navicast works
Main Components
• Data Ingestion & Streaming:
• AIS Data Feed: External Digitraffic AIS feed, which streams vessel positional data.
• MQTT Client (mqtt_client.py): Connects and subscribes to the AIS feed.
• ais_streaming Service (referenced in the README): Likely similar to the MQTT client, ensuring continuous data flow.
• Data Storage:
• PostgreSQL Database: Stores raw AIS data (as defined in the provided schema) and prediction results.
• Backup/Alternate Storage: There is also an anaconda_projects/db folder (with an SQLite file) that may assist in development/testing.
• Machine Learning (Prediction) Module:
• Prediction Service (prediction_service.py): Runs the XGBoost model (stored in vessel_prediction_model.pkl) to generate vessel position predictions.
• ML Model Documentation: Files like ML_Model.ipynb and ML_MODEL.pdf detail model creation and feature engineering.
• Backend API:
• API Server (api_server.py): Built using FastAPI; serves HTTP endpoints (e.g., /vessels) for the frontend to fetch real-time tracking data.
• Frontend:
• Static Files (static/ folder): Contains HTML, CSS, JavaScript (with Leaflet.js) for the interactive visualization of vessels and predictions.
• Assets & Branding: Includes logos and images (e.g., NAVICAST-logo) for a consistent UI presentation.
Relationships and Interactions
• The AIS Data Feed streams external data, which is received by the MQTT Client.
• The MQTT Client processes this data and forwards or stores it into the PostgreSQL Database.
• The Prediction Service processes historical and real-time data from the database and uses the tuned XGBoost model to predict future positions. The output is then written back to the database.
• The API Server polls the database to serve the current and predicted vessel positions via REST endpoints.
• The Frontend queries the FastAPI service to render an interactive map using Leaflet.js, displaying both current and predicted positions.
• Optionally, the system includes batch or real-time data processing pipelines (as hinted by the documentation and notebooks).
Architectural Patterns & Design Principles
• Separation of Concerns:
• Clearly distinct layers for data ingestion, storage, processing, and presentation.
• Real-Time Data Processing:
• Use of MQTT for low-latency streaming and FastAPI for serving fresh data.
• Machine Learning Integration:
• Dedicated ML modules (with model tuning via Optuna) that operate on organized data from the database.
• Modularity and Scalability:
• Components can be developed, tested, and scaled independently (e.g., swapping out data sources or ML models, scaling the API server).