How thakur-sahab-01/suryanet-ai-v2 works
1. Repo shape
- This is a three-runtime system: a Python backend for data ingestion/inference, a React/TypeScript dashboard for visualization and operator interaction, and a C++/CUDA/TensorRT edge path for GPU inference acceleration.
- The root models/ directory holds deployed artifacts; python/scripts/ and python/models/ are the training and model-definition side; dashboard/ is the client side.
2. Primary runtime: Python API
- The main service is python/api/server.py, a FastAPI application exposed over HTTP and WebSocket.
- It acts as the orchestration layer: loads the inference model, fetches live GOES data, assembles predictions, serves explainability outputs, and returns satellite-risk and copilot responses to the UI.
3. Data ingestion boundary
- Live input comes from NOAA GOES public JSON feeds via python/data/goes_fetcher.py; this is the only clearly external live data dependency.
- Historical/offline solar data is handled separately through python/data/solexs_loader.py, hel1os_loader.py, and preprocessor.py, which convert FITS/ZIP sources into processed tabular/time-series formats used for training and alignment.
4. Offline preprocessing and dataset generation
- The offline pipeline is distinct from runtime inference: scripts under python/scripts/ process batch data, train models, and export artifacts.
- process_solexs_batch.py, process_hel1os_batch.py, and train_deep.py/train_legacy.py suggest a separation between data engineering, model training, and deployment artifact generation.
5. Model layer split
- The architecture has two independent model paths under python/models/:
- Deep learning (deep/model.py, dataset.py, loss.py) for the ONNX-exported model.
- Legacy/traditional ML (legacy/) for anomaly detection, flare classification, and forecasting.
- ensemble_v2.py is the fusion boundary that combines both paths into the final risk output.
6. Deployed model artifacts
- The models/ directory contains the runtime artifacts consumed by the backend: suryanet.onnx plus external weights, and legacy model files such as flare_classifier.json.
- This indicates the production inference path is designed to be artifact-driven, not trained in place.
7. Inference outputs and scoring
- The API returns multi-horizon flare probabilities, class-specific probabilities, and a final ensemble risk score with a risk level.
- The ensemble boundary is explicit: deep learning contributes the main score, while the legacy ML stack provides an auxiliary score that is fused into the final decision.
8. Explainability subsystem
- python/explainability/ contains both SHAP and Integrated Gradients implementations.
- This is a separate explanation plane attached to the model API, not embedded in the dashboard logic; the frontend just renders the returned explanations.
9. Copilot subsystem
- python/copilot/agent.py and knowledge.py implement a bilingual Hindi/English assistant.
- It is a distinct query/response service that can optionally use a Hugging Face LLM token, with a local knowledge base fallback, and is exposed through the /copilot API.
10. Frontend presentation layer
- dashboard/ is a Vite + React + TypeScript application.
- src/hooks/usePredictions.ts is the main data-access layer to the backend; src/components/ is organized by visualization responsibility: GOES flux, probability charts, class breakdowns, risk gauge, satellite matrix, model card, alert banner, and copilot chat.
11. Dashboard-to-backend data flow
- The dashboard is a consumer only: it pulls prediction/state data from the FastAPI endpoints and listens to the WebSocket stream for live updates.
- The UI is structured around four operational views: live monitoring, forecast charts, satellite risk, and AI copilot.
12. Edge inference boundary
- The cpp/include/suryanet/ headers define a separate deployment path for TensorRT/CUDA inference, preprocessing, and server plumbing.
- Because only headers are present in the tree, this looks like an accelerator/runtime boundary rather than the main application path; it likely mirrors the Python inference behavior for GPU/edge deployment.
13. Deployment and packaging
- docker/ contains separate images for the API, dashboard, and CUDA runtime, with docker-compose.yml wiring them together.
- run.bat provides a Windows bootstrap path, so the repo supports both local development and containerized deployment.
14. Storage and state
- There is no external database dependency in the architecture.
- Persistent assets are file-based model/data artifacts, while live GOES feeds and recent alerts are kept in server memory/cache, which keeps the runtime stateless aside from local files.
15. What to show in an architecture diagram
- The key diagram boundaries are: external NOAA feeds → Python data pipeline → model ensemble → FastAPI/WebSocket → React dashboard, with side branches for explainability, copilot, and C++/TensorRT acceleration.
- The offline training/export path should be shown as separate from the live inference path, feeding the artifact store in models/.
Main entry points: server.py API server · suryanet.onnx Model files · ensemble_v2.py Ensemble · usePredictions.ts Predictions hook
Components 20
External Inputs
Outside data and client consumers
- GOES feed · Public NOAA GOES live feed consumed by the backend
Python Backend
Live API, data ingestion, inference, explainability, and copilot services
- API server · HTTP and WebSocket orchestration layer for live inference and UI responses
python/api/server.py - GOES fetcher · Fetches live NOAA GOES data for runtime inference
python/data/goes_fetcher.py - Preprocess · Aligns and prepares incoming data for inference
python/data/preprocessor.py - Ensemble · Combines deep and legacy model outputs into final risk scoring
python/models/ensemble_v2.py - Deep model · Deep learning inference path backing the main score
python/models/deep/model.py - Legacy ML · Auxiliary anomaly, flare, and forecast models
python/models/legacy/ensemble.py - Explainability · Produces model explanations returned by the API
python/explainability/shap_explainer.py - Copilot · Hindi/English assistant with knowledge base and optional HF LLM
python/copilot/agent.py
Offline Training
Batch processing, model training, and export pipeline
- Training scripts · Processes datasets, trains models, and exports runtime artifacts
python/scripts/train_deep.py - Data loaders · Converts SOLExS and HEL1OS sources into processed training data
python/data/solexs_loader.py - Export ONNX · Exports trained deep models to deployable ONNX artifacts
python/scripts/export_to_onnx.py
Dashboard
React/TypeScript operator UI
- Dashboard · Operator interface for monitoring, charts, and copilot
dashboard/src/App.tsx - Predictions hook · Main dashboard data-access layer for backend predictions and stream updates
dashboard/src/hooks/usePredictions.ts - Charts · Flux, probability, class, risk, and satellite visualizations
dashboard/src/components/GOESFluxChart.tsx - Copilot panel · Frontend surface for AI copilot responses
dashboard/src/components/AICopilot.tsx
Edge Runtime
C++/CUDA/TensorRT acceleration boundary
- CUDA runtime · TensorRT/CUDA inference, preprocessing, and server plumbing headers
cpp/include/suryanet/tensorrt_engine.hpp - Edge server · Edge inference server boundary for accelerated deployment
cpp/include/suryanet/inference_server.hpp
Model Artifacts
Deployed files consumed by inference
- Model files · Runtime ONNX and legacy model files loaded by the backend
models/suryanet.onnx - Flare model · Representative legacy artifact used in production scoring
models/flare_classifier.json
Connections 21
- GOES feed → GOES fetcher: live feed · Runtime NOAA input
- GOES fetcher → Preprocess: raw data · Fetch output is normalized for inference
- Preprocess → Ensemble: features · Prepared runtime features enter scoring
- Ensemble → Deep model: deep score · Primary deep-learning inference path
- Ensemble → Legacy ML: aux score · Legacy model outputs contribute to fusion
- Deep model → Model files: loads · Deep runtime artifacts are consumed from files
- Legacy ML → Model files: loads · Legacy runtime artifacts are consumed from files
- Model files → API server: model load · Backend loads deployed artifacts at runtime
- API server → Dashboard: HTTP/WebSocket · Serves predictions and live updates to the UI
- Dashboard → Predictions hook: uses · UI data flow is centralized in the predictions hook
- Predictions hook → Charts: feeds · Prediction state drives monitoring and forecast views
- Predictions hook → Copilot panel: feeds · Copilot panel consumes API responses
- API server → Explainability: explanations · Backend attaches model explanations to responses
- API server → Copilot: /copilot · Dedicated assistant endpoint and response flow
- Training scripts → Data loaders: batch data · Training pipeline starts from offline source processing
- Training scripts → Export ONNX: exports · Training produces deployable ONNX artifacts
- Export ONNX → Model files: writes · Exported model files are placed in the artifact store
- Data loaders → Training scripts: datasets · Processed historical data feeds model training
- CUDA runtime → Edge server: accelerated path · Headers define the GPU/edge inference boundary
- Edge server → API server: optional accel · Edge deployment mirrors backend inference behavior
- Flare model → Legacy ML: class model · Representative deployed legacy model artifact