# coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model architecture

> Architecture diagram of the GitHub repository coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model
- Repository: https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model

## Overview

1. This repo is organized around a single end-to-end solar flare pipeline: ingest Aditya-L1 FITS telemetry, detect flare candidates with physics-based rules, label them against external flare catalogs, train an LSTM forecasting model, then serve both detection and prediction through a live dashboard.

2. The offline data-processing path lives in `src/` and is staged as four sequential batch steps:
   - `data_ingestion.py` builds a unified `master_data.csv` from raw HEL1OS and SoLEXS satellite files.
   - `statistical_trigger.py` filters the stream with a Poisson significance test to produce `triggered_data.csv`.
   - `morphological_filter.py` applies flare-shape validation to generate `flare_catalog.csv`.
   - `benchmarking.py` matches detected events against GOES/NASA DONKI-style references to produce labeled training data.

3. `src/pipeline.py` is the orchestration boundary for that offline pipeline. Its role is to run the four preprocessing stages in order and materialize the intermediate datasets into `outputs/`. This is the main data-prep boundary between raw instrument data and everything downstream.

4. The detection logic is split into two conceptual engines. `nowcast.py` is the real-time flare detector that operates on current telemetry and significance thresholds, while `forecast.py` loads the trained model and emits future flare probabilities. Together they form the online inference layer.

5. The physics-first detection subsystem is important architecturally because it gates the ML layer. The system does not rely on the neural network alone; it first uses statistical triggering and morphological constraints to reject noise, glitches, and non-flare spikes before labeling an event as a candidate flare.

6. The ML training path is separated from runtime inference. `src/forecast.ipynb` is the training notebook, and the trained artifacts are stored under `outputs/` as metadata and model/scaler files. That means training is an offline, GPU-friendly workflow, while the serving side only needs inference-capable dependencies.

7. The serving layer is `src/server.py`, which exposes the application state and data through REST endpoints. It appears to be stateful, maintaining replay position, play/pause, speed, and seek controls so the dashboard can behave like a time-based simulation rather than a passive chart viewer.

8. The API is the bridge between backend computation and visualization. Endpoints provide:
   - current flux/timestamp/progress,
   - nowcast alert state and flare class,
   - forecast probability and upcoming events,
   - chart-ready historical slices,
   - the detected flare catalog,
   - and replay controls such as play, pause, reset, speed, and seek.

9. The `dashboard/` folder is a static frontend built with HTML, CSS, and JavaScript. It renders live plots, probability indicators, alert cards, and event logs, likely polling the FastAPI backend rather than pushing via websockets. This keeps the UI decoupled from the inference logic.

10. `outputs/model_meta.json` is a key runtime configuration artifact. It likely defines the model’s feature schema, thresholds, and evaluation metadata so the server can interpret the forecast model consistently without hardcoding training details in the API layer.

11. The main external data boundary is the PRADAN portal for Level-1 Aditya-L1 data, with NASA DONKI used as a supplementary labeling source during benchmarking. Those sources are only needed in the preprocessing/labeling stages, not for dashboard operation once the derived CSVs and model artifacts exist.

12. Tooling and runtime are split by purpose: Python for all backend/data work, PyTorch for model training and inference, FastAPI/Uvicorn for serving, and standard browser technologies for the UI. The repo also distinguishes between a general Python environment and a separate GPU training environment, which reflects the separation between offline model development and online deployment.

Main entry points: [`outputs/model_meta.json`](https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/outputs/model_meta.json) (Outputs), [`src/server.py`](https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/src/server.py) (API server), [`src/pipeline.py`](https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/src/pipeline.py) (Offline pipeline), [`src/benchmarking.py`](https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/src/benchmarking.py) (Benchmark events)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_offline["Offline pipeline"]
  node_ingestion["Ingest data<br/>batch step<br/>[data_ingestion.py]"]
  node_trigger["Trigger filter<br/>physics filter"]
  node_morph["Shape filter<br/>validation step"]
  node_bench["Benchmark events<br/>labeling step<br/>[benchmarking.py]"]
  node_pipeline["Offline pipeline<br/>orchestrator<br/>[pipeline.py]"]
  node_outputs[("Outputs<br/>artifacts<br/>[model_meta.json]")]
  node_train_nb["Train model<br/>notebook training<br/>[forecast.ipynb]"]
  node_meta["Model meta<br/>runtime config<br/>[model_meta.json]"]
end

subgraph group_online["Online serving"]
  node_nowcast["Nowcast<br/>real-time detector<br/>[nowcast.py]"]
  node_forecast["Forecast<br/>inference model<br/>[forecast.py]"]
  node_server["API server<br/>stateful API<br/>[server.py]"]
  node_dashboard["Dashboard<br/>static frontend<br/>[index.html]"]
  node_appjs["UI logic<br/>frontend script<br/>[app.js]"]
  node_style["UI styles<br/>stylesheet<br/>[style.css]"]
end

subgraph group_external["External sources"]
  node_pradan[("PRADAN<br/>data portal")]
  node_donki[("NASA DONKI<br/>flare catalog")]
end

node_pradan -->|"raw telemetry"| node_ingestion
node_ingestion -->|"master data"| node_trigger
node_trigger -->|"candidates"| node_morph
node_morph -->|"flare events"| node_bench
node_donki -->|"reference labels"| node_bench
node_pipeline -->|"orchestrates"| node_ingestion
node_pipeline -->|"orchestrates"| node_trigger
node_pipeline -->|"orchestrates"| node_morph
node_pipeline -->|"orchestrates"| node_bench
node_bench -->|"labeled data"| node_outputs
node_train_nb -->|"trained artifacts"| node_outputs
node_outputs -->|"model config"| node_meta
node_outputs -->|"model files"| node_forecast
node_outputs -->|"catalog/data"| node_server
node_nowcast -->|"alert state"| node_server
node_forecast -->|"probabilities"| node_server
node_server -->|"REST data"| node_dashboard
node_dashboard -->|"bootstraps"| node_appjs
node_dashboard -->|"styles"| node_style
node_appjs -.->|"polls"| node_server

click node_ingestion "https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/src/data_ingestion.py"
click node_trigger "https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/src/statistical_trigger.py"
click node_morph "https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/src/morphological_filter.py"
click node_bench "https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/src/benchmarking.py"
click node_pipeline "https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/src/pipeline.py"
click node_outputs "https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/outputs/model_meta.json"
click node_train_nb "https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/src/forecast.ipynb"
click node_meta "https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/outputs/model_meta.json"
click node_nowcast "https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/src/nowcast.py"
click node_forecast "https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/src/forecast.py"
click node_server "https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/src/server.py"
click node_dashboard "https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/dashboard/index.html"
click node_appjs "https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/dashboard/app.js"
click node_style "https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/dashboard/style.css"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ingestion,node_trigger,node_morph,node_bench,node_pipeline,node_outputs,node_train_nb,node_meta toneBlue
class node_nowcast,node_forecast,node_server,node_dashboard,node_appjs,node_style toneAmber
class node_pradan,node_donki toneMint
```

## Components

### Offline pipeline

Batch preprocessing, labeling, and model preparation for solar flare data.

- [Ingest data](https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/src/data_ingestion.py): batch step · Builds unified master telemetry data from raw instrument files. · `src/data_ingestion.py`
- [Trigger filter](https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/src/statistical_trigger.py): physics filter · Applies statistical significance testing to isolate candidate events. · `src/statistical_trigger.py`
- [Shape filter](https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/src/morphological_filter.py): validation step · Validates flare morphology and rejects non-flare spikes. · `src/morphological_filter.py`
- [Benchmark events](https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/src/benchmarking.py): labeling step · Matches detected events against external references to create labeled data. · `src/benchmarking.py`
- [Offline pipeline](https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/src/pipeline.py): orchestrator · Runs preprocessing stages in order and materializes intermediate datasets. · `src/pipeline.py`
- [Outputs](https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/outputs/model_meta.json): artifacts · Derived CSVs, model metadata, and trained assets used downstream. · `outputs/model_meta.json`
- [Train model](https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/src/forecast.ipynb): notebook training · Offline LSTM training workflow for flare forecasting. · `src/forecast.ipynb`
- [Model meta](https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/outputs/model_meta.json): runtime config · Serialized schema, thresholds, and evaluation metadata for inference. · `outputs/model_meta.json`

### Online serving

Real-time detection, forecasting, API state, and dashboard presentation.

- [Nowcast](https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/src/nowcast.py): real-time detector · Detects active flare candidates from current telemetry using significance rules. · `src/nowcast.py`
- [Forecast](https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/src/forecast.py): inference model · Loads trained artifacts and emits future flare probabilities. · `src/forecast.py`
- [API server](https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/src/server.py): stateful API · Maintains replay state and exposes telemetry, alerts, catalog, and controls. · `src/server.py`
- [Dashboard](https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/dashboard/index.html): static frontend · Browser UI for live plots, alerts, probabilities, and event logs. · `dashboard/index.html`
- [UI logic](https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/dashboard/app.js): frontend script · Fetches backend state and renders charts and controls. · `dashboard/app.js`
- [UI styles](https://github.com/coderanirban71/aditya-l1-solar-flare-nowcasting-and-forecasting-model/blob/main/dashboard/style.css): stylesheet · Presentation layer for the dashboard. · `dashboard/style.css`

### External sources

Upstream instrument and reference data used during preprocessing and benchmarking.

- **PRADAN**: data portal · Aditya-L1 Level-1 satellite data source.
- **NASA DONKI**: flare catalog · Reference flare events used for benchmarking and labeling.

## Connections

- PRADAN → Ingest data: raw telemetry · Feeds Level-1 Aditya-L1 files into preprocessing.
- Ingest data → Trigger filter: master data · Passes unified stream to statistical triggering.
- Trigger filter → Shape filter: candidates · Hands off statistically significant events for shape validation.
- Shape filter → Benchmark events: flare events · Validated candidates are matched to external references.
- NASA DONKI → Benchmark events: reference labels · Provides external flare catalog matches for benchmarking.
- Offline pipeline → Ingest data: orchestrates · Executes preprocessing stages in sequence.
- Offline pipeline → Trigger filter: orchestrates · Executes preprocessing stages in sequence.
- Offline pipeline → Shape filter: orchestrates · Executes preprocessing stages in sequence.
- Offline pipeline → Benchmark events: orchestrates · Executes preprocessing stages in sequence.
- Benchmark events → Outputs: labeled data · Materializes training-ready datasets and catalogs.
- Train model → Outputs: trained artifacts · Produces model files and metadata for serving.
- Outputs → Model meta: model config · Provides feature schema and thresholds to runtime.
- Outputs → Forecast: model files · Supplies trained forecast artifacts for inference.
- Outputs → API server: catalog/data · Supplies derived datasets and runtime assets.
- Nowcast → API server: alert state · Provides real-time detection status and flare class.
- Forecast → API server: probabilities · Provides future flare likelihoods to the API.
- API server → Dashboard: REST data · Serves state, charts, alerts, and controls to the UI.
- Dashboard → UI logic: bootstraps · HTML loads client-side application logic.
- Dashboard → UI styles: styles · HTML page uses the dashboard stylesheet.
- UI logic → API server: polls · Fetches live state and control responses from the backend.
