# knamlx/ionoautoml architecture

> Architecture diagram of the GitHub repository knamlx/ionoautoml, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/knamlx/ionoautoml
- Repository: https://github.com/knamlx/ionoautoml

## Overview

IonoAutoML is a research pipeline for collecting ionospheric and geophysical observations, preparing station time series, engineering forecasting features, and evaluating foF2 models. The main workflow runs from external data sources through cleaning, time-grid normalization, optional filtering, feature creation, and station-wise ML experiments; outputs support quality and latitude-zone analysis. The README documents the stages and capabilities, while sampled source confirms collection, transformation, filtering, feature generation, model fitting, and batch dispatch. Some subsystem internals are unsampled, so only documented or directly evidenced links are shown.

Main entry points: [`collect_hf_data.py`](https://github.com/knamlx/ionoautoml/blob/main/collect_hf_data.py) (Data collection), [`normalize_time_grid.py`](https://github.com/knamlx/ionoautoml/blob/main/normalize_time_grid.py) (Time normalization), [`run_ml_baselines.py`](https://github.com/knamlx/ionoautoml/blob/main/run_ml_baselines.py) (Model training), [`clean_collected_data.py`](https://github.com/knamlx/ionoautoml/blob/main/clean_collected_data.py) (Data cleaning)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_collection["Data collection"]
  node_collect["Data collection<br/>[collect_hf_data.py]"]
  node_rawdata["Raw observations"]
end

subgraph group_preparation["Data preparation"]
  node_clean["Data cleaning"]
  node_cleaned["Cleaned tables"]
  node_normalize["Time normalization"]
  node_stationdata["Station time grids"]
  node_filter["Series filtering"]
  node_features["Feature engineering<br/>[build_features.py]"]
  node_featuredata["Feature datasets"]
  node_stationconfig["Station selection"]
end

subgraph group_modeling["Forecasting"]
  node_experimentconfig["Experiment settings"]
  node_batch["Station batch runner"]
  node_training["Model training"]
  node_evaluation["Walk-forward evaluation"]
  node_artifacts["Metrics and predictions"]
  node_progress["Run progress"]
end

subgraph group_analysis["Results analysis"]
  node_review["ML results review"]
  node_latitude["Latitude-zone analysis"]
  node_reports["Analysis reports"]
end

node_researcher(("Researcher"))
node_giro{{"GIRO DIDBase"}}
node_gfz{{"GFZ Kp API"}}
node_omni{{"NASA OMNIWeb"}}
node_noaa{{"NOAA SWPC"}}

node_researcher -->|"starts collection"| node_collect
node_collect -->|"fetches observations"| node_giro
node_collect -->|"fetches indices"| node_gfz
node_collect -->|"fetches solar wind"| node_omni
node_collect -.->|"fetches live data"| node_noaa
node_collect -->|"writes raw data"| node_rawdata
node_researcher -->|"runs cleaning"| node_clean
node_rawdata -->|"reads tables"| node_clean
node_clean -->|"writes cleaned tables"| node_cleaned
node_researcher -->|"runs normalization"| node_normalize
node_cleaned -->|"reads processed data"| node_normalize
node_normalize -->|"writes station grids"| node_stationdata
node_researcher -->|"runs filtering"| node_filter
node_stationdata -->|"reads time grids"| node_filter
node_filter -->|"filtered input"| node_features
node_stationdata -->|"reads station data"| node_features
node_features -->|"writes training tables"| node_featuredata
node_stationconfig -->|"configures selection"| node_normalize
node_researcher -->|"starts station runs"| node_batch
node_batch -->|"dispatches station jobs"| node_training
node_experimentconfig -->|"configures experiment"| node_training
node_featuredata -->|"provides features"| node_training
node_training -->|"evaluates forecasts"| node_evaluation
node_evaluation -->|"saves metrics"| node_artifacts
node_artifacts -->|"reads run status"| node_progress
node_artifacts -->|"reviews results"| node_review
node_artifacts -->|"summarizes model quality"| node_latitude
node_latitude -->|"builds analysis reports"| node_reports

click node_collect "https://github.com/knamlx/ionoautoml/blob/main/collect_hf_data.py"
click node_clean "https://github.com/knamlx/ionoautoml/blob/main/clean_collected_data.py"
click node_normalize "https://github.com/knamlx/ionoautoml/blob/main/normalize_time_grid.py"
click node_filter "https://github.com/knamlx/ionoautoml/blob/main/filter_time_series.py"
click node_features "https://github.com/knamlx/ionoautoml/blob/main/build_features.py"
click node_stationconfig "https://github.com/knamlx/ionoautoml/blob/main/configs/station_selection.json"
click node_experimentconfig "https://github.com/knamlx/ionoautoml/tree/main/configs/experiments"
click node_batch "https://github.com/knamlx/ionoautoml/blob/main/run_station_batch.py"
click node_training "https://github.com/knamlx/ionoautoml/blob/main/run_ml_baselines.py"
click node_evaluation "https://github.com/knamlx/ionoautoml/blob/main/run_ml_baselines.py"
click node_progress "https://github.com/knamlx/ionoautoml/blob/main/watch_run_progress.py"
click node_review "https://github.com/knamlx/ionoautoml/blob/main/notebooks/ml_results_review.ipynb"
click node_latitude "https://github.com/knamlx/ionoautoml/blob/main/scripts/build_latitude_zone_article.py"
click node_reports "https://github.com/knamlx/ionoautoml/tree/main/reports"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_collect,node_rawdata toneBlue
class node_clean,node_cleaned,node_normalize,node_stationdata,node_filter,node_features,node_featuredata,node_stationconfig toneAmber
class node_experimentconfig,node_batch,node_training,node_evaluation,node_artifacts,node_progress,node_gfz toneMint
class node_review,node_latitude,node_reports toneRose
class node_researcher,node_giro,node_omni,node_noaa toneIndigo
```

## Components

### Data collection

- [Data collection](https://github.com/knamlx/ionoautoml/blob/main/collect_hf_data.py): `collect_hf_data.py`
- **Raw observations**

### Data preparation

- [Data cleaning](https://github.com/knamlx/ionoautoml/blob/main/clean_collected_data.py): `clean_collected_data.py`
- **Cleaned tables**
- [Time normalization](https://github.com/knamlx/ionoautoml/blob/main/normalize_time_grid.py): `normalize_time_grid.py`
- **Station time grids**
- [Series filtering](https://github.com/knamlx/ionoautoml/blob/main/filter_time_series.py): `filter_time_series.py`
- [Feature engineering](https://github.com/knamlx/ionoautoml/blob/main/build_features.py): `build_features.py`
- **Feature datasets**
- [Station selection](https://github.com/knamlx/ionoautoml/blob/main/configs/station_selection.json): `configs/station_selection.json`

### Forecasting

- [Experiment settings](https://github.com/knamlx/ionoautoml/tree/main/configs/experiments): `configs/experiments`
- [Station batch runner](https://github.com/knamlx/ionoautoml/blob/main/run_station_batch.py): `run_station_batch.py`
- [Model training](https://github.com/knamlx/ionoautoml/blob/main/run_ml_baselines.py): `run_ml_baselines.py`
- [Walk-forward evaluation](https://github.com/knamlx/ionoautoml/blob/main/run_ml_baselines.py): `run_ml_baselines.py`
- **Metrics and predictions**
- [Run progress](https://github.com/knamlx/ionoautoml/blob/main/watch_run_progress.py): `watch_run_progress.py`

### Results analysis

- [ML results review](https://github.com/knamlx/ionoautoml/blob/main/notebooks/ml_results_review.ipynb): `notebooks/ml_results_review.ipynb`
- [Latitude-zone analysis](https://github.com/knamlx/ionoautoml/blob/main/scripts/build_latitude_zone_article.py): `scripts/build_latitude_zone_article.py`
- [Analysis reports](https://github.com/knamlx/ionoautoml/tree/main/reports): `reports`

### Other components

- **Researcher**
- **GIRO DIDBase**
- **GFZ Kp API**
- **NASA OMNIWeb**
- **NOAA SWPC**

## Connections

- Researcher → Data collection: starts collection (evidence: `README.md`)
- Data collection → GIRO DIDBase: fetches observations (evidence: `README.md`)
- Data collection → GFZ Kp API: fetches indices (evidence: `README.md`)
- Data collection → NASA OMNIWeb: fetches solar wind (evidence: `README.md`)
- Data collection → NOAA SWPC: fetches live data (evidence: `README.md`)
- Data collection → Raw observations: writes raw data (evidence: `README.md`)
- Researcher → Data cleaning: runs cleaning (evidence: `README.md`)
- Raw observations → Data cleaning: reads tables (evidence: `README.md`)
- Data cleaning → Cleaned tables: writes cleaned tables (evidence: `clean_collected_data.py`)
- Researcher → Time normalization: runs normalization (evidence: `README.md`)
- Cleaned tables → Time normalization: reads processed data (evidence: `README.md`)
- Time normalization → Station time grids: writes station grids (evidence: `README.md`)
- Researcher → Series filtering: runs filtering (evidence: `README.md`)
- Station time grids → Series filtering: reads time grids (evidence: `README.md`)
- Series filtering → Feature engineering: filtered input (evidence: `README.md`)
- Station time grids → Feature engineering: reads station data (evidence: `README.md`)
- Feature engineering → Feature datasets: writes training tables (evidence: `build_features.py`)
- Station selection → Time normalization: configures selection (evidence: `README.md`)
- Researcher → Station batch runner: starts station runs (evidence: `README.md`)
- Station batch runner → Model training: dispatches station jobs (evidence: `run_station_batch.py`)
- Experiment settings → Model training: configures experiment (evidence: `README.md`)
- Feature datasets → Model training: provides features (evidence: `README.md`)
- Model training → Walk-forward evaluation: evaluates forecasts (evidence: `run_ml_baselines.py`)
- Walk-forward evaluation → Metrics and predictions: saves metrics (evidence: `README.md`)
- Metrics and predictions → Run progress: reads run status (evidence: `watch_run_progress.py`)
- Metrics and predictions → ML results review: reviews results (evidence: `README.md`)
- Metrics and predictions → Latitude-zone analysis: summarizes model quality (evidence: `scripts/build_latitude_zone_article.py`)
- Latitude-zone analysis → Analysis reports: builds analysis reports (evidence: `README.md`)
