How aseemprasad/air-quality-intelligence works
This is a local-first air-quality monitoring and forecasting platform. Its principal workflow collects OpenAQ observations and Open-Meteo weather, parses and deduplicates records, stores them as partitioned Parquet, and uses quality checks and analytics to support anomaly/event detection and forecasting. Results are exposed through a FastAPI service. The graph also includes the documented control-plane database and the main intelligence capabilities. Some analytics, quality, and serving links are based on README architecture statements; their implementation details were not sampled.
Main entry points: orchestrator.py Ingestion orchestration · main.py FastAPI application · parquet_io.py Partitioned Parquet · openaq.py Air-quality connector
Components 26
Ingestion
- Ingestion orchestration
src/aq_engine/ingestion/orchestrator.py - Air-quality connector
src/aq_engine/connectors/openaq.py - Weather connector
src/aq_engine/connectors/open_meteo.py - Connector framework
src/aq_engine/connectors/base.py - Canonical record models
src/aq_engine/connectors/models.py
Quality & Storage
- Record contracts
src/aq_engine/quality/contracts.py - Deduplication & hashing
src/aq_engine/quality/deduplication.py - Quality rules
src/aq_engine/quality/rules.py - Record validation
src/aq_engine/quality/validator.py - Quarantine
src/aq_engine/quality/quarantine.py - Partitioned Parquet
src/aq_engine/storage/parquet_io.py
Intelligence
- Hourly / daily aggregation
src/aq_engine/analytics/aggregation.py - Historical baselines
src/aq_engine/analytics/baselines.py - Anomaly detection
src/aq_engine/analytics/anomaly.py - Pollution events
src/aq_engine/analytics/events.py - Station health
src/aq_engine/analytics/station_health.py - Forecast features
src/aq_engine/ml/features.py - Model training
src/aq_engine/ml/training.py - Forecast baselines
src/aq_engine/ml/baselines.py - Forecast inference
src/aq_engine/ml/inference.py
Serving & Control
- Control-plane database
src/aq_engine/storage/db.py - FastAPI application
src/aq_engine/api/main.py - Observation endpoints
src/aq_engine/api/observations.py
Other components
- Operator / scheduler
- OpenAQ
- Open-Meteo
Connections 33
- Operator / scheduler → Ingestion orchestration: triggers
- Ingestion orchestration → Air-quality connector: initializes
src/aq_engine/ingestion/orchestrator.py - Ingestion orchestration → Weather connector: initializes
src/aq_engine/ingestion/orchestrator.py - Air-quality connector → OpenAQ: fetches data
src/aq_engine/connectors/openaq.py - Weather connector → Open-Meteo: fetches data
src/aq_engine/connectors/open_meteo.py - Air-quality connector → Connector framework: inherits
src/aq_engine/connectors/openaq.py - Weather connector → Connector framework: inherits
src/aq_engine/connectors/open_meteo.py - Air-quality connector → Canonical record models: parses into
src/aq_engine/connectors/openaq.py - Weather connector → Canonical record models: parses into
src/aq_engine/connectors/open_meteo.py - Ingestion orchestration → Deduplication & hashing: deduplicates
src/aq_engine/ingestion/orchestrator.py - Ingestion orchestration → Partitioned Parquet: writes records
src/aq_engine/ingestion/orchestrator.py - Ingestion orchestration → Control-plane database: records run state
src/aq_engine/ingestion/orchestrator.py - Ingestion orchestration → Record contracts: uses contracts
src/aq_engine/ingestion/orchestrator.py - Deduplication & hashing → Partitioned Parquet: reads records
src/aq_engine/quality/deduplication.py - Record validation → Quality rules: applies
src/aq_engine/quality/validator.py - Partitioned Parquet → Record validation: validated by
- Record validation → Quarantine: routes rejects
- Partitioned Parquet → Hourly / daily aggregation: transforms
README.md - Hourly / daily aggregation → Historical baselines: supports
- Hourly / daily aggregation → Anomaly detection: analyzes
README.md - Anomaly detection → Pollution events: identifies events
- Hourly / daily aggregation → Station health: informs health
- Partitioned Parquet → Forecast features: builds features
README.md - Forecast features → Model training: trains on
README.md - Forecast features → Forecast inference: provides features
src/aq_engine/ml/inference.py - Model training → Forecast inference: provides model
src/aq_engine/ml/inference.py - Forecast baselines → Forecast inference: fallback predictions
src/aq_engine/ml/inference.py - FastAPI application → Observation endpoints: registers router
src/aq_engine/api/main.py - Operator / scheduler → FastAPI application: requests results
README.md - FastAPI application → Forecast inference: serves forecasts
- FastAPI application → Pollution events: serves events
README.md - FastAPI application → Anomaly detection: serves context
README.md - FastAPI application → Control-plane database: checks health