# riyad-77/smartfellah architecture

> Architecture diagram of the GitHub repository riyad-77/smartfellah, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/riyad-77/smartfellah
- Repository: https://github.com/riyad-77/smartfellah

## Overview

Smartfellah is an agricultural intelligence backend: it combines crop-reference documents, commune and field data, weather and satellite observations, and serves agronomic advice and risk assessments through an API. A documented document-ingestion workflow extracts PDF content, consolidates crop records, and loads them into PostgreSQL; scheduled workflows gather weather and satellite data. The API and a separate risk engine query stored data. Model training is another capability. Source excerpts are partial and the README is empty, so unverified connections are omitted.

Main entry points: [`init.sql`](https://github.com/riyad-77/smartfellah/blob/main/init.sql) (Agriculture Database), [`dags/dag_documents_agri.py`](https://github.com/riyad-77/smartfellah/blob/main/dags/dag_documents_agri.py) (Document Workflow), [`dags/ingest_meteo.py`](https://github.com/riyad-77/smartfellah/blob/main/dags/ingest_meteo.py) (Weather Collection), [`dags/ingest_gee.py`](https://github.com/riyad-77/smartfellah/blob/main/dags/ingest_gee.py) (Satellite Indices)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_documents["Crop References"]
  node_docs_dag["Document Workflow"]
  node_extract["PDF Extraction"]
  node_consolidate["Crop Consolidation<br/>[Consolidation.py]"]
  node_crop_load["Crop Data Loader"]
  node_crop_data["Crop JSON Records"]
  node_pdfs["Agronomy PDFs"]
end

subgraph group_ingestion["Data Ingestion"]
  node_db[("Agriculture Database<br/>[init.sql]")]
  node_daily_dag["Daily Ingestion"]
  node_weather["Weather Collection<br/>[ingest_meteo.py]"]
  node_satellite["Satellite Indices<br/>[ingest_gee.py]"]
  node_forecast_dag["Forecast Workflow"]
  node_communes["Commune Seeding<br/>[seed_communes.py]"]
end

subgraph group_services["Advice Services"]
  node_api["Smartfellah API<br/>[api_smartfellah.py]"]
  node_risk["Disease Risk Engine<br/>[risk_engine.py]"]
end

subgraph group_models["Risk Models"]
  node_disease_train["Disease Model Training"]
  node_model_train["Forecast Model Training<br/>[train_models.py]"]
  node_models_store["Trained Models"]
end

node_farmer(("Farmer or Client"))
node_openmeteo(("Open-Meteo"))
node_earth_engine(("Google Earth Engine"))
node_openweather(("OpenWeather"))

node_farmer -->|"requests advice"| node_api
node_api -->|"queries"| node_db
node_risk -->|"queries observations"| node_db
node_docs_dag -->|"runs extraction"| node_extract
node_docs_dag -->|"runs consolidation"| node_consolidate
node_docs_dag -->|"runs loader"| node_crop_load
node_docs_dag -->|"orders after extraction"| node_consolidate
node_docs_dag -->|"orders after consolidation"| node_crop_load
node_crop_load -->|"loads crop records"| node_db
node_daily_dag -->|"runs before satellite"| node_weather
node_daily_dag -->|"runs after weather"| node_satellite
node_weather -->|"fetches weather"| node_openmeteo
node_weather -->|"writes observations"| node_db
node_satellite -->|"fetches indices"| node_earth_engine
node_satellite -->|"writes indices"| node_db
node_forecast_dag -->|"fetches forecasts"| node_openweather
node_forecast_dag -->|"stores forecasts"| node_db
node_communes -->|"updates communes"| node_db
node_disease_train -->|"reads weather"| node_db
node_disease_train -.->|"saves classifier"| node_models_store
node_model_train -.->|"saves models"| node_models_store

click node_api "https://github.com/riyad-77/smartfellah/blob/main/api_smartfellah.py"
click node_risk "https://github.com/riyad-77/smartfellah/blob/main/risk_engine.py"
click node_db "https://github.com/riyad-77/smartfellah/blob/main/init.sql"
click node_docs_dag "https://github.com/riyad-77/smartfellah/blob/main/dags/dag_documents_agri.py"
click node_extract "https://github.com/riyad-77/smartfellah/blob/main/extract_agri_to_json.py"
click node_consolidate "https://github.com/riyad-77/smartfellah/blob/main/Consolidation.py"
click node_crop_load "https://github.com/riyad-77/smartfellah/blob/main/ingestion_smartfellah.py"
click node_crop_data "https://github.com/riyad-77/smartfellah/tree/main/FT_Clean"
click node_pdfs "https://github.com/riyad-77/smartfellah/tree/main/FT_Vegetal"
click node_daily_dag "https://github.com/riyad-77/smartfellah/blob/main/dags/bifolia_pipeline.py"
click node_weather "https://github.com/riyad-77/smartfellah/blob/main/dags/ingest_meteo.py"
click node_satellite "https://github.com/riyad-77/smartfellah/blob/main/dags/ingest_gee.py"
click node_forecast_dag "https://github.com/riyad-77/smartfellah/blob/main/dags/dag_meteo_forecast.py"
click node_communes "https://github.com/riyad-77/smartfellah/blob/main/seed_communes.py"
click node_disease_train "https://github.com/riyad-77/smartfellah/blob/main/train_disease_model.py"
click node_model_train "https://github.com/riyad-77/smartfellah/blob/main/train_models.py"
click node_models_store "https://github.com/riyad-77/smartfellah/tree/main/models"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_docs_dag,node_extract,node_consolidate,node_crop_load,node_crop_data,node_pdfs,node_farmer toneBlue
class node_db,node_daily_dag,node_weather,node_satellite,node_forecast_dag,node_communes toneAmber
class node_api,node_risk toneMint
class node_disease_train,node_model_train,node_models_store toneRose
class node_openmeteo,node_earth_engine,node_openweather toneIndigo
```

## Components

### Crop References

- [Document Workflow](https://github.com/riyad-77/smartfellah/blob/main/dags/dag_documents_agri.py): `dags/dag_documents_agri.py`
- [PDF Extraction](https://github.com/riyad-77/smartfellah/blob/main/extract_agri_to_json.py): `extract_agri_to_json.py`
- [Crop Consolidation](https://github.com/riyad-77/smartfellah/blob/main/Consolidation.py): `Consolidation.py`
- [Crop Data Loader](https://github.com/riyad-77/smartfellah/blob/main/ingestion_smartfellah.py): `ingestion_smartfellah.py`
- [Crop JSON Records](https://github.com/riyad-77/smartfellah/tree/main/FT_Clean): `FT_Clean`
- [Agronomy PDFs](https://github.com/riyad-77/smartfellah/tree/main/FT_Vegetal): `FT_Vegetal`

### Data Ingestion

- [Agriculture Database](https://github.com/riyad-77/smartfellah/blob/main/init.sql): `init.sql`
- [Daily Ingestion](https://github.com/riyad-77/smartfellah/blob/main/dags/bifolia_pipeline.py): `dags/bifolia_pipeline.py`
- [Weather Collection](https://github.com/riyad-77/smartfellah/blob/main/dags/ingest_meteo.py): `dags/ingest_meteo.py`
- [Satellite Indices](https://github.com/riyad-77/smartfellah/blob/main/dags/ingest_gee.py): `dags/ingest_gee.py`
- [Forecast Workflow](https://github.com/riyad-77/smartfellah/blob/main/dags/dag_meteo_forecast.py): `dags/dag_meteo_forecast.py`
- [Commune Seeding](https://github.com/riyad-77/smartfellah/blob/main/seed_communes.py): `seed_communes.py`

### Advice Services

- [Smartfellah API](https://github.com/riyad-77/smartfellah/blob/main/api_smartfellah.py): `api_smartfellah.py`
- [Disease Risk Engine](https://github.com/riyad-77/smartfellah/blob/main/risk_engine.py): `risk_engine.py`

### Risk Models

- [Disease Model Training](https://github.com/riyad-77/smartfellah/blob/main/train_disease_model.py): `train_disease_model.py`
- [Forecast Model Training](https://github.com/riyad-77/smartfellah/blob/main/train_models.py): `train_models.py`
- [Trained Models](https://github.com/riyad-77/smartfellah/tree/main/models): `models`

### Other components

- **Farmer or Client**
- **Open-Meteo**
- **Google Earth Engine**
- **OpenWeather**

## Connections

- Farmer or Client → Smartfellah API: requests advice (evidence: `api_smartfellah.py`)
- Smartfellah API → Agriculture Database: queries (evidence: `api_smartfellah.py`)
- Disease Risk Engine → Agriculture Database: queries observations (evidence: `risk_engine.py`)
- Document Workflow → PDF Extraction: runs extraction (evidence: `dags/dag_documents_agri.py`)
- Document Workflow → Crop Consolidation: runs consolidation (evidence: `dags/dag_documents_agri.py`)
- Document Workflow → Crop Data Loader: runs loader (evidence: `dags/dag_documents_agri.py`)
- Document Workflow → Crop Consolidation: orders after extraction (evidence: `dags/dag_documents_agri.py`)
- Document Workflow → Crop Data Loader: orders after consolidation (evidence: `dags/dag_documents_agri.py`)
- Crop Data Loader → Agriculture Database: loads crop records (evidence: `ingestion_smartfellah.py`)
- Daily Ingestion → Weather Collection: runs before satellite (evidence: `dags/bifolia_pipeline.py`)
- Daily Ingestion → Satellite Indices: runs after weather (evidence: `dags/bifolia_pipeline.py`)
- Weather Collection → Open-Meteo: fetches weather (evidence: `dags/ingest_meteo.py`)
- Weather Collection → Agriculture Database: writes observations (evidence: `dags/ingest_meteo.py`)
- Satellite Indices → Google Earth Engine: fetches indices (evidence: `dags/ingest_gee.py`)
- Satellite Indices → Agriculture Database: writes indices (evidence: `dags/ingest_gee.py`)
- Forecast Workflow → OpenWeather: fetches forecasts (evidence: `dags/dag_meteo_forecast.py`)
- Forecast Workflow → Agriculture Database: stores forecasts (evidence: `dags/dag_meteo_forecast.py`)
- Commune Seeding → Agriculture Database: updates communes (evidence: `seed_communes.py`)
- Disease Model Training → Agriculture Database: reads weather (evidence: `train_disease_model.py`)
- Disease Model Training → Trained Models: saves classifier
- Forecast Model Training → Trained Models: saves models
