# elhachimi-ch/climatefiller architecture

> Architecture diagram of the GitHub repository elhachimi-ch/climatefiller, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/elhachimi-ch/climatefiller
- Repository: https://github.com/elhachimi-ch/climatefiller

## Overview

##### Project type and purpose
- ClimateFiller is a Python framework (library + CLI tool) for manipulating in-situ climate time-series: gap-filling, bias-correction (ML + ERA5-Land), outlier detection (Isolation Forest, LOF, quantiles).
- It is not a multi-service web app but a modular data-processing toolkit with external dependencies (CDS API, Google Earth Engine).

##### File structure
- Top-level scripts:
- climatefiller.py (core class/module)
- kitchen.py, kitchen_v2.py (main entrypoints/orchestrators)
- quantilesdetector.py, gap_filling.py, et0_models_calibrations.py, temp_test.py (specialized workflows)
- Library modules: lib.py, utils.py
- Data folder: raw CSVs (climate_ts.csv, los_angeles_…, ws*.csv)
- catboost_info/: ML model training logs, checkpoints
- Config & docs: environment.yml, requirements.txt, .readthedocs.yaml, docs/ & source/ (Sphinx)
- Test artifacts: test-utils.ipynb

##### README for additional insights
- External services:
- Climate Data Store API (cdsapi)
- Google Earth Engine (earthengine API)
- Installation flows: conda env + CLI run (kitchen.py)
- Demo usage: DataFrame → ClimateFiller class → missing_data → outlier elimination

##### Key recommendations
- Flow from raw input → processing → output → model storage.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Orchestrator / CLI
    subgraph "Orchestrator / CLI"
        Kitchen["kitchen.py"]:::orchestrator
        KitchenV2["kitchen_v2.py"]:::orchestrator
    end

    %% Data Sources
    subgraph "Data Sources"
        LocalCSV["Local CSV Files"]:::data
        CDS["COPERNICUS CDS API"]:::external
        GEE["Google Earth Engine API"]:::external
    end

    %% Ingestion & Pre-processing
    subgraph "Ingestion & Pre-processing"
        DFLoader["DataFrame Loader"]:::processing
        Utils["utils.py"]:::processing
        Lib["lib.py"]:::processing
    end

    %% Core Processing Modules
    subgraph "Core Processing Modules"
        ClimateFiller["climatefiller.py"]:::processing
        Outlier["quantilesdetector.py"]:::processing
        GapFiller["gap_filling.py"]:::processing
        BiasCorrector["et0_models_calibrations.py"]:::processing
    end

    %% Specialized Workflows
    subgraph "Specialized Workflows"
        ETOPrep["eto_preparation_r3_davis.py"]:::processing
        TempTest["temp_test.py"]:::processing
    end

    %% Model Training & Storage
    subgraph "Model Training & Storage"
        CatBoost["catboost_info/"]:::storage
    end

    %% Documentation & Tests
    subgraph "Documentation & Tests"
        Docs["docs/ & source/"]:::docs
        TestNB["test-utils.ipynb"]:::docs
    end

    %% Connections
    Kitchen -->|".run workflow"| DFLoader
    Kitchen -->|".run workflow"| ClimateFiller
    Kitchen -->|".run workflow"| Outlier
    Kitchen -->|".run workflow"| GapFiller
    Kitchen -->|".run workflow"| BiasCorrector
    Kitchen -->|".run workflow"| ETOPrep
    Kitchen -->|".run workflow"| TempTest

    KitchenV2 -->|".run workflow"| DFLoader
    KitchenV2 -->|".run workflow"| ClimateFiller
    KitchenV2 -->|".run workflow"| Outlier
    KitchenV2 -->|".run workflow"| GapFiller
    KitchenV2 -->|".run workflow"| BiasCorrector
    KitchenV2 -->|".run workflow"| ETOPrep
    KitchenV2 -->|".run workflow"| TempTest

    LocalCSV -->|".load_csv()"| DFLoader
    CDS -->|".fetch()"| DFLoader
    GEE -->|".authenticate & query()"| DFLoader

    DFLoader -->|"uses"| Utils
    Utils -->|"helpers"| ClimateFiller
    Utils -->|"helpers"| Outlier
    Utils -->|"helpers"| GapFiller
    Utils -->|"helpers"| BiasCorrector

    Lib -->|"extensions"| ClimateFiller
    Lib -->|"extensions"| Outlier
    Lib -->|"extensions"| GapFiller
    Lib -->|"extensions"| BiasCorrector

    DFLoader -->|"feeds data"| ClimateFiller
    DFLoader -->|"feeds data"| Outlier
    DFLoader -->|"feeds data"| GapFiller
    DFLoader -->|"feeds data"| BiasCorrector

    BiasCorrector -->|"load/train model"| CatBoost
    CatBoost -->|"model data"| BiasCorrector

    Docs -.->|"supports"| Kitchen
    Docs -.->|"supports"| ClimateFiller
    Docs -.->|"supports"| Outlier
    Docs -.->|"supports"| GapFiller
    Docs -.->|"supports"| BiasCorrector

    TestNB -.->|"examples"| Kitchen

    %% Click Events
    click LocalCSV "https://github.com/elhachimi-ch/climatefiller/tree/main/data/"
    click ClimateFiller "https://github.com/elhachimi-ch/climatefiller/blob/main/climatefiller.py"
    click Utils "https://github.com/elhachimi-ch/climatefiller/blob/main/utils.py"
    click Lib "https://github.com/elhachimi-ch/climatefiller/blob/main/lib.py"
    click Outlier "https://github.com/elhachimi-ch/climatefiller/blob/main/quantilesdetector.py"
    click GapFiller "https://github.com/elhachimi-ch/climatefiller/blob/main/gap_filling.py"
    click BiasCorrector "https://github.com/elhachimi-ch/climatefiller/blob/main/et0_models_calibrations.py"
    click ETOPrep "https://github.com/elhachimi-ch/climatefiller/blob/main/eto_preparation_r3_davis.py"
    click TempTest "https://github.com/elhachimi-ch/climatefiller/blob/main/temp_test.py"
    click Kitchen "https://github.com/elhachimi-ch/climatefiller/blob/main/kitchen.py"
    click KitchenV2 "https://github.com/elhachimi-ch/climatefiller/blob/main/kitchen_v2.py"
    click CatBoost "https://github.com/elhachimi-ch/climatefiller/tree/main/catboost_info/"
    click Docs "https://github.com/elhachimi-ch/climatefiller/tree/main/docs/"
    click TestNB "https://github.com/elhachimi-ch/climatefiller/blob/main/test-utils.ipynb"

    %% Styles
    classDef data fill:#BBDEFB,stroke:#1E88E5,color:#0D47A1,stroke-width:1px;
    classDef processing fill:#C8E6C9,stroke:#2E7D32,color:#1B5E20,stroke-width:1px;
    classDef external fill:#FFE0B2,stroke:#FB8C00,color:#E65100,stroke-width:1px;
    classDef orchestrator fill:#E0E0E0,stroke:#757575,color:#424242,stroke-width:1px;
    classDef storage fill:#B3E5FC,stroke:#0288D1,color:#01579B,stroke-width:1px,stroke-dasharray: 5 5;
    classDef docs fill:#E1BEE7,stroke:#8E24AA,color:#4A148C,stroke-width:1px,stroke-dasharray: 2 2;
```
