# nidhaloff/igel architecture

> Architecture diagram of the GitHub repository nidhaloff/igel, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nidhaloff/igel
- Repository: https://github.com/nidhaloff/igel

## Overview

##### Project Type and Purpose
- This is a CLI-first, configuration-driven machine-learning orchestration tool (igel) written in Python.
- It provides commands for fit/train, evaluate, predict, experiment, export (to ONNX), and serve (via a REST API).
- It also exposes a Python API (`Igel` class) and organizes examples, tests, docs, and a FastAPI-based server.

##### High-Level Architecture Overview
- Layers:
1. Presentation Layer:
- CLI entrypoint (`igel/__main__.py`)
- Python API (`Igel` class in `igel/igel.py`)
- Example scripts (`examples/*.py`)
2. Core Engine Layer:
- Config parser & validator (`igel/configs.py`, `pyproject.toml`, CLI parsers)
- Data ingestion (`igel/data.py`)
- Preprocessing (`igel/preprocessing.py`)
- Model orchestration (fit/evaluate/predict in `igel/igel.py`)
- Auto-ML subpackage (`igel/auto/*`)
- Hyperparameter support (`igel/hyperparams.py`)
3. Serving Layer:
- FastAPI server adapter (`igel/servers/fastapi_server.py`)
- Uvicorn runner (invoked under the hood)
4. Persistence Layer & Artifacts:
- File-system–based `model_results` folder (serialized models, JSON descriptions, CSV predictions)
- Example data under `examples/data`
5. External Dependencies:
- Pandas, scikit-learn, FastAPI, Uvicorn, ONNX exporter

##### Main Components and Interactions
- CLI Tool
- Parses commands (`fit`, `evaluate`, `predict`, `experiment`, `export`, `serve`) and flags (`--data_path`, `--yaml_path`, `--model_results_dir`, etc.).
- Instantiates the core `Igel` engine with a `cmd` argument.
- Config Loader
- Reads YAML/JSON using `igel/configs.py`.
- Produces a configuration object driving the pipeline.
- Data Layer
- `igel/data.py` uses pandas under the hood, configured by `dataset.read_data_options`.
- Splits and dispatches DataFrames to preprocessing.
- Preprocessing
- Handles missing values, encoding, scaling as directed by config.
- Outputs processed features and targets.
- Model Lifecycle
1. Fit/Train: calls the chosen algorithm (scikit-learn estimator or AutoML)
2. Evaluate: loads model artifact, scores on new data, writes `evaluation.json`
3. Predict: loads model artifact, writes `predictions.csv`
4. Export: converts sklearn model to ONNX
- Auto-ML
- Special pipelines under `igel/auto/` for Image/Text/Structured tasks.
- Serving
- FastAPI app defines `/predict` endpoint.
- On start (`igel serve`), the engine loads `model_results` and mounts the model.
- Uvicorn runs the ASGI app.
- Tests & Examples
- Unit tests (`tests/`) validate each layer.
- Example scripts show API usage and serve as template integrations.

##### Architectural Patterns and Principles
- Layered Architecture: clear separation between CLI/UI, core engine, serving, external libs.
- Pipeline Pattern: data flows through ingestion → preprocessing → modeling → persistence.
- Adapter Pattern: CLI and Python API adapt user requests into core engine calls; FastAPI adapter wraps the engine for REST.
- Configuration-Driven: YAML/JSON drives behavior, minimizing code changes.
- Modular Design: each concern lives in its own module or package (`auto`, `servers`, `preprocessing`, etc.).

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% CLI/UI Layer
    subgraph "CLI/UI" 
        direction TB
        CLI["igel CLI"]:::cli
        API["Python API (Igel class)"]:::cli
    end

    %% Core Engine Layer
    subgraph "Core Engine" 
        direction TB
        CFG["Config Loader & Validator"]:::core
        CFGFile["YAML/JSON Config"]:::core
        DATA["Data Reader"]:::core
        PRE["Preprocessor"]:::core
        ORCH["Model Orchestration"]:::core
        ONNX["ONNX Exporter"]:::core
        AUTO["Auto-ML Pipelines"]:::core
        HYP["Hyperparameter Support"]:::core
        UTIL["Utility Functions"]:::core
    end

    %% Serving Layer
    subgraph "Serving" 
        direction TB
        FASTAPI["FastAPI Server"]:::serving
        UVIC["Uvicorn Runner"]:::serving
    end

    %% Persistence Layer
    subgraph "Persistence" 
        direction TB
        MODEL["model_results folder"]:::persistence
        EXDATA["Example Data"]:::persistence
    end

    %% External Dependencies
    subgraph "External Dependencies" 
        direction TB
        PANDAS["pandas"]:::external
        SKLEARN["scikit-learn"]:::external
        FASTAPI_LIB["FastAPI"]:::external
        UVIC_LIB["Uvicorn"]:::external
        ONNX_LIB["ONNX"]:::external
    end

    %% Connections
    CLI -->|invokes| CFG
    API -->|calls| CFG
    CFG -->|reads| CFGFile
    CFG -->|validates| PRE
    CFG -->|drives| DATA
    DATA -->|uses pandas| PANDAS
    DATA -->|sends data to| PRE
    PRE -->|produces features| ORCH
    ORCH -->|saves artifacts to| MODEL
    MODEL -->|loads for evaluate/predict| ORCH
    ORCH -->|exports model to ONNX| ONNX
    ONNX -->|writes to| MODEL
    ORCH -->|calls Auto-ML| AUTO
    ORCH -->|uses hyperparams| HYP
    ORCH -->|uses utils| UTIL

    CLI -->|serve command| FASTAPI
    FASTAPI -->|loads model from| MODEL
    FASTAPI -->|serves endpoint| UVIC
    UVIC -->|runs ASGI app| FASTAPI_LIB
    FASTAPI -->|calls| ORCH

    EXDATA -->|sample input| DATA

    %% External libs usage
    PRE -->|may use| SKLEARN
    ORCH -->|uses| SKLEARN
    ORCH -->|uses| ONNX_LIB

    %% Click Events
    click CLI "https://github.com/nidhaloff/igel/blob/master/igel/__main__.py"
    click API "https://github.com/nidhaloff/igel/blob/master/igel/igel.py"
    click CFG "https://github.com/nidhaloff/igel/blob/master/igel/configs.py"
    click CFGFile "https://github.com/nidhaloff/igel/blob/master/pyproject.toml"
    click DATA "https://github.com/nidhaloff/igel/blob/master/igel/data.py"
    click PRE "https://github.com/nidhaloff/igel/blob/master/igel/preprocessing.py"
    click ORCH "https://github.com/nidhaloff/igel/blob/master/igel/igel.py"
    click ONNX "https://github.com/nidhaloff/igel/blob/master/igel/igel.py"
    click AUTO "https://github.com/nidhaloff/igel/blob/master/igel/auto/__init__.py"
    click HYP "https://github.com/nidhaloff/igel/blob/master/igel/hyperparams.py"
    click UTIL "https://github.com/nidhaloff/igel/blob/master/igel/utils.py"
    click FASTAPI "https://github.com/nidhaloff/igel/blob/master/igel/servers/fastapi_server.py"
    click UVIC "https://github.com/nidhaloff/igel/blob/master/igel/servers/helper.py"
    click EXDATA "https://github.com/nidhaloff/igel/tree/master/examples/data/"
    click EXDATA "https://github.com/nidhaloff/igel/tree/master/examples/data/"
    click AUTO "https://github.com/nidhaloff/igel/blob/master/igel/auto/cnn.py"
    click AUTO "https://github.com/nidhaloff/igel/blob/master/igel/auto/defaults.py"
    click AUTO "https://github.com/nidhaloff/igel/blob/master/igel/auto/example.py"
    click AUTO "https://github.com/nidhaloff/igel/blob/master/igel/auto/models.py"

    %% Styles
    classDef cli fill:#D0E8FF,stroke:#339,stroke-width:1px
    classDef core fill:#DFFFD0,stroke:#393,stroke-width:1px
    classDef serving fill:#F0D0FF,stroke:#939,stroke-width:1px
    classDef persistence fill:#F0F0F0,stroke:#666,stroke-width:1px
    classDef external fill:#FFE8D0,stroke:#933,stroke-width:1px
```
