# dhis2-chap/chap-core architecture

> Architecture diagram of the GitHub repository dhis2-chap/chap-core, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dhis2-chap/chap-core
- Repository: https://github.com/dhis2-chap/chap-core

## Overview

##### Project type and purpose
- This repository is the “Chap modelling platform,” a Python-based modeling framework with:
- A core library (`chap_core/`) implementing data ingestion, preprocessing, forecasting, evaluation and simulation.
- Multiple adapters (CLI, REST API, GluonTS, R, Python external models).
- A RESTful service layer with asynchronous task processing (Celery/RQ).
- Docker-based deployment and orchestration via docker-compose.
- It’s primarily a backend modeling toolkit and service platform, not a UI-driven full-stack app.

##### File structure
- Top-level directories/files:
- `chap_core/`: business logic, adapters, services, database, models, runners, simulation, testing utilities.
- `rest_api_src/`: the FastAPI (or similar) service, Celery tasks, routers, data models.
- `external_models/`: examples of containerized or scripted models in R, Python, Stan, MLflow.
- `scripts/`: utility and tutorial scripts showing common workflows (data cleaning, evaluation, prediction).
- `compose*.yml`, `Dockerfile*`: container definitions for API, workers, integration tests.
- `tests/`: unit, integration, end-to-end tests for every component.
- Architectural cues:
- A hexagonal/clean architecture—core logic decoupled from transport via “adaptors.”
- Asynchronous worker pattern (Celery/RQ) for long-running tasks.
- Pluggable external model interfaces (via MLproject, Docker, Python/R wrappers).
- Persistent storage via a relational database and local cache layers.

##### README for insights
- Confirms it’s a modeling platform with documentation site and GitHub workflows for building images, testing, publishing.
- Mentions CLI (`chap-cli`), API, external models, and developer documentation.

##### Detail
- Every layer from client entry point, through adapters, core modules, DB/cache, to external services.
- Don’t omit auxiliary components like local cache manager, logging, and Docker helper functions—they clarify operational concerns.

## Diagram (Mermaid)

```mermaid
graph TD
    %% Client Layer
    subgraph "Client Layer"
        CLIEntrypoint["CLI Entrypoint"]:::client
        CLITool["CLI Tool"]:::client
    end
    CLIEntrypoint -->|calls| CLITool

    %% Service Layer
    subgraph "Service Layer"
        subgraph "REST API"
            RESTAdaptor["REST API Adaptor"]:::service
            APIServer["FastAPI Server & Routers"]:::service
        end
        MessageBroker["Message Broker (RabbitMQ/Redis)"]:::infra
        CeleryTasks["Celery Task Definitions"]:::service
        WorkerFuncs["Worker Functions"]:::service
        DBWorkerFuncs["DB Worker Helpers"]:::service
    end
    APIServer -->|HTTP/JSON| RESTAdaptor
    RESTAdaptor -->|invokes| DataIngest
    APIServer -->|pushTaskAMQP| MessageBroker
    MessageBroker -->|queues| CeleryTasks
    CeleryTasks -->|calls| WorkerFuncs
    WorkerFuncs -->|uses| DataIngest
    WorkerFuncs -->|dbOps| DBWorkerFuncs

    %% Core Logic Layer
    subgraph "Core Logic Layer"
        DataIngest["Data Ingestion & Preprocessing"]:::core
        Evaluator["Forecasting & Evaluation Suite"]:::core
        Simulator["Simulation Engines"]:::core
        ModelRegistry["Model Registry & Protocols"]:::core
        ExternalInterface["External Model Interface"]:::core
    end
    CLITool -->|sync| DataIngest
    CLITool -->|calls| Evaluator
    CLITool -->|calls| Simulator
    DataIngest -->|readsWrites| ORMTables
    Evaluator -->|readsWrites| ORMTables
    Simulator -->|readsWrites| ORMTables
    DataIngest -->|calls| ExternalInterface

    %% Persistence Layer
    subgraph "Persistence Layer"
        DB["Relational Database"]:::db
        ORMTables["DB ORM & Tables"]:::db
        CacheManager["Local Cache Manager"]:::db
        JSONSession["JSON Session Store"]:::db
    end
    ORMTables --> DB
    CacheManager --> DB
    JSONSession --> CacheManager

    %% External Services & Data Sources
    subgraph "External Services & Data Sources"
        GEE["Google Earth Engine"]:::external
        DHIS2["DHIS2 / Data Portals"]:::external
        GluonTSAdapter["GluonTS Adapter"]:::external
        ExternalContainer["External Model Container"]:::external
    end
    DataIngest -->|HTTPAPI| GEE
    DataIngest -->|HTTPAPI| DHIS2
    ExternalInterface -->|docker| ExternalContainer
    ExternalInterface -->|mlflow| ExternalContainer
    ModelRegistry -->|uses| ExternalContainer
    ModelRegistry -->|uses| GluonTSAdapter

    %% Task Queue Workers
    subgraph "Task Queue Workers"
        BGWorker["Background Tasks Worker"]:::service
        RQWorker["RQ Worker Implementation"]:::service
    end
    MessageBroker -->|push| BGWorker
    MessageBroker -->|push| RQWorker
    BGWorker -->|calls| DataIngest
    RQWorker -->|calls| DataIngest

    %% Deployment & Orchestration
    subgraph "Deployment & Orchestration"
        DockerCompose["Docker Compose Files"]:::infra
    end

    %% Click Events
    click CLITool "https://github.com/dhis2-chap/chap-core/blob/master/chap_core/adaptors/command_line_interface.py"
    click CLIEntrypoint "https://github.com/dhis2-chap/chap-core/blob/master/chap_core/chap_cli.py"
    click RESTAdaptor "https://github.com/dhis2-chap/chap-core/blob/master/chap_core/adaptors/rest_api.py"
    click APIServer "https://github.com/dhis2-chap/chap-core/blob/master/chap_core/rest_api_src/v1/rest_api.py"
    click CeleryTasks "https://github.com/dhis2-chap/chap-core/blob/master/chap_core/rest_api_src/celery_tasks.py"
    click WorkerFuncs "https://github.com/dhis2-chap/chap-core/blob/master/chap_core/rest_api_src/worker_functions.py"
    click DBWorkerFuncs "https://github.com/dhis2-chap/chap-core/blob/master/chap_core/rest_api_src/db_worker_functions.py"
    click DataIngest "https://github.com/dhis2-chap/chap-core/blob/master/chap_core/data/datasets.py"
    click Evaluator "https://github.com/dhis2-chap/chap-core/blob/master/chap_core/assessment/evaluator_suites.py"
    click Simulator "https://github.com/dhis2-chap/chap-core/blob/master/chap_core/simulation/simulator.py"
    click ModelRegistry "https://github.com/dhis2-chap/chap-core/blob/master/chap_core/predictor/model_registry.py"
    click ExternalInterface "https://github.com/dhis2-chap/chap-core/blob/master/chap_core/external/external_model.py"
    click ORMTables "https://github.com/dhis2-chap/chap-core/blob/master/chap_core/database/database.py"
    click ORMTables "https://github.com/dhis2-chap/chap-core/blob/master/chap_core/database/tables.py"
    click CacheManager "https://github.com/dhis2-chap/chap-core/blob/master/chap_core/database/local_db_cache.py"
    click JSONSession "https://github.com/dhis2-chap/chap-core/blob/master/chap_core/database/json_session.py"
    click GEE "https://github.com/dhis2-chap/chap-core/blob/master/chap_core/google_earth_engine/xee_interface.py"
    click GluonTSAdapter "https://github.com/dhis2-chap/chap-core/blob/master/chap_core/adaptors/gluonts.py"
    click BGWorker "https://github.com/dhis2-chap/chap-core/blob/master/chap_core/worker/background_tasks_worker.py"
    click RQWorker "https://github.com/dhis2-chap/chap-core/blob/master/chap_core/worker/rq_worker.py"
    click DockerCompose "https://github.com/dhis2-chap/chap-core/blob/master/compose.yml"

    %% Styles
    classDef client fill:#cce5ff,stroke:#004085;
    classDef service fill:#d4edda,stroke:#155724;
    classDef core fill:#fff3cd,stroke:#856404;
    classDef db fill:#f8d7da,stroke:#721c24;
    classDef external fill:#e2e3e5,stroke:#6c757d;
    classDef infra fill:#f5c6cb,stroke:#721c24;
```
