# kohakublueleaf/hakuriver architecture

> Architecture diagram of the GitHub repository kohakublueleaf/hakuriver, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kohakublueleaf/hakuriver
- Repository: https://github.com/kohakublueleaf/hakuriver

## Overview

##### Main Components
- Host Server (hakuriver.host): This module acts as the cluster “brain”. It coordinates task submissions from clients, allocates tasks to available compute nodes based on free CPU cores, maintains a persistent SQLite database (located via the configuration in the db folder and default_config.toml), and handles status updates and heartbeats from runners.
- Runner Agent (hakuriver.runner): Running on each compute node, this component registers itself with the host, reports available CPU core counts, monitors task executions, and uses system tools (like numactl) for task core isolation.
- Client (hakuriver.client): This provides the command-line interface for users. It submits tasks to the host server, queries task statuses, and can issue control commands (like kill or list nodes).
- Storage: Although not a coded component, the architecture relies on two storage types:
– Shared Storage (shared_dir): Used to store scripts and task outputs, accessible with the same path by all nodes.
– Local Temporary Storage (local_temp_dir): Each runner’s local scratch space.
- Hakurun Utility (demo_hakurun.py and the hakurun CLI in the file tree): A local tool for expanding command-line arguments and running combinations of tasks in parallel. It is integral for pre-submission testing and batch processing of similar tasks.

##### File Structure for Implementation Details
- The source code is located under src/hakuriver and src/hakurun. Under “hakuriver”, you have well-organized subdirectories for cli, core, database (db), and utilities, indicating clear module separation.
- The “cli” directories in both locations (src/hakuriver/cli and src/hakurun) are dedicated to user interactions.
- Key modules (client.py, host.py, runner.py) are duplicated in both cli and core directories, suggesting standard interfaces for external commands and core functionality.
- The “utils” directory shows support functionality such as configuration loading, logging, and even a snowflake generator for unique identifiers (likely for task IDs).

##### Architectural Patterns and Design Principles
- Service Separation: The clear separation between Host, Runner, and Client simplifies management and scalability.
- Modularity: The file/directory structure indicates that each component (CLI, core, DB, utils) is independently maintained.
- Lightweight Resource Management: Emphasize the architecture’s focus on CPU-bound tasks and minimalistic multi-node coordination (as opposed to full-fledged HPC schedulers).

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Client Layer"
        Client["HakuRiver Client"]:::client
    end

    subgraph "Cluster Core"
        Host["HakuRiver Host"]:::host
        Runner["HakuRiver Runner"]:::runner
        Database["SQLite Database"]:::database
        SharedStorage["Shared Storage"]:::storage
        LocalTemp["Local Temp Storage"]:::storage
    end

    subgraph "Utility Tools"
        Hakurun["Hakurun Utility"]:::utility
    end

    Client -->|"SubmitTask/StatusQuery"| Host
    Host -->|"TaskDispatch/HeartbeatReq"| Runner
    Runner -->|"Heartbeat/TaskResult"| Host
    Host <-->|"RecordTaskInfo"| Database
    Host -->|"ReadWrite"| SharedStorage
    Runner -->|"ReadWrite"| SharedStorage
    Runner -->|"LocalScratch"| LocalTemp
    Hakurun -->|"PrepareTasks"| Client

    click Client "https://github.com/kohakublueleaf/hakuriver/blob/main/src/hakuriver/cli/client.py"
    click Client "https://github.com/kohakublueleaf/hakuriver/blob/main/src/hakuriver/core/client.py"

    click Host "https://github.com/kohakublueleaf/hakuriver/blob/main/src/hakuriver/cli/host.py"
    click Host "https://github.com/kohakublueleaf/hakuriver/blob/main/src/hakuriver/core/host.py"

    click Runner "https://github.com/kohakublueleaf/hakuriver/blob/main/src/hakuriver/cli/runner.py"
    click Runner "https://github.com/kohakublueleaf/hakuriver/blob/main/src/hakuriver/core/runner.py"

    click Database "https://github.com/kohakublueleaf/hakuriver/tree/main/src/hakuriver/db"
    click Database "https://github.com/kohakublueleaf/hakuriver/blob/main/src/hakuriver/utils/default_config.toml"

    click Hakurun "https://github.com/kohakublueleaf/hakuriver/blob/main/demo_hakurun.py"
    click Hakurun "https://github.com/kohakublueleaf/hakuriver/tree/main/src/hakurun"

    classDef client fill:#f9e79f,stroke:#b7950b,stroke-width:2px;
    classDef host fill:#aed6f1,stroke:#21618c,stroke-width:2px;
    classDef runner fill:#a9dfbf,stroke:#27ae60,stroke-width:2px;
    classDef database fill:#f5cba7,stroke:#e67e22,stroke-width:2px;
    classDef storage fill:#d7bde2,stroke:#8e44ad,stroke-width:2px;
    classDef utility fill:#fad7a0,stroke:#d35400,stroke-width:2px;
```
