# ai-hypercomputer/kithara architecture

> Architecture diagram of the GitHub repository ai-hypercomputer/kithara, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ai-hypercomputer/kithara
- Repository: https://github.com/ai-hypercomputer/kithara

## Overview

##### Main Components
- Core Library (kithara package): This is the central part of the system and can be subdivided further:
- Configuration: Contains default settings and utilities (kithara/config).
- Dataset Management: Components for data loading, processing, and streaming (kithara/dataset).
- Models: Modules for model construction, inference, conversion, and checkpoint compatibility (kithara/model, and its submodules like hf_compatibility, kerashub, and maxtext).
- Training/Finetuning: Trainer components including callbacks (kithara/trainer and kithara/callbacks).
- Optimizers: Including wrappers for libraries like optax (kithara/optimizers).
- Distributed Training: Abstractions for data sharding and multi-device support (kithara/distributed with its data and sharding subdirectories).
- Utilities: Helper modules for logging, safety tensor handling, and others (kithara/utils).
- External Orchestration and Infrastructure:
- Ray Integration: Directories and example files under the “ray” folder (for GPU and TPU setups), along with cluster YAMLs. This shows how the library integrates with distributed cluster orchestration.
- Cloud/TPU Support: Examples and configuration for deploying on TPUs (GCE, GKE, QR configurations in the “ray/TPU” folder).
- Checkpointing: Integration with Orbax for asynchronous, distributed checkpointing.
- Documentation & Examples:
- The “docs” directory that describes setup, API usage, troubleshooting, and practical guides.
- The “examples” directory (including Colab notebooks and scripts) that demonstrates usage—for instance, fine-tuning examples using SFT + LoRA.
- Testing & Performance:
- The “tests” and “perf” directories provide insight into the testing strategy and performance benchmarks for the system.

##### Relationships and Interactions
- The Client/Application Layer (via examples) communicates with the core library (kithara modules).
- The core library components interact among themselves; for example:
- The dataset module feeds data to the trainer.
- The model module is used by the trainer for fine-tuning.
- The distributed module manages multi-device/multi-host training which wraps around the training and model components.
- External orchestration (Ray, TPU/GPU cluster configs) is integrated with the library to manage distributed compute resources and data sharding.
- Checkpointing and logging utilities support both fault tolerance and observability in training execution.

##### Architectural Patterns/Design Principles
- Modular Design: Each major responsibility (dataset, model, trainer, optimizers, distributed operations) is encapsulated in its own package.
- Distributed and Asynchronous Design: Abstracting distributed training, data streaming, and checkpointing to support multi-host and multi-device setups.
- Integration with External Tools: Leveraging Ray for orchestration and HuggingFace compatibility for model interchange.
- Configurability: Use of YAML configuration files (e.g., default.yaml) and PyConfig modules to make behavior customizable.

##### Relevant Technologies and Frameworks
- Google TPUs and GPU support (hardware abstraction layer).
- Ray (for distributed orchestration and data loading via Ray.data).
- Orbax (for async distributed checkpointing).
- HuggingFace (for native model compatibility and conversion utilities).
- Optionally, any cloud orchestration frameworks (GCE, GKE) as indicated by cluster configuration files.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Internal Core Library
    subgraph "Internal Core Library"
        A1["Configuration Module"]:::core
        A2["Dataset Module"]:::core
        A3["Model Module"]:::core
            subgraph "Model Module Details"
                A3a["General Model"]:::core
                A3b["HuggingFace Compatibility"]:::core
                A3c["Kerashub & Checkpoint Compatibility"]:::core
                A3d["Maxtext Specifics"]:::core
            end
        A4["Training/Finetuning Module"]:::core
        A5["Callback Support Module"]:::core
        A6["Optimizers Module"]:::core
        A7["Distributed Training Module"]:::core
            subgraph "Distributed Training Details"
                A7a["Distributed Core"]:::core
                A7b["Data Management"]:::core
                A7c["Sharding"]:::core
                A7d["Model Sharding"]:::core
            end
        A8["Utilities Module"]:::core
    end

    %% External Integrations
    subgraph "External Integrations"
        B1["Ray Integration: GPU"]:::orchestration
        B2["Ray Integration: TPU"]:::orchestration
        B3["Job Submission Script"]:::orchestration
    end

    %% Support & Testing
    subgraph "Support & Testing"
        C1["Documentation"]:::support
        C2["Examples"]:::support
        C3["Testing"]:::support
        C4["Performance Benchmarks"]:::support
    end

    %% Relationships within Core Library
    A1 -->|"supports"| A2
    A1 -->|"supports"| A3
    A1 -->|"supports"| A4
    A1 -->|"supports"| A5
    A1 -->|"supports"| A6
    A1 -->|"supports"| A7
    A1 -->|"supports"| A8

    A2 -->|"feeds"| A4
    A3 -->|"used_by"| A4
    A4 -->|"triggers"| A5
    A7 -->|"coordinates"| A4
    A7 -->|"shards_data"| A2
    A7 -->|"shards_model"| A3

    %% External Integrations with Core
    B1 -->|"orchestrates"| A7
    B2 -->|"orchestrates"| A7
    B3 -->|"orchestrates"| A7

    %% Support & Testing interactions
    C2 -->|"interacts_with"| A4
    C1 -->|"documents"| A1
    C3 -->|"tests"| A4
    C4 -->|"benchmarks"| A4

    %% Click Events for Core Library
    click A1 "https://github.com/ai-hypercomputer/kithara/tree/main/kithara/config"
    click A2 "https://github.com/ai-hypercomputer/kithara/tree/main/kithara/dataset"
    click A3a "https://github.com/ai-hypercomputer/kithara/tree/main/kithara/model"
    click A3b "https://github.com/ai-hypercomputer/kithara/tree/main/kithara/model/hf_compatibility"
    click A3c "https://github.com/ai-hypercomputer/kithara/tree/main/kithara/model/kerashub"
    click A3d "https://github.com/ai-hypercomputer/kithara/tree/main/kithara/model/maxtext"
    click A4 "https://github.com/ai-hypercomputer/kithara/tree/main/kithara/trainer"
    click A5 "https://github.com/ai-hypercomputer/kithara/tree/main/kithara/callbacks"
    click A6 "https://github.com/ai-hypercomputer/kithara/tree/main/kithara/optimizers"
    click A7 "https://github.com/ai-hypercomputer/kithara/tree/main/kithara/distributed"
    click A7b "https://github.com/ai-hypercomputer/kithara/tree/main/kithara/distributed/data"
    click A7c "https://github.com/ai-hypercomputer/kithara/tree/main/kithara/distributed/sharding"
    click A7d "https://github.com/ai-hypercomputer/kithara/tree/main/kithara/distributed/sharding/models"
    click A8 "https://github.com/ai-hypercomputer/kithara/tree/main/kithara/utils"

    %% Click Events for External Integrations
    click B1 "https://github.com/ai-hypercomputer/kithara/tree/main/ray/GPU"
    click B2 "https://github.com/ai-hypercomputer/kithara/tree/main/ray/TPU"
    click B3 "https://github.com/ai-hypercomputer/kithara/blob/main/ray/submit_job.py"

    %% Click Events for Support & Testing
    click C1 "https://github.com/ai-hypercomputer/kithara/tree/main/docs"
    click C2 "https://github.com/ai-hypercomputer/kithara/tree/main/examples"
    click C3 "https://github.com/ai-hypercomputer/kithara/tree/main/tests"
    click C4 "https://github.com/ai-hypercomputer/kithara/tree/main/perf"

    %% Styles
    classDef core fill:#AED6F1,stroke:#1B4F72,stroke-width:2px;
    classDef orchestration fill:#A9DFBF,stroke:#1E8449,stroke-width:2px;
    classDef support fill:#F9E79F,stroke:#B9770E,stroke-width:2px;
```
