How ai-hypercomputer/kithara works
Main Components
• Core Library (kithara package): This is the central part of the system and can be subdivided further:
• Configuration: Contains default settings and utilities (kithara/config).
• Dataset Management: Components for data loading, processing, and streaming (kithara/dataset).
• Models: Modules for model construction, inference, conversion, and checkpoint compatibility (kithara/model, and its submodules like hf_compatibility, kerashub, and maxtext).
• Training/Finetuning: Trainer components including callbacks (kithara/trainer and kithara/callbacks).
• Optimizers: Including wrappers for libraries like optax (kithara/optimizers).
• Distributed Training: Abstractions for data sharding and multi-device support (kithara/distributed with its data and sharding subdirectories).
• Utilities: Helper modules for logging, safety tensor handling, and others (kithara/utils).
• External Orchestration and Infrastructure:
• Ray Integration: Directories and example files under the “ray” folder (for GPU and TPU setups), along with cluster YAMLs. This shows how the library integrates with distributed cluster orchestration.
• Cloud/TPU Support: Examples and configuration for deploying on TPUs (GCE, GKE, QR configurations in the “ray/TPU” folder).
• Checkpointing: Integration with Orbax for asynchronous, distributed checkpointing.
• Documentation & Examples:
• The “docs” directory that describes setup, API usage, troubleshooting, and practical guides.
• The “examples” directory (including Colab notebooks and scripts) that demonstrates usage—for instance, fine-tuning examples using SFT + LoRA.
• Testing & Performance:
• The “tests” and “perf” directories provide insight into the testing strategy and performance benchmarks for the system.
Relationships and Interactions
• The Client/Application Layer (via examples) communicates with the core library (kithara modules).
• The core library components interact among themselves; for example:
• The dataset module feeds data to the trainer.
• The model module is used by the trainer for fine-tuning.
• The distributed module manages multi-device/multi-host training which wraps around the training and model components.
• External orchestration (Ray, TPU/GPU cluster configs) is integrated with the library to manage distributed compute resources and data sharding.
• Checkpointing and logging utilities support both fault tolerance and observability in training execution.
Architectural Patterns/Design Principles
• Modular Design: Each major responsibility (dataset, model, trainer, optimizers, distributed operations) is encapsulated in its own package.
• Distributed and Asynchronous Design: Abstracting distributed training, data streaming, and checkpointing to support multi-host and multi-device setups.
• Integration with External Tools: Leveraging Ray for orchestration and HuggingFace compatibility for model interchange.
• Configurability: Use of YAML configuration files (e.g., default.yaml) and PyConfig modules to make behavior customizable.
Relevant Technologies and Frameworks
• Google TPUs and GPU support (hardware abstraction layer).
• Ray (for distributed orchestration and data loading via Ray.data).
• Orbax (for async distributed checkpointing).
• HuggingFace (for native model compatibility and conversion utilities).
• Optionally, any cloud orchestration frameworks (GCE, GKE) as indicated by cluster configuration files.