# contextualai/halos architecture

> Architecture diagram of the GitHub repository contextualai/halos, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/contextualai/halos
- Repository: https://github.com/contextualai/halos

## Overview

##### Project type and purpose
- This is an Open-Source, modular ML training framework (“HALOs”) for aligning large language models (LLMs) with various human‐centered loss functions (DPO, KTO, PPO, etc.).
- It provides:
- CLI entry points (launch.py, label.py)
- Hydra‐based config management (`config/`, `accelerate_config/`)
- Data loading, training, sampling, and logging modules
- Multi‐GPU/FSDP orchestration via Accelerate
- Easy extensibility for new losses, dataloaders, trainers, and models

##### File structure
– Top‐level:
- launch.py & label.py: main CLI entrypoints
- install.sh, scripts/: environment setup and multi‐node launch scripts
- accelerate_config/: ready‐to‐use Hugging Face Accelerate FSDP configs
- config/: Hydra configs for losses, models, global settings
- examples/: sample feedback data formats
- train/: core library modules (dataloader, models, trainers, sampling, utils)
– Patterns & architectural choices:
- Plugin pattern: new loss = yaml + trainer subclass + dataloader
- Hydra for hierarchical config composition
- Accelerate + FSDP for distributed training
- Separation of concerns: CLI, config, data, training, evaluation

##### README for additional insights
- Core stacks: Python, PyTorch, Hydra, Accelerate, FSDP, Wandb (for logging), Hugging Face Hub
- Execution flow:
1. User runs `accelerate launch ... launch.py loss=X model=Y datasets=[...] ...`
2. Hydra instantiates the config hierarchy (loss, model, trainer settings)
3. Modules in train/ are wired together: dataloader → model → trainer → sampler/label
4. Accelerate wraps the training loop to spawn processes, handle inter‐GPU comms via NCCL
5. Checkpoints & metrics saved to disk and logged to Wandb

a. Main components
- User CLI: launch.py, label.py
- Config Layer: Hydra (`config/*.yaml`), Accelerate (`accelerate_config/*.yaml`)
- Data Layer: Dataloaders in train/dataloader.py + examples/
- Model Layer: train/models.py (LLM wrappers, PEFT/LoRA)
- Training Layer: train/trainers.py, train/sample.py (trainer subclasses, sampling logic)
- Orchestration: Accelerate & FSDP (multi‐GPU, multi‐node management)
- Logging & Checkpointing: Wandb, local disk (cache_dir)
- External Services: Hugging Face Hub (model & dataset), Wandb, openAI API (for evaluation)

b. Relationships and interactions
1. User issues CLI command → invokes `launch.py`
2. `launch.py` uses Hydra to parse & merge configs (global + loss + model + datasets)
3. Hydra config drives instantiation of:
– DataLoader (e.g., UnpairedPreferenceDataLoader)
– Trainer (e.g., KTOTrainer, DPOTrainer)
– Model (LLaMA, Mistral, Gemma, etc.)
4. Accelerate intercepts the Python entrypoint to:
– Spawn N processes across GPUs/nodes
– Wrap the Trainer’s `train()` loop in FSDP for model parallelism
– Handle distributed backend (NCCL) sync & checkpoint broadcasting
5. Trainer calls DataLoader → yields batches → forward pass through Model
6. Loss computed by Trainer → backward pass → FSDP optimizer step
7. Metrics & checkpoints logged to Wandb & disk
8. Optional: `label.py` uses trained model to generate feedback datasets

c. Architectural patterns & design principles
– Modular plugin architecture: new losses/models/dataloaders as independent units
– Separation of concerns: config vs code vs orchestration vs data
– Declarative config (Hydra) driving imperative code
– Distributed training abstraction (Accelerate + FSDP)
– Extensibility: add yaml + subclass to plug into existing pipeline

d. Key technologies & frameworks
– Python 3, PyTorch
– Hydra for config management
– Hugging Face Accelerate for distributed training & FSDP
– Wandb for experiment tracking
– Hugging Face Hub for model/dataset I/O
– Optional: FlashAttention, PEFT/LoRA

##### Final advice
– Be as detailed as possible: show every plugin boundary and integration point
– Don’t over‐compress: capture all Hydra config layers (global, loss, model)
– Show both control flow (CLI → Trainer) and data flow (batches → model → gradients)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interface
    subgraph "User Interface"
        CLI["launch.py"]:::code
        LabelCLI["label.py"]:::code
    end

    %% Config Layer
    subgraph "Config Layer"
        HydraGlobal["config/config.yaml"]:::config
        HydraLoss["config/loss/*.yaml"]:::config
        HydraModel["config/model/*.yaml"]:::config
        HydraTemplate["config/template.jinja"]:::config
        AccelerateConfig["accelerate_config/*.yaml"]:::config
    end

    %% Data Layer
    subgraph "Data Layer"
        DataLoader["train/dataloader.py"]:::code
        Example1["examples/binary_feedback.json"]:::storage
        Example2["examples/pairwise_feedback.json"]:::storage
    end

    %% Model Layer
    subgraph "Model Layer"
        Models["train/models.py"]:::code
    end

    %% Training Layer
    subgraph "Training Layer"
        Trainers["train/trainers.py"]:::code
        Sample["train/sample.py"]:::code
    end

    %% Orchestration Layer
    subgraph "Orchestration Layer"
        Accelerate["Accelerate & FSDP"]:::orchestration
        Scripts["scripts/*.sh"]:::code
    end

    %% Utilities & Setup
    subgraph "Utilities & Setup"
        Utils["train/utils.py"]:::code
        Install["install.sh"]:::code
    end

    %% External Services
    subgraph "External Services"
        HFHub["Hugging Face Hub"]:::external
        Wandb["Wandb"]:::external
        OpenAI["OpenAI API"]:::external
    end

    %% Hardware & Storage
    subgraph "Infrastructure"
        Disk["Local Disk / cache_dir"]:::storage
        GPUs["GPUs"]:::hardware
    end

    %% Relationships
    CLI -->|"invokes"| HydraGlobal
    CLI -->|"invokes"| HydraLoss
    CLI -->|"invokes"| HydraModel
    CLI -->|"invokes"| HydraTemplate
    CLI -->|"invokes"| AccelerateConfig

    HydraGlobal -->|"instantiates"| DataLoader
    HydraLoss -->|"instantiates"| Trainers
    HydraModel -->|"instantiates"| Models
    AccelerateConfig -->|"configures"| Accelerate

    HFHub -->|"sources datasets/models"| DataLoader
    DataLoader -->|"yields batches"| Trainers
    Models -->|"wrapped by"| Trainers
    Trainers -->|"uses sampling"| Sample
    Trainers -->|"wrapped by"| Accelerate
    Accelerate -->|"spawns processes on"| GPUs

    Trainers -->|"logs metrics"| Wandb
    Trainers -->|"saves checkpoints"| Disk

    Models -->|"push/pull models"| HFHub

    LabelCLI -->|"loads model"| Models

    %% Click Events
    click CLI "https://github.com/contextualai/halos/blob/main/launch.py"
    click LabelCLI "https://github.com/contextualai/halos/blob/main/label.py"
    click HydraGlobal "https://github.com/contextualai/halos/blob/main/config/config.yaml"
    click HydraLoss "https://github.com/contextualai/halos/blob/main/config/loss/*.yaml"
    click HydraModel "https://github.com/contextualai/halos/blob/main/config/model/*.yaml"
    click HydraTemplate "https://github.com/contextualai/halos/blob/main/config/template.jinja"
    click AccelerateConfig "https://github.com/contextualai/halos/blob/main/accelerate_config/*.yaml"
    click DataLoader "https://github.com/contextualai/halos/blob/main/train/dataloader.py"
    click Example1 "https://github.com/contextualai/halos/blob/main/examples/binary_feedback.json"
    click Example2 "https://github.com/contextualai/halos/blob/main/examples/pairwise_feedback.json"
    click Models "https://github.com/contextualai/halos/blob/main/train/models.py"
    click Trainers "https://github.com/contextualai/halos/blob/main/train/trainers.py"
    click Sample "https://github.com/contextualai/halos/blob/main/train/sample.py"
    click Scripts "https://github.com/contextualai/halos/blob/main/scripts/*.sh"
    click Utils "https://github.com/contextualai/halos/blob/main/train/utils.py"
    click Install "https://github.com/contextualai/halos/blob/main/install.sh"

    %% Styles
    classDef code fill:#D0E8FF,stroke:#004E89,color:#000;
    classDef config fill:#E2F0CB,stroke:#558B2F,color:#000;
    classDef external fill:#FFE082,stroke:#E65100,color:#000;
    classDef orchestration fill:#FFF9C4,stroke:#F9A825,color:#000;
    classDef storage fill:#FFCDD2,stroke:#C62828,color:#000;
    classDef hardware fill:#ECEFF1,stroke:#607D8B,color:#000;
```
