# zymrael/savanna architecture

> Architecture diagram of the GitHub repository zymrael/savanna, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zymrael/savanna
- Repository: https://github.com/zymrael/savanna

## Overview

##### Core Components
- Config & Launch Infrastructure:
– Show a “Configuration” module encompassing directories like “configs”, “launcher”, “tuner”, and “tools”. These files and scripts (e.g., SLURM jobs, search_config files, launch scripts) are used to specify model parameters, distributed training settings, checkpoint handling, and hyperparameter tuning.
- Data Management & Preprocessing:
– Represent the “Data Layer” that reads and prepares input (from savanna/data, scripts/preprocess_data.py, and other related files). This module handles tokenization (byte–level for DNA/character data) and dataset splits.
- Distributed Training Engine:
– Illustrate the “Training Runtime” provided by the “savanna” package. This is where the distributed training logic lives (e.g., savanna/distributed, savanna/initialize, savanna/mpu for model parallel and data-parallel integration). Indicate integration with external frameworks:
- DeepSpeed – for ZeRO sharding.
- TransformerEngine – for FP8 operations.
- Aspects borrowed from GPT-NeoX (directory structure and configuration).
- Model & Kernel Modules:
– Create a block for “Model Components” (from savanna/model) that includes submodules for different operators (attention, convolutional multi-hybrid operations, hyena variants, positional embeddings, etc.). Also include a submodule for “Kernel Implementations” (e.g., savanna/kernels and embedded Triton source in savanna/kernels/triton_src) which implements custom convolutions and context–parallel algorithms (including “a2a” and “p2p” techniques).
- Tuning & Experimentation:
– Mark out a “Tuner/Hyperparameter Search” component (from the tuner directory) that interacts with the configuration and distributed training modules to adjust model parameters and optimize performance.
- Testing & Regression:
– Represent “Testing” (from tests and launcher/tests) that performs end-to-end regression and unit testing. This ensures correctness of new changes in architecture and distributed aspects.

##### Relationships and Flow
- Data Flow: Data (raw datasets) is preprocessed (tokenization, sharding) and fed into the training runtime.
- Training Pipeline: The preprocessed data goes into the model (using savanna/model) and custom kernels are called from the kernel modules. Data moves through various distributed ops managed by savanna/distributed and mpu.
- Configuration and Control Flow: The launcher scripts (in launcher/, tools/, and tuner/) read the configuration files, set up the distributed environment (using SLURM scripts and search_configs), and then launch the training job. Transformers such as DeepSpeed and methods from GPT-NeoX are then wired in.
- Custom Kernel Integration: Emphasize that custom Triton kernels are part of the back-end computation flow used in the convolutional operations.

##### Architectural Patterns & Technologies
- Distributed Systems Patterns: Indicate use of master–worker (or orchestrator–agent) patterns in distributed training (with the launcher coordinating via SLURM and Python scripts).

## Diagram (Mermaid)

```mermaid
flowchart TD

%% Configuration & Launch Subgraph
subgraph "Configuration & Launch"
    CFG_dir["configs"]:::config
    CFG_lnch["launcher"]:::config
    CFG_tun["tuner"]:::config
    CFG_tools["tools"]:::config
    CFG_file1["launcher/config_parser.py"]:::config
    CFG_file2["tuner/tuner.py"]:::config
    CFG_file3["tools/slurm_template.py"]:::config
end

%% Data Ingestion & Preprocessing Subgraph
subgraph "Data Ingestion & Preprocessing"
    DATA_dir["savanna/data"]:::data
    DATA_conf["configs/data"]:::data
    DATA_script["scripts/preprocess_data.py"]:::data
    DATA_test["tests/data"]:::data
end

%% Distributed Training Engine Subgraph
subgraph "Distributed Training Engine"
    TRAIN_dist["savanna/distributed.py"]:::engine
    TRAIN_init["savanna/initialize.py"]:::engine
    TRAIN_mpu["savanna/mpu"]:::engine
    TRAIN_lnch["launcher/40b/pretrain"]:::engine
end

%% Model & Custom Operators Subgraph
subgraph "Model & Custom Operators"
    MODEL_mod["savanna/model"]:::model
end

%% Kernel Implementations Subgraph
subgraph "Kernel Implementations"
    KERNEL_dir["savanna/kernels"]:::kernel
    KERNEL_triton["savanna/kernels/triton_src"]:::kernel
end

%% Hyperparameter Tuner Subgraph
subgraph "Hyperparameter Tuner"
    TUNER_file["tuner/tuner.py"]:::tuner
    TUNER_search["tuner/search_configs"]:::tuner
    TUNER_lnch1["launcher/7b-context-ext"]:::tuner
    TUNER_lnch2["launcher/context-scaling"]:::tuner
end

%% Testing & Regression Subgraph
subgraph "Testing & Regression"
    TEST_func["tests/functional"]:::testing
    TEST_unit["tests/unit"]:::testing
    TEST_conf["tests/test_configs"]:::testing
    TEST_lnch["launcher/tests"]:::testing
    TEST_reg["run_regression_test.sh"]:::testing
end

%% External Integrations
DEEPSPEED["DeepSpeed"]:::external
TRANSFORMER["TransformerEngine"]:::external
GPTNEOX["GPT-NeoX"]:::external

%% Connections between major components using representative nodes
CFG_lnch -->|"launches"| DATA_dir
DATA_dir -->|"feeds"| TRAIN_dist
TRAIN_dist -->|"triggers"| MODEL_mod
MODEL_mod -->|"invokes"| KERNEL_dir
MODEL_mod -->|"uses"| KERNEL_triton
TUNER_file -->|"adjusts"| CFG_lnch
TUNER_file -->|"optimizes"| TRAIN_dist
CFG_lnch -->|"orchestrates"| TRAIN_dist

%% External integrations with Distributed Training Engine
TRAIN_dist -->|"integrates"| DEEPSPEED
TRAIN_dist -->|"integrates"| TRANSFORMER
TRAIN_dist -->|"reflects"| GPTNEOX

%% Testing relationships
TEST_func -->|"validates"| CFG_lnch
TEST_func -->|"validates"| TRAIN_dist
TEST_func -->|"validates"| MODEL_mod
TEST_func -->|"validates"| KERNEL_dir

%% Distributed communication within Training Engine
TRAIN_dist -->|"initializes_parallel"| TRAIN_mpu
TRAIN_mpu -->|"gradient_exchange"| TRAIN_mpu

%% Click Events for Configuration & Launch
click CFG_dir "https://github.com/zymrael/savanna/tree/main/configs"
click CFG_lnch "https://github.com/zymrael/savanna/tree/main/launcher"
click CFG_tun "https://github.com/zymrael/savanna/tree/main/tuner"
click CFG_tools "https://github.com/zymrael/savanna/tree/main/tools"
click CFG_file1 "https://github.com/zymrael/savanna/blob/main/launcher/config_parser.py"
click CFG_file2 "https://github.com/zymrael/savanna/blob/main/tuner/tuner.py"
click CFG_file3 "https://github.com/zymrael/savanna/blob/main/tools/slurm_template.py"

%% Click Events for Data Management & Preprocessing
click DATA_dir "https://github.com/zymrael/savanna/tree/main/savanna/data"
click DATA_conf "https://github.com/zymrael/savanna/tree/main/configs/data"
click DATA_script "https://github.com/zymrael/savanna/blob/main/scripts/preprocess_data.py"
click DATA_test "https://github.com/zymrael/savanna/tree/main/tests/data"

%% Click Events for Distributed Training Engine
click TRAIN_dist "https://github.com/zymrael/savanna/blob/main/savanna/distributed.py"
click TRAIN_init "https://github.com/zymrael/savanna/blob/main/savanna/initialize.py"
click TRAIN_mpu "https://github.com/zymrael/savanna/tree/main/savanna/mpu"
click TRAIN_lnch "https://github.com/zymrael/savanna/tree/main/launcher/40b/pretrain"

%% Click Event for Model & Custom Operators
click MODEL_mod "https://github.com/zymrael/savanna/tree/main/savanna/model"

%% Click Events for Kernel Implementations
click KERNEL_dir "https://github.com/zymrael/savanna/tree/main/savanna/kernels"
click KERNEL_triton "https://github.com/zymrael/savanna/tree/main/savanna/kernels/triton_src"

%% Click Events for Hyperparameter Tuner
click TUNER_file "https://github.com/zymrael/savanna/blob/main/tuner/tuner.py"
click TUNER_search "https://github.com/zymrael/savanna/tree/main/tuner/search_configs"
click TUNER_lnch1 "https://github.com/zymrael/savanna/tree/main/launcher/7b-context-ext"
click TUNER_lnch2 "https://github.com/zymrael/savanna/tree/main/launcher/context-scaling"

%% Click Events for Testing & Regression
click TEST_func "https://github.com/zymrael/savanna/tree/main/tests/functional"
click TEST_unit "https://github.com/zymrael/savanna/tree/main/tests/unit"
click TEST_conf "https://github.com/zymrael/savanna/tree/main/tests/test_configs"
click TEST_lnch "https://github.com/zymrael/savanna/tree/main/launcher/tests"
click TEST_reg "https://github.com/zymrael/savanna/blob/main/run_regression_test.sh"

%% Styles
classDef config fill:#ffcccc,stroke:#cc0000,stroke-width:2px;
classDef data fill:#ccffcc,stroke:#009900,stroke-width:2px;
classDef engine fill:#cce5ff,stroke:#004085,stroke-width:2px;
classDef model fill:#fff2cc,stroke:#ff9900,stroke-width:2px;
classDef kernel fill:#d0d0d0,stroke:#808080,stroke-width:2px;
classDef tuner fill:#fdfd96,stroke:#cccc00,stroke-width:2px;
classDef testing fill:#e6e6fa,stroke:#9932cc,stroke-width:2px;
classDef external fill:#ffdab9,stroke:#d2691e,stroke-width:2px;
```
