# zymrael/vortex architecture

> Architecture diagram of the GitHub repository zymrael/vortex, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zymrael/vortex
- Repository: https://github.com/zymrael/vortex

## Overview

##### Project’s Main Components
- Inference & Generation Engine:
- Components such as generate.py, generate_transformers.py, and run scripts serve as entry points for model inference and generation.
- They use configurations found in the “configs” directory (which include various pre-trained model settings like Evo 2 40B).
- Core Library Modules:
- “vortex/model”: Includes the model architecture, attention mechanisms, layer definitions, caching, embedding modules, and utility functions.
- “vortex/ops”: Contains the specialized high-performance operators. Under this umbrella you have:
∙ “attn” – custom CUDA kernels for flash attention (with various CUDA source files defining forward and backward passes).
∙ “conv” – modules for convolution operations.
∙ “hyena_se” and “hyena_x” – implementations and tests for Hyena layers.
∙ Other utility interfaces for hardware and kernel management.
- Auxiliary Modules:
- “tokenizer”: A separate tokenizer utility.
- “test”: A comprehensive suite to validate model behavior (forward/backward propagation, numeric tests, etc).
- Build, Configuration, and Deployment Artifacts:
- Dockerfile, Makefile, and setup scripts (setup.py, local_setup.py) help with environment and dependency management (compiling CUDA kernels, setting up custom ops).
- Configuration files in “configs” define model parameters, caching behavior, and layers settings.
- Pre-commit, license, and documentation files help with project maintenance.

##### Relationships and Interactions Between Components
- The entry–point scripts (generate.py, run) load a selected configuration (from the “configs” folder) and then initialize the model.
- The model initialization (in “vortex/model”) utilizes a variety of custom operators imported from “vortex/ops”. For example, when an inference is performed, the model’s attention layers invoke the flash attention kernels available in “vortex/ops/attn”.
- The tokenizer module provides text-to-token encoding that is a prerequisite for model inference.
∙ From “generate.py” → “vortex/model” → “vortex/ops” components.
∙ Test modules in “test” interact with both the model and ops components for validating forward/backward propagation.
- The build and deployment artifacts (like Dockerfile and Makefile) indicate external processes that compile CUDA components and set up the environment.

##### Architectural Patterns and Design Principles
- Modularity: The separation into “model” and “ops” shows a clear separation of concerns – the model architecture is abstracted from the hardware-optimized operations.
- Extensibility: The modular design allows users to import specific kernels or interfaces (e.g., hyena_ops.interface) to integrate with custom multi-hybrid models.
- Performance Optimization: Emphasis on custom CUDA kernels (multiple .cu and C++ files) demonstrate performance-focused design.
- Custom Build Process: Use of Makefile, Docker, and custom setup scripts for environment reproducibility and hardware compatibility.

##### Incorporate Relevant Technologies, Frameworks, and Libraries
- CUDA and C++ for kernel implementations.
- Python (with integration to PyTorch and possibly Transformer Engine as noted in the README) for higher-level operations.
- Docker for containerized deployment.
- Standard testing frameworks for validating numerical performance in “test”.

[Entry Points/CLI] → [Configuration Loader (configs)] → [Inference Engine / Model (vortex/model)].
- Within the [Inference Engine / Model] component, subdivide to detail:
∙ Model components: attention, layers, placement of caching.
∙ Reference to specialized operations as separate modules.
- Another module showing [Ops Library] with subdivisions for:
∙ Flash Attention (vortex/ops/attn) with its CUDA kernel files.
∙ Convolution ops (vortex/ops/conv).
∙ Hyena layers (vortex/ops/hyena_se and vortex/ops/hyena_x).
∙ Data flow from entry scripts to configuration, then to model execution, and finally invoking ops.
∙ Interactions with the tokenizer before data is fed into the model.
- Build and deployment flows as separate blocks showing Docker, Makefile, and setup scripts interacting with the rest of the system.

##### Detail
- Labels and annotations for:
∙ “CLI/Entry Scripts”: generate.py, run, run_generate.
∙ “Model Module”: vortex/model with subcomponents – attention, layers, caching, utilities.
∙ “Ops/Kernels”: vortex/ops (highlighting attn, conv, hyena_se, hyena_x directories).
∙ “Configurations”: configs directory for model-specific parameters.
∙ “Utilities”: tokenizer and test suite.
∙ “Build/Deployment”: Dockerfile, Makefile, setup scripts.
∙ From configuration to model initialization.
∙ From model to ops when invoking CUDA kernels.
∙ From CLI to model for executing inference/generation tasks.
- Optionally, include notations for external dependencies like PyTorch, CUDA libraries, and Transformer Engine, to capture the environment in which Vortex operates.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Entry Points"
        E1("generate.py"):::entry
        E2("generate_transformers.py"):::entry
        E3("run"):::entry
        E4("run_generate"):::entry
    end

    CFG("Configuration Loader (configs)"):::config

    Core("Core Model (vortex/model)"):::core

    subgraph "Ops Library"
        O1("Flash Attention Kernels (vortex/ops/attn)"):::ops
        O2("Convolution Ops (vortex/ops/conv)"):::ops
        O3("Hyena SE Layers (vortex/ops/hyena_se)"):::ops
        O4("Hyena X Layers (vortex/ops/hyena_x)"):::ops
    end

    subgraph "Auxiliary Utilities"
        T1("Tokenizer (tokenizer)"):::utility
        T2("Test Suite (test)"):::utility
    end

    subgraph "Build/Deployment"
        B1("Dockerfile"):::build
        B2("Makefile"):::build
        B3("setup.py"):::build
        B4("local_setup.py"):::build
        B5("pyproject.toml"):::build
        B6("requirements.txt"):::build
    end

    %% Connections from Entry Points to Configuration Loader
    E1 --> CFG
    E2 --> CFG
    E3 --> CFG
    E4 --> CFG

    %% Connection from Configuration Loader to Core Model
    CFG --> Core

    %% Tokenizer input flows into Core Model
    T1 --> Core

    %% Core Model invokes Ops Library components
    Core --> O1
    Core --> O2
    Core --> O3
    Core --> O4

    %% Test suite interacts with Core Model and Ops Library
    T2 --> Core
    T2 --> O1

    %% Build/Deployment artifacts interact with Core Model
    B1 --> Core
    B2 --> Core
    B3 --> Core
    B4 --> Core
    B5 --> Core
    B6 --> Core

    %% Build/Deployment artifacts also support Ops Library (representative connection)
    B1 --> O1
    B2 --> O1
    B3 --> O1
    B4 --> O1
    B5 --> O1
    B6 --> O1

    %% Click Events for Entry Points
    click E1 "https://github.com/zymrael/vortex/blob/main/generate.py"
    click E2 "https://github.com/zymrael/vortex/blob/main/generate_transformers.py"
    click E3 "https://github.com/zymrael/vortex/tree/main/run"
    click E4 "https://github.com/zymrael/vortex/tree/main/run_generate"

    %% Click Event for Configurations
    click CFG "https://github.com/zymrael/vortex/tree/main/configs"

    %% Click Event for Core Model
    click Core "https://github.com/zymrael/vortex/tree/main/vortex/model"

    %% Click Events for Ops Library
    click O1 "https://github.com/zymrael/vortex/tree/main/vortex/ops/attn"
    click O2 "https://github.com/zymrael/vortex/tree/main/vortex/ops/conv"
    click O3 "https://github.com/zymrael/vortex/tree/main/vortex/ops/hyena_se"
    click O4 "https://github.com/zymrael/vortex/tree/main/vortex/ops/hyena_x"

    %% Click Events for Auxiliary Utilities
    click T1 "https://github.com/zymrael/vortex/blob/main/tokenizer/tokenizer.json"
    click T2 "https://github.com/zymrael/vortex/tree/main/test"

    %% Click Events for Build/Deployment Artifacts
    click B1 "https://github.com/zymrael/vortex/tree/main/Dockerfile"
    click B2 "https://github.com/zymrael/vortex/tree/main/Makefile"
    click B3 "https://github.com/zymrael/vortex/blob/main/setup.py"
    click B4 "https://github.com/zymrael/vortex/blob/main/local_setup.py"
    click B5 "https://github.com/zymrael/vortex/blob/main/pyproject.toml"
    click B6 "https://github.com/zymrael/vortex/blob/main/requirements.txt"

    %% Styles
    classDef entry fill:#AED6F1,stroke:#1F618D,stroke-width:2px;
    classDef config fill:#FAD7A0,stroke:#D35400,stroke-width:2px;
    classDef core fill:#85C1E9,stroke:#2874A6,stroke-width:2px;
    classDef ops fill:#A9DFBF,stroke:#27AE60,stroke-width:2px;
    classDef utility fill:#F9E79F,stroke:#B7950B,stroke-width:2px;
    classDef build fill:#F5CBA7,stroke:#AF601A,stroke-width:2px;
```
