# modalminds/mm-eureka architecture

> Architecture diagram of the GitHub repository modalminds/mm-eureka, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/modalminds/mm-eureka
- Repository: https://github.com/modalminds/mm-eureka

## Overview

##### Main Components
a. Repository/Infrastructure:
- • • • • •
- GitHub workflows, Dockerfiles, and examples indicate that the project supports CI/CD and containerized deployment as well as running training scripts on both single-node and multi-node setups.

b. Project Core Library (openrlhf Package):
- CLI Tools: Located in openrlhf/cli, these provide entry points for interactive chat, batch inference, and various training commands (e.g., train_sft, train_ppo, train_dpo, etc.).
- Datasets: Under openrlhf/datasets, there are processes to prepare, filter, and structure multiple types of data (reward datasets, SFT datasets, unpaired preference datasets, etc.) used for training.
- Models: The openrlhf/models directory contains model definitions, including the base model and specialized modules (under lmm_kits) for both the common functionality and specific adaptations (like Qwen2.5-VL). These are tightly coupled with processing and patching functions.
- Trainers: In openrlhf/trainer (and its subdirectories, including the ray-based ones), multiple trainers are implemented for different RL algorithms (PPO, DPO, KD, KTO, PRM, etc.), along with utilities such as experience makers, KL controllers, and distributed training utilities.
- Utilities: The openrlhf/utils directory includes helper functions related to logging, distributed training, deepspeed integration, and general-purpose processes.

c. External Interfaces & Deployment:
- Docker: The dockerfile directory and related shell scripts indicate containerized environments for running training/inference.
- Examples: Located in the examples folder, these scripts (for example, training on single vs. multi-node setups) act as practical usage guides and entry points.

##### Relationships and Interactions
- The data flow starts with the Dataset component: data preprocessing and filtering, passing validated data to the trainers.
- The Model component receives processed data, either for fine-tuning (SFT) or RL-based training. The trainers (e.g., PPO, DPO, etc.) are the orchestration layer where the actual reinforcement learning algorithms are applied using these models.
- CLI tools serve as the interface for users to trigger training, inference, or batch processing. They interact with the underlying trainers and datasets.
- Distributed training utilities (especially under the ray modules) integrate with external frameworks (e.g., vLLM) to handle scalable, multi-node training.
- Docker and CI/CD (GitHub workflows) wrap around the entire system to ensure repeatable environments and integration pipelines.

##### Architectural Patterns and Principles
- Modular Separation of Concerns: Components are clearly segregated into CLI, dataset processing, model definitions, and training algorithms.
- Layered Architecture: Data flows from raw input (datasets) → processing → learning (models and trainers) → external interfaces (CLI and deployment scripts).
- Distributed and Scalable Design: The separation of trainer modules (with specialized submodules for distributed training like the ray directories and vllm workers) highlights scalability through parallelism.
- Reinforcement Learning Focus: Specific design choices (e.g., omitting KL divergence, online data filtering, specialized trainers) reflect a pattern tuned towards RL for multimodal reasoning.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Infrastructure"
        A1["CI/CD & Deployment Scripts"] 
        subgraph "Docker"
            D1["Dockerfile"]
            D2["Entrypoint Script"]
        end
    end

    subgraph "Core Library (openrlhf)"
        B1["CLI Tools"]
        B2["Datasets"]
        B3["Models & LMM Kits"]
        B4["Trainers (RL Algorithms)"]
        B5["Utilities"]
    end

    subgraph "External Interfaces"
        C1["Example Scripts"]
    end

    %% Data flow connections
    B2 -->|"processedData"| B4
    B3 -->|"modelUpdates"| B4
    B1 -->|"triggerTraining"| B4
    B5 -->|"support"| B4

    %% Infrastructure interactions
    A1 ---|"integrates"| B1
    D1 ---|"containerizes"| B4
    D2 ---|"supportsContainerEntry"| B4
    C1 -->|"initiateWorkflow"| B4

    %% Styles for color coding
    style A1 fill:#f9c,stroke:#333,stroke-width:2px
    style D1 fill:#f9c,stroke:#333,stroke-width:2px
    style D2 fill:#f9c,stroke:#333,stroke-width:2px
    style B1 fill:#bbf,stroke:#333,stroke-width:2px
    style B2 fill:#bbf,stroke:#333,stroke-width:2px
    style B3 fill:#bbf,stroke:#333,stroke-width:2px
    style B4 fill:#bbf,stroke:#333,stroke-width:2px
    style B5 fill:#bbf,stroke:#333,stroke-width:2px
    style C1 fill:#fd9,stroke:#333,stroke-width:2px

    %% Click Events mapping
    click A1 "https://github.com/modalminds/mm-eureka/blob/MM-EUREKA-Qwen/.github/workflows/python-package.yml"
    click D1 "https://github.com/modalminds/mm-eureka/tree/MM-EUREKA-Qwen/dockerfile/Dockerfile"
    click D2 "https://github.com/modalminds/mm-eureka/blob/MM-EUREKA-Qwen/dockerfile/docker-entrypoint.sh"
    click B1 "https://github.com/modalminds/mm-eureka/tree/MM-EUREKA-Qwen/openrlhf/cli"
    click B2 "https://github.com/modalminds/mm-eureka/tree/MM-EUREKA-Qwen/openrlhf/datasets"
    click B3 "https://github.com/modalminds/mm-eureka/tree/MM-EUREKA-Qwen/openrlhf/models"
    click B4 "https://github.com/modalminds/mm-eureka/tree/MM-EUREKA-Qwen/openrlhf/trainer"
    click B5 "https://github.com/modalminds/mm-eureka/tree/MM-EUREKA-Qwen/openrlhf/utils"
    click C1 "https://github.com/modalminds/mm-eureka/tree/MM-EUREKA-Qwen/examples/scripts"
```
