# chenyiqun/mmoa-rag architecture

> Architecture diagram of the GitHub repository chenyiqun/mmoa-rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/chenyiqun/mmoa-rag
- Repository: https://github.com/chenyiqun/mmoa-rag

## Overview

##### Project Type & Purpose
- MMOA-RAG is a specialized full-stack RAG (Retrieval-Augmented Generation) system that implements a multi-module optimization pipeline (Query Rewriter, Retriever, Selector, Generator) via multi-agent reinforcement learning (MAPPO), built on top of the LLaMA-Factory framework.
- Key use cases:
- Warm-start fine-tuning (SFT) of modules
- Real-time retrieval via a microservice
- Joint RL optimization of multiple modules
- End-to-end evaluation of RAG performance

##### Top-Level File Structure & Architectural Patterns
- LLaMA-Factory/: the core Python package, organized by functionality:
- api/ – HTTP/WebSocket endpoints for chat and control
- chat/ – engine abstractions (HF, vLLM, custom)
- data/ – loaders, preprocessors, collation, template & processor plugins
- model/ – adapters, loaders, quantization & kernel utils
- train/ – workflows & trainers for SFT, PPO (QR, S, G), DPO, KTO, RM
- eval/ – evaluation templates & runner
- webui/ – React-style components + backend interface for a local dashboard
- docker/: prebuilt environments for GPU (CUDA), ROCm, NPU inference/training
- evaluation/: standalone scripts for standard benchmarks (CEval, MMLU, CMLLU)
- examples/: ready-to-run YAML configs for accelerate, deepspeed, QLoRA, full/train/inference
- scripts/ & root Python scripts: data-prep (qr_s_g_sft_data_alpaca.py, get_ppo_data_alpaca.py), evaluation launcher (evaluate_qr_s_g.py), retrieval server (flask_server.py + run_server.sh)
- tests/: unit & e2e coverage across data, model, chat, train, eval

##### README Insights
- Two-machine setup:
- Machine A runs Retrieval API (Faiss + GPU via flask_server.py) for sub-10ms top-k lookups
- Machine B runs SFT warm-start (run_sft.sh) and MAPPO joint training (run_mappo.sh) using LLaMA-Factory trainers
- Data directories (`./data/*`) and model checkpoints (`./models/*`) must be prepared ahead

##### Key Components & Interactions
A. Data Layer
- Raw datasets (AmbigQA, Alpaca, etc.)
- Generated SFT data (qr_s_g_sft_data_alpaca.py)
- Generated PPO trajectories (get_ppo_data_alpaca.py)
- Checkpoint storage (`./models`)
B. Retrieval Microservice (Machine A)
- flask_server.py exposes /search endpoint
- Faiss index + GPU inference container (docker/docker-cuda)
- Clients: Query Rewriter during RL rollouts, any inference pipeline
C. Core ML Pipeline (Machine B)
- SFT Trainer
- run_sft.sh → LLaMA-Factory train/sft workflow → produces warm-started QR, S, G models
- MAPPO Trainer
- run_mappo.sh → LLaMA-Factory train/ppo/trainer_qr_s_g.py → orchestrates agents:
– Query Rewriter agent calls Retrieval API
– Selector agent picks top documents
– Generator agent produces output
- Inter-agent communication via in-memory buffers defined in workflow.py
D. Evaluation Module
- evaluate_qr_s_g.py invokes LLaMA-Factory eval/evaluator.py with mappings in evaluation/ceval, mmlu, cmmlu
E. Web UI (optional)
- webui.py (standalone) or LLaMA-Factory/src/llamafactory/webui/runner.py
- Components for chat, train, infer, eval dashboards

##### Architectural Patterns & Principles
- Modular Microservices: Retrieval separated for real-time performance
- Pipeline Stages: Data-prep → SFT fine-tuning → MAPPO RL → Evaluation
- Plugin-based extensibility: data processors, model utilities, evaluation templates
- Config-driven examples: YAML + shell scripts for reproducibility

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Machine A: Retrieval Host
    subgraph "Machine A: Retrieval Host"
        direction TB
        FS["Flask Server"]:::service
        RS["run_server.sh"]:::service
        DC["Docker GPU Env"]:::service
        FS -->|starts with| RS
        RS -->|uses| DC
    end

    %% Machine B: Training Host
    subgraph "Machine B: Training Host"
        direction TB

        subgraph "Data Layer"
            direction TB
            RAW["Raw Datasets"]:::data
            SFTDATA["SFT Data Generation"]:::data
            PPODATA["PPO Trajectories"]:::data
            CHECK["Model Checkpoints"]:::data
        end

        subgraph "Core ML Pipeline"
            direction TB
            SFTSH["run_sft.sh"]:::core
            SFTW["workflow.py (SFT)"]:::core
            SFTT["trainer.py (SFT)"]:::core
            MAPPOSH["run_mappo.sh"]:::core
            MAPPOW["workflow.py (PPO)"]:::core
            QRTRAIN["trainer_qr_s_g.py"]:::core
            PPOUTIL["ppo_utils.py"]:::core
        end

        subgraph "Evaluation Module"
            direction TB
            EVALLA["evaluate_qr_s_g.py"]:::core
            EVALCORE["evaluator.py"]:::core
            CEVAL["ceval Templates"]:::data
            MMLU["mmlu Templates"]:::data
            CMLLU["cmmlu Templates"]:::data
        end
    end

    %% Optional Web UI
    subgraph "Web UI (Optional)"
        direction TB
        WEBUI["webui.py"]:::ui
        RUNNER["runner.py"]:::ui
        COMPONENTS["components/"]:::ui
        APIUI["api Endpoints"]:::ui
    end

    %% Data Flow Arrows
    RAW -->|feed into| SFTDATA
    SFTDATA -->|train data→| SFTSH
    PPODATA -->|trajectories→| MAPPOSH
    SFTSH -->|calls| SFTW
    SFTW -->|invokes| SFTT
    MAPPOSH -->|calls| MAPPOW
    MAPPOW -->|orchestrates agents| QRTRAIN
    MAPPOW -->|utilizes| PPOUTIL
    QRTRAIN -->|HTTP query→| FS
    FS -->|returns top-k docs→| QRTRAIN
    QRTRAIN -->|docs→| MAPPOW
    EVALLA -->|invokes| EVALCORE
    EVALCORE -->|uses| CEVAL
    EVALCORE -->|uses| MMLU
    EVALCORE -->|uses| CMLLU

    %% Click Events
    click SFTDATA "https://github.com/chenyiqun/mmoa-rag/blob/main/qr_s_g_sft_data_alpaca.py"
    click PPODATA "https://github.com/chenyiqun/mmoa-rag/blob/main/get_ppo_data_alpaca.py"
    click RAW "https://github.com/chenyiqun/mmoa-rag/tree/main/./data/"
    click CHECK "https://github.com/chenyiqun/mmoa-rag/tree/main/./models/"
    click FS "https://github.com/chenyiqun/mmoa-rag/blob/main/flask_server.py"
    click RS "https://github.com/chenyiqun/mmoa-rag/blob/main/run_server.sh"
    click DC "https://github.com/chenyiqun/mmoa-rag/tree/main/LLaMA-Factory/docker/docker-cuda/"
    click SFTSH "https://github.com/chenyiqun/mmoa-rag/blob/main/run_sft.sh"
    click SFTW "https://github.com/chenyiqun/mmoa-rag/blob/main/LLaMA-Factory/src/llamafactory/train/sft/workflow.py"
    click SFTT "https://github.com/chenyiqun/mmoa-rag/blob/main/LLaMA-Factory/src/llamafactory/train/sft/trainer.py"
    click MAPPOSH "https://github.com/chenyiqun/mmoa-rag/blob/main/run_mappo.sh"
    click MAPPOW "https://github.com/chenyiqun/mmoa-rag/blob/main/LLaMA-Factory/src/llamafactory/train/ppo/workflow.py"
    click QRTRAIN "https://github.com/chenyiqun/mmoa-rag/blob/main/LLaMA-Factory/src/llamafactory/train/ppo/trainer_qr_s_g.py"
    click PPOUTIL "https://github.com/chenyiqun/mmoa-rag/blob/main/LLaMA-Factory/src/llamafactory/train/ppo/ppo_utils.py"
    click EVALLA "https://github.com/chenyiqun/mmoa-rag/blob/main/evaluate_qr_s_g.py"
    click EVALCORE "https://github.com/chenyiqun/mmoa-rag/blob/main/LLaMA-Factory/src/llamafactory/eval/evaluator.py"
    click CEVAL "https://github.com/chenyiqun/mmoa-rag/tree/main/LLaMA-Factory/evaluation/ceval/"
    click MMLU "https://github.com/chenyiqun/mmoa-rag/tree/main/LLaMA-Factory/evaluation/mmlu/"
    click CMLLU "https://github.com/chenyiqun/mmoa-rag/tree/main/LLaMA-Factory/evaluation/cmmlu/"
    click WEBUI "https://github.com/chenyiqun/mmoa-rag/blob/main/webui.py"
    click RUNNER "https://github.com/chenyiqun/mmoa-rag/blob/main/LLaMA-Factory/src/llamafactory/webui/runner.py"
    click COMPONENTS "https://github.com/chenyiqun/mmoa-rag/tree/main/LLaMA-Factory/src/llamafactory/webui/components/"
    click APIUI "https://github.com/chenyiqun/mmoa-rag/tree/main/LLaMA-Factory/src/llamafactory/api/"

    %% Styles
    classDef service fill:#ADD8E6,stroke:#000;
    classDef core fill:#90EE90,stroke:#000;
    classDef data fill:#FFA500,stroke:#000;
    classDef ui fill:#DDA0DD,stroke:#000;
```
