How chenyiqun/mmoa-rag works
Project Type & Purpose
• MMOA-RAG is a specialized full-stack RAG (Retrieval-Augmented Generation) system that implements a multi-module optimization pipeline (Query Rewriter, Retriever, Selector, Generator) via multi-agent reinforcement learning (MAPPO), built on top of the LLaMA-Factory framework.
• Key use cases:
• Warm-start fine-tuning (SFT) of modules
• Real-time retrieval via a microservice
• Joint RL optimization of multiple modules
• End-to-end evaluation of RAG performance
Top-Level File Structure & Architectural Patterns
• LLaMA-Factory/: the core Python package, organized by functionality:
• api/ – HTTP/WebSocket endpoints for chat and control
• chat/ – engine abstractions (HF, vLLM, custom)
• data/ – loaders, preprocessors, collation, template & processor plugins
• model/ – adapters, loaders, quantization & kernel utils
• train/ – workflows & trainers for SFT, PPO (QR, S, G), DPO, KTO, RM
• eval/ – evaluation templates & runner
• webui/ – React-style components + backend interface for a local dashboard
• docker/: prebuilt environments for GPU (CUDA), ROCm, NPU inference/training
• evaluation/: standalone scripts for standard benchmarks (CEval, MMLU, CMLLU)
• examples/: ready-to-run YAML configs for accelerate, deepspeed, QLoRA, full/train/inference
• scripts/ & root Python scripts: data-prep (qr_s_g_sft_data_alpaca.py, get_ppo_data_alpaca.py), evaluation launcher (evaluate_qr_s_g.py), retrieval server (flask_server.py + run_server.sh)
• tests/: unit & e2e coverage across data, model, chat, train, eval
README Insights
• Two-machine setup:
• Machine A runs Retrieval API (Faiss + GPU via flask_server.py) for sub-10ms top-k lookups
• Machine B runs SFT warm-start (run_sft.sh) and MAPPO joint training (run_mappo.sh) using LLaMA-Factory trainers
• Data directories (./data/*) and model checkpoints (./models/*) must be prepared ahead
Key Components & Interactions
A. Data Layer
• Raw datasets (AmbigQA, Alpaca, etc.)
• Generated SFT data (qr_s_g_sft_data_alpaca.py)
• Generated PPO trajectories (get_ppo_data_alpaca.py)
• Checkpoint storage (./models)
B. Retrieval Microservice (Machine A)
• flask_server.py exposes /search endpoint
• Faiss index + GPU inference container (docker/docker-cuda)
• Clients: Query Rewriter during RL rollouts, any inference pipeline
C. Core ML Pipeline (Machine B)
• SFT Trainer
• run_sft.sh → LLaMA-Factory train/sft workflow → produces warm-started QR, S, G models
• MAPPO Trainer
• run_mappo.sh → LLaMA-Factory train/ppo/trainer_qr_s_g.py → orchestrates agents:
– Query Rewriter agent calls Retrieval API
– Selector agent picks top documents
– Generator agent produces output
• Inter-agent communication via in-memory buffers defined in workflow.py
D. Evaluation Module
• evaluate_qr_s_g.py invokes LLaMA-Factory eval/evaluator.py with mappings in evaluation/ceval, mmlu, cmmlu
E. Web UI (optional)
• webui.py (standalone) or LLaMA-Factory/src/llamafactory/webui/runner.py
• Components for chat, train, infer, eval dashboards
Architectural Patterns & Principles
• Modular Microservices: Retrieval separated for real-time performance
• Pipeline Stages: Data-prep → SFT fine-tuning → MAPPO RL → Evaluation
• Plugin-based extensibility: data processors, model utilities, evaluation templates
• Config-driven examples: YAML + shell scripts for reproducibility