# petergriffinjin/search-r1 architecture

> Architecture diagram of the GitHub repository petergriffinjin/search-r1, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/petergriffinjin/search-r1
- Repository: https://github.com/petergriffinjin/search-r1

## Overview

##### Main Components
- Data Processing & Preparation:
– Scripts under “scripts/data_process/” (e.g., nq_search.py, nq_rag.py) process raw datasets (like NQ) and corpora.
– The “example” directory provides sample input files such as corpus.jsonl.
- Retrieval/Search Engine Component:
– Located within “search_r1/search”, this component contains scripts for building indexes (build_index.sh, index_builder.py) as well as the retrieval server (retrieval_server.py, retrieval_request.py).
– The “retrieval_launch.sh” script is used to start a local search engine server.
- LLM Agent and Inference:
– The “search_r1/llm_agent” subdirectory holds code (e.g., generation.py, tensor_helper.py) that likely encapsulates the logic allowing the LLM to interact with the search engine, generating queries and processing responses.
– The top-level “infer.py” facilitates model inference and provides a user interface to test or demonstrate the trained model.
- RL Training Pipeline:
– Training scripts (train_ppo.sh, train_grpo.sh) and training orchestration under “verl/trainer/” implement reinforcement learning (RL) for training the LLM to call the search engine.
– These scripts invoke various training configurations (PPO, SFT, etc.).
- Fundamental Model & Utility Libraries:
– The “verl” directory is extensive and contains:
o Model definitions and implementations (e.g., within “verl/models/llama”, “verl/models/transformers”) that adjust standard LLM architectures.
o Controllers (found in “verl/single_controller”) and worker implementations for distributed or parallel processing.
o Utility modules (within “verl/utils”) covering logging, configuration, performance, and distributed computing.
– Third-party integrations such as vllm (inside “verl/third_party/vllm”) are integrated to optimize model loading and inference.

##### Relationships and Data Flow
- Train Time:
– Data flows from raw input (e.g., datasets and corpora) processed via the data processing scripts.
– The training pipeline leverages processes orchestrated by RL training scripts and the trainer modules in “verl/trainer”.
– During RL training, the LLM (as defined in “verl/models”) interacts with the search engine component (which can be accessed locally via “search_r1/search/retrieval_server.py”) to simulate and optimize tool-calling.
- Inference / Deployment:
– User requests (questions) are input via the “infer.py” script.
– The LLM uses the llm_agent to generate queries and calls the local search service.
– Results flow back to the client, showcased via the inference outputs.

##### Architectural Patterns
- Separation of Concerns:
– Notice the clear separation between data processing, model training (RL), inference, and retrieval system function. Each major functionality is encapsulated in its own directory or module.
- Modular Design:
– Using separate submodules (e.g., “verl/models”, “verl/trainer”, “search_r1”) supports modular and scalable design, making it easier to replace components (such as the retriever) or extend functionality.
- Distributed & Parallel Processing:
– Several modules under “verl/single_controller” and “verl/workers” indicate that the project leverages distributed frameworks (sometimes via Ray) for scalable training and inference.
- Integration of External Services:
– The design explicitly supports launching a standalone search engine (retrieval_server.py) that the LLM interacts with through REST API calls.

## Diagram (Mermaid)

```mermaid
graph TD
    %% Data Preparation Layer
    subgraph "Data Preparation"
        DP1["Data Process Scripts"]:::dataproc
        DP2["Example Input"]:::dataproc
    end

    %% Retrieval/Search Engine Layer
    subgraph "Retrieval Engine"
        RS["Retrieval Server"]:::retrieval
        IB["Index Builder"]:::retrieval
        RR["Retrieval Request"]:::retrieval
        LS["Launch Search Engine"]:::retrieval
    end

    %% LLM Agent & Inference Layer
    subgraph "LLM Agent & Inference"
        LA["LLM Agent"]:::inference
        II["Inference Interface"]:::inference
    end

    %% RL Training Pipeline Layer
    subgraph "RL Training Pipeline"
        TP1["Train PPO"]:::training
        TP2["Train GRPO"]:::training
        TM["Trainer Modules"]:::training
    end

    %% Fundamental Model & Utilities Layer
    subgraph "Fundamental Model & Utilities"
        LM["LLM Models"]:::model
        CW["Controller & Workers"]:::model
        UM["Utility Modules"]:::model
        TPV["Third Party: vllm"]:::model
    end

    %% Data Flow Relationships
    DP1 -- "data_ingestion" --> TM
    TM -- "model_training" --> LM
    TM -- "simulate_query" --> RS
    RS -- "reward_signals" --> TM

    %% Inference Flow
    II -- "user_query" --> LA
    LA -- "generate_query" --> RS
    RS -- "return_results" --> LA
    LA -- "deliver_response" --> II

    %% Internal Fundamental Model & Utilities relationships
    TM -- "orchestrates" --> CW
    TM -- "config_logging" --> UM
    LM -- "optimized_by" --> TPV
    LM -- "supports_inference" --> LA

    %% Click Events for Data Preparation
    click DP1 "https://github.com/petergriffinjin/search-r1/tree/main/scripts/data_process"
    click DP2 "https://github.com/petergriffinjin/search-r1/tree/main/example"

    %% Click Events for Retrieval/Search Engine
    click RS "https://github.com/petergriffinjin/search-r1/blob/main/search_r1/search/retrieval_server.py"
    click IB "https://github.com/petergriffinjin/search-r1/blob/main/search_r1/search/index_builder.py"
    click RR "https://github.com/petergriffinjin/search-r1/blob/main/search_r1/search/retrieval_request.py"
    click LS "https://github.com/petergriffinjin/search-r1/blob/main/retrieval_launch.sh"

    %% Click Events for LLM Agent & Inference
    click LA "https://github.com/petergriffinjin/search-r1/tree/main/search_r1/llm_agent"
    click II "https://github.com/petergriffinjin/search-r1/blob/main/infer.py"

    %% Click Events for RL Training Pipeline
    click TP1 "https://github.com/petergriffinjin/search-r1/blob/main/train_ppo.sh"
    click TP2 "https://github.com/petergriffinjin/search-r1/blob/main/train_grpo.sh"
    click TM "https://github.com/petergriffinjin/search-r1/tree/main/verl/trainer"

    %% Click Events for Fundamental Model & Utilities
    click LM "https://github.com/petergriffinjin/search-r1/tree/main/verl/models"
    click CW "https://github.com/petergriffinjin/search-r1/tree/main/verl/single_controller"
    click UM "https://github.com/petergriffinjin/search-r1/tree/main/verl/utils"
    click TPV "https://github.com/petergriffinjin/search-r1/tree/main/verl/third_party/vllm"

    %% Styles
    classDef dataproc fill:#add8e6,stroke:#333,stroke-width:2px;
    classDef training fill:#ffedba,stroke:#333,stroke-width:2px;
    classDef retrieval fill:#ffcccb,stroke:#333,stroke-width:2px;
    classDef inference fill:#d0f0c0,stroke:#333,stroke-width:2px;
    classDef model fill:#e0e0e0,stroke:#333,stroke-width:2px;
```
