# castorini/rank_llm architecture

> Architecture diagram of the GitHub repository castorini/rank_llm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/castorini/rank_llm
- Repository: https://github.com/castorini/rank_llm

## Overview

##### Project Type and Purpose
- This is a Python-based ranking toolkit (“rank_llm”) that provides:
- Retrieval of candidate documents (via Pyserini or external services)
- Reranking using large language models (pairwise, pointwise, listwise)
- Evaluation (TREC-style metrics)
- Cost and response analysis
- A lightweight API server for programmatic access
- Demo scripts and training utilities

##### Top-Level Components
- Core Library (`src/rank_llm/…`):
- retrieve/ – abstractions for retrieving candidate docs (Pyserini, service APIs)
- rerank/ – implementations of reranking strategies (identity, pointwise, pairwise, listwise)
- evaluation/ – TREC evaluation wrappers
- analysis/ – cost estimation & response analysis
- api/server.py – RESTful HTTP server exposing retrieval+rering endpoints
- data.py – data-model definitions (topics, indices)
- scripts/ – CLI entrypoints for common workflows
- Demos (`src/rank_llm/demo/…`): jupyter-style scripts showcasing various LLM rerankers
- Training (`training/…`):
- configs/ – Deepspeed & ZeRO3 configs
- scripts/ – shell scripts to launch training on different LLMs
- utils/ – data, model, training helper functions
- Tests (`test/…`): unit tests for each major submodule
- External Dependencies:
- Pyserini index
- OpenAI, Gemini, Qwen, etc. LLM APIs
- TREC datasets

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Layer
    subgraph "User Layer"
        CLI["CLI Tools"]:::user
        Demo["Demo Suite"]:::user
        RESTClient["REST Client"]:::user
    end

    %% API Layer
    subgraph "API Layer"
        APIServer["API Server\n(server.py)"]:::api
    end

    %% Core Processing
    subgraph "Core Processing"
        Retrieval["Retrieval Module"]:::core
        Reranking["Reranking Module"]:::core
        Evaluation["Evaluation Module"]:::core
        Analysis["Analysis Module"]:::core
        RetrieveAndRerank["Retrieve-and-Rerank Script"]:::core
        DataModels["Data Models\n(data.py)"]:::core
        CLIScripts["CLI Entrypoints\n(scripts)"]:::core
    end

    %% Data Stores
    subgraph "Data Stores"
        Index["Pyserini Index"]:::dataStore
        Configs["Config & Templates"]:::dataStore
        Caches["TREC Result Caches"]:::dataStore
    end

    %% External Services
    subgraph "External Services"
        LLMs["LLM APIs\n(OpenAI, Gemini, Qwen)"]:::external
        PyseriniService["Pyserini Service API"]:::external
        Logging["Logging/Monitoring"]:::external
    end

    %% Training Pipeline
    subgraph "Training Pipeline"
        TrainMain["Training Main\n(train_rankllm.py)"]:::train
        TrainConfigs["Training Configurations"]:::train
        TrainScripts["Training Launch Scripts"]:::train
        TrainUtils["Training Utilities"]:::train
    end

    %% Tests
    subgraph "Tests"
        TestRetrieve["Tests - Retrieval"]:::test
        TestRerank["Tests - Reranking"]:::test
        TestEval["Tests - Evaluation"]:::test
        TestAnalysis["Tests - Analysis"]:::test
        TestAPI["Tests - API Server"]:::test
    end

    %% Connections: User to Core/API
    CLI -->|uses| Retrieval
    CLI -->|runs| CLIScripts
    Demo -->|uses| Reranking
    RESTClient -->|calls| APIServer

    %% API to Core
    APIServer -->|calls| Retrieval
    APIServer -->|calls| Reranking
    APIServer -->|calls| Evaluation
    APIServer -->|calls| Analysis

    %% Core flows
    Retrieval -->|returns docs| Reranking
    Reranking -->|produces rankings| Evaluation
    Evaluation -->|computes metrics| Analysis

    DataModels --> Retrieval
    DataModels --> Reranking

    RetrieveAndRerank --> Retrieval
    RetrieveAndRerank --> Reranking

    CLIScripts --> RetrieveAndRerank

    %% Core to Data Stores
    Retrieval --> Index
    Retrieval --> Configs
    Evaluation --> Caches

    %% Core to External
    Retrieval -->|optionally| PyseriniService
    Reranking -->|calls| LLMs
    APIServer -->|sends logs| Logging
    Analysis -->|sends logs| Logging

    %% Training pipeline flows
    TrainMain --> TrainConfigs
    TrainMain --> TrainUtils
    TrainScripts --> TrainMain

    %% Tests to components
    TestRetrieve --> Retrieval
    TestRerank --> Reranking
    TestEval --> Evaluation
    TestAnalysis --> Analysis
    TestAPI --> APIServer

    %% Click Events
    click Retrieval "https://github.com/castorini/rank_llm/tree/main/src/rank_llm/retrieve"
    click Reranking "https://github.com/castorini/rank_llm/tree/main/src/rank_llm/rerank"
    click Evaluation "https://github.com/castorini/rank_llm/blob/main/src/rank_llm/evaluation/trec_eval.py"
    click Analysis "https://github.com/castorini/rank_llm/blob/main/src/rank_llm/analysis/estimate_costs.py"
    click Analysis "https://github.com/castorini/rank_llm/blob/main/src/rank_llm/analysis/response_analysis.py"
    click APIServer "https://github.com/castorini/rank_llm/blob/main/src/rank_llm/api/server.py"
    click DataModels "https://github.com/castorini/rank_llm/blob/main/src/rank_llm/data.py"
    click CLIScripts "https://github.com/castorini/rank_llm/tree/main/src/rank_llm/scripts"
    click Demo "https://github.com/castorini/rank_llm/tree/main/src/rank_llm/demo"
    click TrainConfigs "https://github.com/castorini/rank_llm/tree/main/training/configs"
    click TrainScripts "https://github.com/castorini/rank_llm/tree/main/training/scripts"
    click TrainUtils "https://github.com/castorini/rank_llm/tree/main/training/utils"
    click TrainMain "https://github.com/castorini/rank_llm/blob/main/training/train_rankllm.py"
    click RetrieveAndRerank "https://github.com/castorini/rank_llm/blob/main/src/rank_llm/retrieve_and_rerank.py"
    click TestRetrieve "https://github.com/castorini/rank_llm/tree/main/test/retrieve"
    click TestRerank "https://github.com/castorini/rank_llm/tree/main/test/rerank"
    click TestEval "https://github.com/castorini/rank_llm/tree/main/test/evaluation"
    click TestAnalysis "https://github.com/castorini/rank_llm/tree/main/test/analysis"
    click TestAPI "https://github.com/castorini/rank_llm/blob/main/test/api/test_server.py"

    %% Styles
    classDef user fill:#A8D5BA,stroke:#333,stroke-width:1px
    classDef core fill:#A4C2F4,stroke:#333,stroke-width:1px
    classDef api fill:#A4C2F4,stroke:#333,stroke-width:1px
    classDef dataStore fill:#F4B183,stroke:#333,stroke-width:1px
    classDef external fill:#F4B183,stroke:#333,stroke-width:1px
    classDef train fill:#C9A3E0,stroke:#333,stroke-width:1px
    classDef test fill:#D3D3D3,stroke:#333,stroke-width:1px
```
