# ryanliu112/compute-optimal-tts architecture

> Architecture diagram of the GitHub repository ryanliu112/compute-optimal-tts, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ryanliu112/compute-optimal-tts
- Repository: https://github.com/ryanliu112/compute-optimal-tts

## Overview

##### Project Type and Purpose
- This is a research-oriented full-stack codebase for “Compute-Optimal Test-Time Scaling” (TTS) with large language models (LLMs).
- It’s not a traditional web-app, but it includes:
- A static project page (index.html + static/)
- A Python codebase under src/ that implements:
– Environment wrappers for math problems (envs/MATH)
– Core reasoning pipelines (src/reason)
– A microservice-based LLM serving layer (src/reason/llm_service)
- Bash scripts to serve models on GPUs and to launch experiments.

##### File Structure Insights
- Top-level: README, static website assets, src/, scripts/, requirements.txt.
- src/envs/MATH: custom environment, dataset loaders, grader, and a LaTeX→Sympy parser built with ANTLR.
- src/reason: reasoning modules subdivided into evaluation, guided_search, inference, reranking, plus llm_service for RPC.
- src/scripts: shell scripts to launch LLM service workers (GPU1-4) and to run TTS methods.
- No database; data are JSONL files in envs/MATH/dataset.

##### Key Components
a. CLI Orchestrator
- scripts/run.sh
- Parses user flags (method, LM path, RM path, width, etc.)
- Invokes the Controller in llm_service via RPC (HOST_ADDR:CONTROLLER_PORT).
b. Controller Service (llm_service)
- Receives tasks (prompt generation, batch size, method) from CLI.
- Dispatches requests to Worker services.
c. Worker Services (llm_service/workers)
- Policy Model Worker(s): load LM (e.g., Llama3, Qwen)
- Reward/Value Model Worker(s): load PRMs (e.g., Math-Shepherd, RLHFlow)
- vLLM Worker (optional)
- Each worker communicates over gRPC/Ray RPC to Controller.
d. Reasoning Pipeline (src/reason)
- inference: converts prompts → LM calls → raw text outputs
- reranking: processes multiple generations using value models
- guided_search: tree search for CoT or DVTS methods
- evaluation: applies MATH grader to compare predictions against ground truth
e. Math Environment (src/envs/MATH)
- dataset loader reads JSONL files
- prompt.py constructs task-specific prompts
- grader.py / verify_utils.py grade symbolic output
- latex2sympy submodule parses LaTeX math into Sympy expressions
f. Static Website
- index.html + static/ for project landing page (independent, not in experiment flow)

##### Relationships and Data Flow
- User runs run.sh → CLI Orchestrator
→ Workers query external model weights (HuggingFace cache or local path)
→ Workers return generated sequences → Controller
→ Controller passes text to inference module → guided_search/reranking as configured
→ guided_search or reranking may again call workers for additional model calls
→ Final answers passed to evaluation → dataset loader + grader
→ Results written to output JSONL
- Serving scripts (serve_gpu*.sh) launch Controller + Workers in tmux sessions

##### Architectural Patterns and Principles
- Microservice pattern for model serving (Controller + Workers)
- Modular separation of concerns:
- envs/MATH for task definition
- reason/* for algorithmic logic
- llm_service for scalable inference
- Use of external tools: ANTLR for parsing, Ray/RPC for inter-process communication

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Experiment Orchestration
    subgraph "Experiment Orchestration"
        CLI["CLI Orchestrator\nsrc/scripts/run.sh"]:::cli
        ServeScripts["serve_gpu*.sh scripts"]:::cli
    end

    %% Model Serving
    subgraph "Model Serving"
        Controller["Controller Service\nsrc/reason/llm_service"]:::service
        Policy["Policy Model Worker\nmodel_worker.py"]:::service
        Reward["Reward/Value Model Worker\nreward_model_worker.py"]:::service
        Value["PRM Value Worker\nvalue_model.py"]:::service
        vLLM["vLLM Worker\nvllm_worker.py"]:::service
    end

    %% Reasoning Pipeline
    subgraph "Reasoning Pipeline"
        Inference["Inference Module\nsrc/reason/inference"]:::reason
        Guided["Guided Search Module\ntree.py"]:::reason
        Reranking["Reranking Module\nvote_utils.py"]:::reason
        Evaluation["Evaluation Module\nevaluate.py & evaluator.py"]:::reason
    end

    %% Environment & Evaluation
    subgraph "Environment & Evaluation"
        Dataset[(MATH JSONL datasets)]:::data
        Prompt["Prompt Construction\nprompt.py"]:::env
        Grader["Grader\ngrader.py & verify_utils.py"]:::env
        Parser["LaTeX→Sympy Parser\nlatex2sympy"]:::env
        Results[(Output JSONL)]:::data
    end

    %% Static Website
    Static["Static Website\nindex.html + static/"]:::static

    %% External Dependencies
    Cloud(["HuggingFace Model Hub"]):::external

    %% Connections
    CLI -->|"invoke RPC"| Controller
    ServeScripts -->|launch| Controller
    ServeScripts -->|launch| Policy
    ServeScripts -->|launch| Reward
    ServeScripts -->|launch| Value
    ServeScripts -->|launch| vLLM

    Controller -->|"dispatch"| Policy
    Controller -->|"dispatch"| Reward
    Controller -->|"dispatch"| Value
    Controller -->|"dispatch"| vLLM

    Policy -->|"loads weights"| Cloud
    Reward -->|"loads weights"| Cloud
    Value -->|"loads weights"| Cloud
    vLLM -->|"loads weights"| Cloud

    Controller -->|"generate(prompt)→text"| Inference
    Inference -->|"LM call"| Policy
    Inference -->|"output"| Guided
    Guided -->|"may call"| Policy
    Guided -->|"output"| Reranking
    Reranking -->|"value score"| Reward
    Reranking -->|"ranked outputs"| Evaluation

    Evaluation -->|"load data"| Dataset
    Evaluation -->|"construct prompts"| Prompt
    Prompt -->|"parse latex"| Parser
    Evaluation -->|"grade output"| Grader
    Evaluation -.->|"writes results"| Results

    %% Click Events
    click CLI "https://github.com/ryanliu112/compute-optimal-tts/blob/main/src/scripts/run.sh"
    click Controller "https://github.com/ryanliu112/compute-optimal-tts/tree/main/src/reason/llm_service"
    click Policy "https://github.com/ryanliu112/compute-optimal-tts/blob/main/src/reason/llm_service/workers/model_worker.py"
    click Reward "https://github.com/ryanliu112/compute-optimal-tts/blob/main/src/reason/llm_service/workers/reward_model_worker.py"
    click Value "https://github.com/ryanliu112/compute-optimal-tts/blob/main/src/reason/llm_service/workers/Process_Q_Model/value_model.py"
    click vLLM "https://github.com/ryanliu112/compute-optimal-tts/blob/main/src/reason/llm_service/workers/vllm_worker.py"
    click Inference "https://github.com/ryanliu112/compute-optimal-tts/tree/main/src/reason/inference"
    click Guided "https://github.com/ryanliu112/compute-optimal-tts/blob/main/src/reason/guided_search/tree.py"
    click Reranking "https://github.com/ryanliu112/compute-optimal-tts/blob/main/src/reason/reranking/vote_utils.py"
    click Evaluation "https://github.com/ryanliu112/compute-optimal-tts/blob/main/src/reason/evaluation/evaluate.py"
    click Evaluation "https://github.com/ryanliu112/compute-optimal-tts/blob/main/src/reason/evaluation/evaluator.py"
    click Dataset "https://github.com/ryanliu112/compute-optimal-tts/blob/main/src/envs/MATH/data.py"
    click Prompt "https://github.com/ryanliu112/compute-optimal-tts/blob/main/src/envs/MATH/prompt.py"
    click Grader "https://github.com/ryanliu112/compute-optimal-tts/blob/main/src/envs/MATH/grader.py"
    click Parser "https://github.com/ryanliu112/compute-optimal-tts/tree/main/src/envs/MATH/latex2sympy"
    click Static "https://github.com/ryanliu112/compute-optimal-tts/blob/main/index.html"
    click Static "https://github.com/ryanliu112/compute-optimal-tts/tree/main/static/"

    %% Styles
    classDef cli fill:#ffe4b5,stroke:#333,stroke-width:1px,shape:parallelogram
    classDef service fill:#add8e6,stroke:#333,stroke-width:1px
    classDef reason fill:#98fb98,stroke:#333,stroke-width:1px
    classDef env fill:#ffdead,stroke:#333,stroke-width:1px
    classDef data fill:#e6e6fa,stroke:#333,stroke-width:1px,shape:cylinder
    classDef external fill:#d3d3d3,stroke:#333,stroke-width:1px,shape:cloud
    classDef static fill:#ffd700,stroke:#333,stroke-width:1px
```
