How ryanliu112/compute-optimal-tts works
Project Type and Purpose
• This is a research-oriented full-stack codebase for “Compute-Optimal Test-Time Scaling” (TTS) with large language models (LLMs).
• It’s not a traditional web-app, but it includes:
• A static project page (index.html + static/)
• A Python codebase under src/ that implements:
– Environment wrappers for math problems (envs/MATH)
– Core reasoning pipelines (src/reason)
– A microservice-based LLM serving layer (src/reason/llm_service)
• Bash scripts to serve models on GPUs and to launch experiments.
File Structure Insights
• Top-level: README, static website assets, src/, scripts/, requirements.txt.
• src/envs/MATH: custom environment, dataset loaders, grader, and a LaTeX→Sympy parser built with ANTLR.
• src/reason: reasoning modules subdivided into evaluation, guided_search, inference, reranking, plus llm_service for RPC.
• src/scripts: shell scripts to launch LLM service workers (GPU1-4) and to run TTS methods.
• No database; data are JSONL files in envs/MATH/dataset.
Key Components
a. CLI Orchestrator
• scripts/run.sh
• Parses user flags (method, LM path, RM path, width, etc.)
• Invokes the Controller in llm_service via RPC (HOST_ADDR:CONTROLLER_PORT).
b. Controller Service (llm_service)
• Receives tasks (prompt generation, batch size, method) from CLI.
• Dispatches requests to Worker services.
c. Worker Services (llm_service/workers)
• Policy Model Worker(s): load LM (e.g., Llama3, Qwen)
• Reward/Value Model Worker(s): load PRMs (e.g., Math-Shepherd, RLHFlow)
• vLLM Worker (optional)
• Each worker communicates over gRPC/Ray RPC to Controller.
d. Reasoning Pipeline (src/reason)
• inference: converts prompts → LM calls → raw text outputs
• reranking: processes multiple generations using value models
• guided_search: tree search for CoT or DVTS methods
• evaluation: applies MATH grader to compare predictions against ground truth
e. Math Environment (src/envs/MATH)
• dataset loader reads JSONL files
• prompt.py constructs task-specific prompts
• grader.py / verify_utils.py grade symbolic output
• latex2sympy submodule parses LaTeX math into Sympy expressions
f. Static Website
• index.html + static/ for project landing page (independent, not in experiment flow)
Relationships and Data Flow
• User runs run.sh → CLI Orchestrator
→ Workers query external model weights (HuggingFace cache or local path)
→ Workers return generated sequences → Controller
→ Controller passes text to inference module → guided_search/reranking as configured
→ guided_search or reranking may again call workers for additional model calls
→ Final answers passed to evaluation → dataset loader + grader
→ Results written to output JSONL
• Serving scripts (serve_gpu*.sh) launch Controller + Workers in tmux sessions
Architectural Patterns and Principles
• Microservice pattern for model serving (Controller + Workers)
• Modular separation of concerns:
• envs/MATH for task definition
• reason/* for algorithmic logic
• llm_service for scalable inference
• Use of external tools: ANTLR for parsing, Ray/RPC for inter-process communication