How wenlinyao/hdflow works
Project type and purpose
• This is a research-oriented Python codebase (“HDFlow”) that orchestrates experiments for the paper “HDFlow: Enhancing LLM Complex Problem-Solving with Hybrid Thinking and Dynamic Workflows.”
• It is not a classic web app; instead, it is an orchestration pipeline that:
• Loads/preprocesses mathematical and logic datasets
• Runs “fast thinking” and “slow thinking” passes via LLMs (OpenAI, Anthropic or locally-hosted via vLLM)
• Combines those into “hybrid thinking” trajectories
• Emits results for analysis
File-structure analysis
• data/ → raw and intermediate datasets (CSV, JSON) and preprocessing scripts (split.py, process_data.py)
• scripts/ → ancillary scripts for model conversion and vLLM configuration
• src/ → core Python modules:
• hdflow.py & hdflow_hybrid.py (experiment orchestrators)
• load_datasets.py (data I/O)
• model_api.py (abstracts calls to LLM providers)
• experts.py, math_equivalence.py, utils.py, monitor_memory_usage.py, local_exec.py (utility and helper functions)
• src/prompts/ (text‐prompt templates for different reasoning stages)
Key README insights
• The pipeline is invoked via CLI (python hdflow.py …, python hdflow_hybrid.py …)
• Supports multiple backends: OpenAI, Azure, Anthropic, and self-hosted Llama-3 via vLLM
• Uses standard Python data libraries (pandas, datasets, tqdm)
Main components
A. Data Layer
• Raw Datasets (GameOf24.csv, MetaMathQA JSONs, MathInstruct JSONs)
• Preprocessing Scripts (data/process_data.py, data/split.py)
B. Orchestration Layer (src/)
• Experiment Runner #1: hdflow.py (“fast” & “slow” thinking)
• Experiment Runner #2: hdflow_hybrid.py (“hybrid thinking after”)
• Data Loader: load_datasets.py
• Prompt Manager: src/prompts/ (folders with text templates)
• Utilities: experts.py (LLM chain logic), math_equivalence.py, utils.py, monitor_memory_usage.py, local_exec.py
• Model API Adapter: model_api.py
C. External/Infrastructure Services
• OpenAI/Anthropic API endpoints (cloud)
• Azure OpenAI endpoint (if configured)
• vLLM‐based local Llama‐3 server(s) on custom ports
D. Output Layer
• Solution files / logs (JSON or text results stored by hdflow scripts)
• Training data outputs for hybrid tuning
Relationships and data flow
1. CLI input → calls hdflow.py or hdflow_hybrid.py
2. Runner loads dataset via load_datasets.py
3. For each problem instance:
a. Select prompt(s) from src/prompts/
b. Call model_api.py, which dispatches to cloud APIs or local vLLM
c. experts.py encapsulates reasoning steps (fast vs slow chain)
d. monitor_memory_usage.py tracks resource usage
4. Runner writes intermediate outputs (fast_thinking.json, slow_thinking.json)
5. hdflow_hybrid.py ingests those outputs and merges into hybrid_thinking trajectories
6. Final results persist for evaluation
Architectural patterns and principles
• Modular orchestration: each runner script delegates to specialized modules
• Adapter pattern in model_api.py for pluggable LLM backends
• Separation of concerns: data I/O, prompting, reasoning logic, and resource monitoring are decoupled
• Reusable prompt library under src/prompts