How jieyilong/tree-of-thought-puzzle-solver works
Project Type and Purpose
• This is an AI-driven puzzle‐solver toolkit (not a web app).
• It implements a novel Tree-of-Thought orchestration around an LLM (OpenAI GPT) to solve Sudoku puzzles via various “solver types” (zero-shot, one-shot with CoT, few-shot with CoT, ToT).
File Structure
• Top-level runners:
• run_tot.py – CLI entry for the ToT solver
• run_expr.py – CLI entry for batch experiments
• Core orchestration:
• tot/tot.py – the ToT controller that drives multi-round thought tree exploration
• Modular “actors” under actors/:
• prompter.py – builds LLM prompts
• llm.py – wraps the LLM API call (OpenAI)
• parser.py – interprets LLM responses
• checker.py – validates candidate moves/partial solutions
• state.py – represents the current “thought” state / memory
• Common utilities under common/: configuration loader, constants, hyperparameters, enums, misc utils
• Data/experiments:
• data/benchmarks/sudoku – JSON puzzle sets
• experiments/chatbot_based_solvers.py – orchestrates solver variants
• experiments/execution_logs – stores run outputs
• Config file: config.yaml – holds LLM model choice and API key
Extract Additional Insights from README
• The README highlights:
• Four solver modes (zero-shot, one-shot CoT, few-shot CoT, ToT)
• Multi-agent, multi-round interaction with an LLM
• Backtracking support via tree structure of “thoughts”
Main Components
a. CLI Clients
• run_tot.py
• run_expr.py
b. Experiment Manager / Solver Factory
• experiments/chatbot_based_solvers.py
• Chooses solver_type and loads puzzle set from data/benchmarks
c. ToT Controller (tot/tot.py)
• Drives the tree search, backtracking logic, multi-round orchestration
d. Actors (modular components)
• Prompter (prompt builder)
• LLM Wrapper (OpenAI API client)
• Parser (response interpreter)
• Checker (validates/filters moves)
• State (maintains current puzzle state and thought nodes)
e. Data & Config
• Benchmarks (JSON puzzle inputs)
• config.yaml (LLM config)
f. External Service
• OpenAI API (Chat completion endpoint)
g. Logs & Outputs
• experiments/execution_logs
Relationships and Data Flow
• The CLI client invokes the Experiment Manager with a solver type and dataset path.
• Experiment Manager reads puzzle JSON, instantiates the appropriate solver (zero-shot / one-shot / few-shot / ToT).
• For ToT:
• The ToT Controller enters an iterative loop:
– Prompter constructs the next prompt using State.
– LLM Wrapper sends prompt to OpenAI, returns raw response.
– Parser extracts proposed moves/thoughts.
– Checker validates proposals against Sudoku rules.
– State updates the thought tree (accept, branch, backtrack).
– Loop until solution or search limit.
• Solutions are logged to experiments/execution_logs.
Architectural Patterns and Principles
• Modular “actor” pipeline: each responsibility is isolated in its own class.
• Controller pattern: ToT Controller orchestrates actor interactions.
• Data-driven experiments: Experiment Manager loads puzzles and logs results.
• Separation of concerns:
• Prompt engineering (Prompter)
• LLM communication (LLM Wrapper)
• Domain logic (Checker, State)
• Data I/O (Experiment Manager, Parser)
Technologies, Frameworks, Libraries
• Python 3.9+
• OpenAI Python SDK (wrapped by actors/llm.py)
• YAML for config (common/config.py)
• JSON for puzzle datasets
• Optional: logging modules, standard Python libs
Detailed Elements
• Clear labels (e.g., “ToT Controller (tot/tot.py)”)
• Arrowheads to indicate request/response or data updates
• Notes on loop boundaries (max depth, max iterations) near the Controller
• Call out external dependency (OpenAI API) with version/model info (e.g., “gpt-3.5-turbo”)