How lightvector/katago works
Purpose and entry points
KataGo is a C++ Go engine using neural-network-guided Monte Carlo graph search, plus Python tooling for training and self-play. The executable starts at cpp/main.cpp, dispatching subcommands through cpp/command/commandline.cpp. Primary operational interfaces are GTP (cpp/command/gtp.cpp), JSON line-based analysis (cpp/command/analysis.cpp), benchmarking, matches, and self-play.
Game-state foundation
Rules, board mutation, legality, scoring state, and repetition/history semantics form the deterministic game layer consumed by search and protocols. Board models positions; BoardHistory carries game progression and rules-dependent outcome context. Graph-position identity supports transposition-aware search.
Paths: cpp/game/board.cpp, cpp/game/boardhistory.cpp, cpp/game/rules.cpp, cpp/game/graphhash.cpp.
Search and decision pipeline
The search subsystem owns the Monte Carlo graph: nodes, transposition tables, PUCT selection, expansion, backup, time controls, and multithreaded execution. It requests neural evaluations for leaf positions, caches them, then exposes move/analysis results to callers. This is the core boundary between Go semantics and model inference.
Paths: cpp/search/search.cpp, cpp/search/searchnode.cpp, cpp/search/searchpuct.cpp, cpp/search/evalcache.cpp, cpp/search/asyncbot.cpp.
Neural inference abstraction and backends
A backend-neutral evaluator batches encoded board positions, executes the loaded model, and returns policy/value/ownership-style outputs to search. Model loading/version interpretation is separated from accelerator implementation. Supported material runtimes are Eigen CPU, OpenCL, CUDA/cuDNN, TensorRT, and macOS Metal; CUDA and OpenCL include native kernels/helpers.
Paths: cpp/neuralnet/nneval.cpp, cpp/neuralnet/nninputs.cpp, cpp/neuralnet/cudabackend.cpp, cpp/neuralnet/openclbackend.cpp, cpp/neuralnet/trtbackend.cpp, cpp/neuralnet/metalbackend.swift.
Protocol and command surfaces
Command handlers assemble configuration, model evaluators, search bots, and output adapters. GTP is the interactive text-protocol boundary for Go GUIs. The analysis command is a batched JSON service boundary intended for external backend applications. Match, self-play, contribution, and gatekeeper commands reuse the engine for training operations.
Paths: cpp/command/gtp.cpp, cpp/command/analysis.cpp, cpp/command/selfplay.cpp, cpp/command/match.cpp, cpp/command/gatekeeper.cpp.
Training-data and distributed self-play flow
Self-play runs games through the same search/inference stack and serializes positions/training targets; SGF and model/data file handling are separate I/O boundaries. The optional distributed client obtains and executes remote contribution tasks over HTTP. Training configurations encode self-play and gatekeeper job parameters.
Paths: cpp/program/selfplaymanager.cpp, cpp/dataio/trainingwrite.cpp, cpp/dataio/loadmodel.cpp, cpp/distributed/client.cpp, cpp/configs/training/selfplay8mainb18.cfg.
Python training and model lifecycle
Python consumes generated training data, defines PyTorch model architectures, runs optimization/metrics, and exports models for C++ inference. Shell loops coordinate shuffle, train, export, and distributed self-play stages; the public training ecosystem is associated with katagotraining.org.
Paths: python/train.py, python/katago/train/data_processing_pytorch.py, python/katago/train/model_pytorch.py, python/export_model_pytorch.py, python/selfplay/synchronous_loop.sh.
Build and embedded dependencies
CMake builds the native engine and selects platform/backend capabilities. The repository vendors inference-adjacent and portability dependencies, notably CLBlast OpenCL kernels, cuDNN frontend headers, ONNX definitions, HTTP client support, and CoreML conversion code.
Paths: cpp/CMakeLists.txt, cpp/external/clblast, cpp/external/cudnn-frontend, cpp/external/onnx, cpp/external/katagocoreml.
Main entry points: search.cpp Monte Carlo graph search · nneval.cpp Neural evaluator · selfplaymanager.cpp Self-play manager · commandline.cpp Command dispatcher
Components 23
C++ Engine
- Executable entry
cpp/main.cpp - Command dispatcher
cpp/command/commandline.cpp - GTP and analysis · Interactive GTP and JSON-lines analysis interfaces.
cpp/command/gtp.cpp - Training commands · Self-play, match, and gatekeeper operational modes.
cpp/command/selfplay.cpp - Board state · Board mutation, legality, and scoring state.
cpp/game/board.cpp - Game history and rules · Rules-dependent progression, outcomes, and repetition semantics.
cpp/game/boardhistory.cpp - Position identity · Transposition-aware position identity.
cpp/game/graphhash.cpp - Async search bot · Search-facing interface used by commands.
cpp/search/asyncbot.cpp - Monte Carlo graph search · Multithreaded PUCT selection, expansion, backup, and time control.
cpp/search/search.cpp - Search nodes · Nodes and transposition-aware graph state.
cpp/search/searchnode.cpp - Evaluation cache
cpp/search/evalcache.cpp
Neural Inference
- Neural evaluator · Batches encoded positions and returns policy, value, and ownership outputs.
cpp/neuralnet/nneval.cpp - NN input encoding
cpp/neuralnet/nninputs.cpp - Accelerator backends · Eigen, CUDA, OpenCL, TensorRT, and Metal execution backends.
cpp/neuralnet/cudabackend.cpp - Model loading · Loads and interprets exported neural models.
cpp/dataio/loadmodel.cpp
Self-Play & Training
- Self-play manager · Runs self-play games through the engine stack.
cpp/program/selfplaymanager.cpp - Training data writer · Writes positions and training targets.
cpp/dataio/trainingwrite.cpp - Distributed client · Fetches and executes remote contribution tasks.
cpp/distributed/client.cpp - PyTorch training · Consumes self-play data and trains neural architectures.
python/train.py - Model export · Exports trained PyTorch models for C++ inference.
python/export_model_pytorch.py - Training loop · Coordinates shuffle, train, export, and distributed self-play stages.
python/selfplay/synchronous_loop.sh
Build & Dependencies
- CMake build · Builds the engine and selects platform inference capabilities.
cpp/CMakeLists.txt - Embedded dependencies · Inference and portability dependencies including CLBlast, cuDNN frontend, ONNX, and CoreML support.
cpp/external/clblast
Connections 26
- Executable entry → Command dispatcher: dispatches
- Command dispatcher → GTP and analysis: runs
- Command dispatcher → Training commands: runs
- GTP and analysis → Async search bot: requests moves and analysis
- Training commands → Self-play manager: starts
- Async search bot → Monte Carlo graph search: drives
- Monte Carlo graph search → Search nodes: traverses
- Monte Carlo graph search → Board state: evaluates legal positions
- Monte Carlo graph search → Game history and rules: applies game context
- Search nodes → Position identity: uses identity
- Monte Carlo graph search → Evaluation cache: looks up evaluations
- Evaluation cache → Neural evaluator: misses request inference
- Neural evaluator → NN input encoding: encodes positions
- Neural evaluator → Accelerator backends: executes batches
- Model loading → Neural evaluator: loads model into
- Self-play manager → Async search bot: plays games with
- Self-play manager → Training data writer: emits targets
- Distributed client → Self-play manager: runs assigned tasks
- Training data writer → PyTorch training: provides training data
- PyTorch training → Model export: produces trained model
- Model export → Model loading: exports for inference
- Training loop → PyTorch training: orchestrates
- Training loop → Distributed client: coordinates
- CMake build → Executable entry: builds
- CMake build → Accelerator backends: selects capabilities
- Embedded dependencies → Accelerator backends: supports