How pmuens/alphago works
This repository is a modular reference implementation of AlphaGo-style agents for Go, plus a smaller Tic-Tac-Toe demo. The architecture is centered on a shared game-engine core, with separate layers for encoders, search, neural models, agents, data pipelines, and user-facing entrypoints.
1. Core game domain model
The dlgo/gotypes.py, dlgo/goboard*.py, dlgo/scoring.py, and dlgo/zobrist.py modules define the Go rules, board state, scoring, and hashing. This is the foundation used by both classical search agents and neural agents. There is a parallel dlgo/ttt/ package for Tic-Tac-Toe with its own board/types.
2. Encoders as the bridge from board state to tensors
dlgo/encoders/ converts game states into model-ready feature planes. Different encoder variants exist (oneplane, sevenplane, simple, alphago), which act as the boundary between the symbolic Go engine and the learning stack. Encoders are a key architectural seam because they determine what the neural nets “see.”
3. Neural network definitions
dlgo/networks/, dlgo/cnn/, and dlgo/nn/ contain model architectures and low-level neural utilities. The repository includes policy/value-style models, fully connected variants, and CNNs suited to Go. The dlgo/nn/ package also contains generic neural primitives and an MNIST example, indicating the codebase includes reusable learning infrastructure beyond Go.
4. Search and decision-making layer
dlgo/mcts/ implements Monte Carlo Tree Search, which is the main planning component for stronger Go play. dlgo/minimax/ provides minimax/alpha-beta/depth-pruned search, mainly for simpler game settings or baselines. These search modules consume game state and optionally neural evaluations from the agent layer.
5. Agent abstraction layer
dlgo/agent/ contains the strategy interface and concrete agent types: naive bots, policy-gradient agents, prediction-based agents, AlphaGo-style agents, and termination/rollout helpers. This package is the main orchestration boundary between “how to choose a move” and the underlying search/model machinery.
6. Reinforcement learning subsystem
dlgo/rl/ contains self-play simulation, experience collection, and RL algorithms (ac, q, value). This is the learning loop used to improve agents from generated games. The scripts self_play_ac.py, self_play_pg.py, train_ac.py, train_pg.py, and eval_ac_bot.py drive the training lifecycle around this package.
7. Supervised learning and data generation pipeline
dlgo/data/ plus scripts like download_u_go_data.py, load_training_data.py, generate_mcts_games.py, and train_generator.py handle extraction and processing of training examples. The repo also ships pre-generated arrays under generated_games/ (features-*.npy, labels-*.npy), showing a clear split between offline dataset preparation and model training.
8. SGF ingestion and game replay
dlgo/gosgf/ parses SGF files, and sgf_to_game_state.py converts recorded games into internal board states. This is the main path for turning human/game-record data into supervised training samples or replayable game trajectories.
9. External interfaces for playing bots
dlgo/gtp/ implements the Go Text Protocol, allowing the engines to connect to GTP-compatible clients and engines like GNU Go. The scripts bot_v_gnugo.py, bot_v_x_gtp.py, human_v_bot.py, and dlgo/gtp/play_local.py show that the repository supports both local play and interop with external Go programs.
10. Web demo / HTTP frontend
dlgo/httpfrontend/ provides a lightweight server plus static assets for browser-based play. The HTML variants (play_mcts_55.html, play_pg_99.html, etc.) suggest different agent backends can be exposed through the same frontend shell. This is a separate presentation layer from the GTP interface.
11. Script-driven entrypoints and experiments
Top-level scripts such as mcts_go.py, bot_v_bot.py, alphago_mcts_play.py, alpha_beta_go.py, end_to_end.py, web_demo.py, and play_ttt.py are thin executable entrypoints that wire together the core packages for demos, benchmarks, or experiment runs. The repository is therefore organized more as a library plus runnable experiments than as a single monolithic application.
12. Data and model artifacts
Trained agents are persisted as .h5 files, while generated datasets are stored as .npy arrays. The presence of kerasutil.py indicates Keras-based model serialization/loading is part of the workflow. This means the architecture has a clear offline training boundary: data generation → training → saved model artifact → inference in agents/search.
13. Overall flow for Go play
A typical runtime path is: board state created in goboard → encoded by encoders → evaluated by a neural network in networks/cnn/nn → guided by mcts or policy/value agents in agent/ → exposed via GTP or HTTP frontend. The same core game state can also be fed into RL/self-play loops to produce new experience for training.
14. Scope and boundaries
The codebase is not just “an AlphaGo model”; it is a layered Go AI toolkit with reusable pieces for board representation, SGF parsing, encoders, learning, search, protocol adapters, and demos. The main architectural boundary is between the game engine/domain layer and the learning/search layers, with encoders and agent interfaces mediating between them.
Main entry points: agent Agents · goboard.py Go board · rl RL loop · encoders Encoders
Components 19
Core game model
Rules, state, scoring, and hashing for Go and Tic-Tac-Toe.
- Go types · Core Go stone, coordinate, and player types.
dlgo/gotypes.py - Go board · Primary Go board and move application logic.
dlgo/goboard.py - Scoring · Game end and territory scoring for Go.
dlgo/scoring.py - Zobrist hash · Board hashing for transposition and repetition handling.
dlgo/zobrist.py - Tic-Tac-Toe · Separate board and types for the Tic-Tac-Toe demo.
dlgo/ttt/tttboard.py
State encoding
Feature extraction boundaries between game state and tensors.
- Encoders · Board-to-tensor encoders such as one-plane, seven-plane, simple, and AlphaGo-style.
dlgo/encoders
Learning stack
Neural models, RL loops, and data preparation.
- Data pipeline · Example generation, sampling, indexing, and parallel processing for training data.
dlgo/data - SGF parser · Parses SGF files and supports replaying recorded games.
dlgo/gosgf - Networks · Go-oriented neural architectures, including AlphaGo, CNN, and MLP variants.
dlgo/networks - NN core · Reusable neural network layers, loaders, and a small MNIST example.
dlgo/nn - RL loop · Self-play, experience storage, and policy/value/Q learning routines.
dlgo/rl - Zero stack · AlphaGo Zero-inspired encoder, agent, and experience pipeline.
dlgo/zero - Artifacts · Pre-generated training arrays and packaged sample data.
generated_games
Search
Planning and move selection algorithms.
Interfaces
External protocols and user-facing frontends.
Entrypoints
Top-level runnable scripts and experiments.
- Scripts · Top-level runnable demos, training jobs, and evaluation scripts.
Connections 25
- Go board → Go types: uses types · Board logic is built on shared Go domain types.
- Scoring → Go board: scores state · Scoring evaluates finished board positions.
- Zobrist hash → Go board: hashes board · Hashing depends on board state.
- Tic-Tac-Toe → Go types: separate types · The Tic-Tac-Toe demo defines its own minimal domain types.
- Encoders → Go board: reads state · Encoders convert board positions into feature planes.
- Data pipeline → Encoders: builds features · Dataset generation uses encoders to produce training examples.
- SGF parser → Go board: replays games · Recorded SGF moves are turned into board states.
- Networks → NN core: built on · Model definitions use reusable neural primitives.
- RL loop → Agents: trains agents · RL algorithms update agent behavior from self-play experience.
- RL loop → Go board: simulates play · Self-play depends on game state transitions.
- Agents → Encoders: encodes inputs · Neural agents depend on feature encoders.
- Agents → Networks: evaluates policy · Predictive and AlphaGo-style agents call into neural models.
- Agents → MCTS: guides search · AlphaGo-style agents use MCTS for move selection.
- Minimax → Go board: searches state · Classic search operates directly on board positions.
- GTP → Agents: wraps bots · GTP exposes agents as external playable engines.
- Web demo → Agents: serves bots · Web play frontends connect to underlying agents.
- Scripts → Agents: wires runtime · Top-level scripts assemble agents, search, and I/O.
- Scripts → RL loop: runs training · Training and self-play entrypoints drive the RL subsystem.
- Scripts → GTP: launches protocol · Some demos connect through the GTP adapter.
- Scripts → Web demo: launches demo · Web demo scripts start the browser frontend.
- Scripts → Data pipeline: generates data · Training scripts and generators build offline datasets.
- Zero stack → Encoders: uses encoding · Zero-style components still depend on state encoding.
- Zero stack → RL loop: self-play loop · Zero subsystem extends the self-play and experience workflow.
- Artifacts → Data pipeline: stores samples · Generated arrays are consumed by the data/training pipeline.
- Artifacts → Scripts: used by · Prepared datasets back the training scripts.