How shubhayu-dev/gambit works
GAMBIT evaluates chess agents and uses their move-level telemetry to drive a closed-loop workflow: load benchmark positions, run agent evaluations, analyze performance, investigate candidate causes, test interventions on held-out positions, and report results. The engine provides chess rules, agents, search, and evaluation; supporting branches add Lichess data, anomaly and disagreement analysis, and move commentary. The README documents these capabilities, while sampled source confirms key evaluation, diagnostic, and integration links; some unsampled subsystem wiring is therefore shown only where explicitly documented.
Main entry points: run_full_pipeline.py Pipeline Runner · runner.py Experiment Runner · search.py Alpha-Beta Search · commentary.py Move Commentary
Components 21
Chess Engine
- Chess Environment
engine/environment.py - Agent Implementations
engine/agents.py - Alpha-Beta Search
engine/search.py - Position Evaluation
engine/evaluation.py
Evaluation Pipeline
- Pipeline Runner
run_full_pipeline.py - Lichess Loader
experiments/lichess_data.py - Experiment Runner
experiments/runner.py - Trajectory Store
experiments/data_store.py
Diagnosis Loop
- Empirical Investigation
diagnosis/diagnostic_experiment.py - Hypothesis Formation
diagnosis/hypotheses.py - Held-out Validation
diagnosis/intervention.py - LLM Diagnosis
diagnosis/llm_reasoner.py
Analysis & Reporting
- Metrics & Profiling
analysis/metrics.py - Anomaly Detection
analysis/anomaly.py - Agent Disagreement
analysis/disagreement.py - Move Commentary
analysis/commentary.py - Rule Commentary
analysis/rule_based_commentary.py - HTML Report
dashboard/report_html.py
Other components
- Evaluator
- Gemini API
- Ollama Service
Connections 21
- Evaluator → Pipeline Runner: starts run
- Pipeline Runner → Lichess Loader: loads positions
- Pipeline Runner → Experiment Runner: runs evaluation
- Experiment Runner → Chess Environment: builds positions
- Experiment Runner → Agent Implementations: requests moves
- Agent Implementations → Alpha-Beta Search: searches moves
- Alpha-Beta Search → Position Evaluation: scores positions
- Alpha-Beta Search → Chess Environment: queries moves
- Experiment Runner → Trajectory Store: writes trajectories
- Pipeline Runner → Metrics & Profiling: summarizes records
- Pipeline Runner → Anomaly Detection: detects anomalies
- Pipeline Runner → Agent Disagreement: compares agents
- Pipeline Runner → Empirical Investigation: investigates blunders
- Empirical Investigation → Hypothesis Formation: forms hypotheses
- Pipeline Runner → Held-out Validation: checks candidates
- Pipeline Runner → LLM Diagnosis: submits evidence
- LLM Diagnosis → Gemini API: requests diagnosis
- Pipeline Runner → HTML Report: generates report
- Rule Commentary → Move Commentary: filters commentary
- Move Commentary → Gemini API: requests annotations
- Move Commentary → Ollama Service: requests annotations