How mabanser/pokerrl works
Project type and purpose
• This is a research-oriented reinforcement learning (RL) codebase for a bachelor’s thesis.
• Purpose: implement and compare Neural Fictitious Self-Play (NFSP) with and without a pretrained Hand-Strength predictor in simplified poker environments.
File structure
• Top-level scripts (train_agents.py, train_handstrength.py, eval_agent.py, etc.) orchestrate training, evaluation, validation, and visualization.
• poker/clubs/: vendored third-party poker engine (game rules, card evaluation, renderers).
• poker/envs/: wrapper making the clubs engine conform to PokerRL’s interface.
• poker/agents/: different agent implementations (NFSP, random, call, simple, Leduc).
• poker/models/: neural networks (hand-strength predictor, NFSP networks) and replay buffers.
• Utility and visualization modules for logging and plotting results.
• Configuration via PokerRL.yml and .bat scripts for conda environment setup.
README for additional insights
• Environment setup relies on Conda and PokerRL.
• Dependencies: PyTorch, PokerRL, clubs, etc.
Diagramming conventions
• Obs → Agent → Action → Env → Reward/Next Obs → Buffer → Model Update
• Green for environment components
• Blue for agent/model components
• Orange for orchestration scripts
Final pointers
• Essential modules but avoid cluttering with every source file.
• Call out extension points—where a future developer would drop in a new agent or environment.