# mabanser/pokerrl architecture

> Architecture diagram of the GitHub repository mabanser/pokerrl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mabanser/pokerrl
- Repository: https://github.com/mabanser/pokerrl

## Overview

##### Project type and purpose
- This is a research-oriented reinforcement learning (RL) codebase for a bachelor’s thesis.
- Purpose: implement and compare Neural Fictitious Self-Play (NFSP) with and without a pretrained Hand-Strength predictor in simplified poker environments.

##### File structure
- Top-level scripts (train_agents.py, train_handstrength.py, eval_agent.py, etc.) orchestrate training, evaluation, validation, and visualization.
- `poker/clubs/`: vendored third-party poker engine (game rules, card evaluation, renderers).
- `poker/envs/`: wrapper making the clubs engine conform to PokerRL’s interface.
- `poker/agents/`: different agent implementations (NFSP, random, call, simple, Leduc).
- `poker/models/`: neural networks (hand-strength predictor, NFSP networks) and replay buffers.
- Utility and visualization modules for logging and plotting results.
- Configuration via PokerRL.yml and .bat scripts for conda environment setup.

##### README for additional insights
- Environment setup relies on Conda and PokerRL.
- Dependencies: PyTorch, PokerRL, clubs, etc.

##### Diagramming conventions
- Obs → Agent → Action → Env → Reward/Next Obs → Buffer → Model Update
- Green for environment components
- Blue for agent/model components
- Orange for orchestration scripts

##### Final pointers
- Essential modules but avoid cluttering with every source file.
- Call out extension points—where a future developer would drop in a new agent or environment.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Environment Layer
    subgraph "Environment Layer"
        direction TB
        Clubs["Clubs Engine"]:::environment
        EnvWrapper["PokerRL Env Wrapper"]:::environment
    end

    %% Agent & Model Layer
    subgraph "Agent & Model Layer"
        direction TB
        subgraph "Agents"
            direction TB
            BaseAgent["Base Agent"]:::agent
            CallAgent["Call Agent"]:::agent
            RandomAgent["Random Agent"]:::agent
            SimpleAgent["Simple Agent"]:::agent
            LeducAgent["Leduc Agent"]:::agent
            NFSPAgent["NFSP Agent"]:::agent
        end
        subgraph "Models & Buffers"
            direction TB
            HSPredictor["Hand-Strength Predictor"]:::model
            NFSPModels["NFSP Policy & Value Nets"]:::model
            ReplayBuffer[(Replay Buffer)]:::buffer
        end
    end

    %% Orchestration / CLI
    subgraph "Orchestration / CLI"
        direction TB
        TrainHS["train_handstrength.py"]:::orchestration
        TrainAgents["train_agents.py"]:::orchestration
        EvalAgents["eval_agent.py"]:::orchestration
        ValidateEnv["validate_env.py"]:::orchestration
        VizAgents["visualize_agents.py"]:::orchestration
        VizHS["visualize_hand_strength.py"]:::orchestration
    end

    %% Utilities & Config
    subgraph "Utilities & Config"
        direction TB
        Utils["Utility Functions"]:::utils
        VizHelpers["Visualization Helpers"]:::utils
        Config["PokerRL.yml"]:::config
        SetupScripts["Install_environment.bat\nCheck_installation.bat"]:::config
    end

    %% Dataflow: Training Pipeline
    TrainAgents -->|"start training"| EnvWrapper
    EnvWrapper -->|"state"| BaseAgent
    EnvWrapper -->|"state"| CallAgent
    EnvWrapper -->|"state"| RandomAgent
    EnvWrapper -->|"state"| SimpleAgent
    EnvWrapper -->|"state"| LeducAgent
    EnvWrapper -->|"state"| NFSPAgent

    BaseAgent -->|"action"| EnvWrapper
    CallAgent -->|"action"| EnvWrapper
    RandomAgent -->|"action"| EnvWrapper
    SimpleAgent -->|"action"| EnvWrapper
    LeducAgent -->|"action"| EnvWrapper
    NFSPAgent -->|"action"| EnvWrapper

    EnvWrapper -->|"next state & reward"| NFSPAgent
    NFSPAgent -->|"store transition"| ReplayBuffer
    ReplayBuffer -->|"training update"| NFSPModels
    NFSPModels -->|"policy/value update"| NFSPAgent

    %% Hand-Strength Predictor Pipeline
    TrainHS -->|"train predictor"| HSPredictor
    HSPredictor -->|"saved model"| NFSPAgent

    %% Evaluation & Visualization
    EvalAgents -->|"run games"| EnvWrapper
    EvalAgents -->|"collect metrics"| VizAgents
    TrainHS -->|"metrics"| VizHS

    VizAgents -->|"plot results"| VizHelpers
    VizHS -->|"plot HS predictions"| VizHelpers

    %% Validation
    ValidateEnv -->|"test interface"| EnvWrapper

    %% Utilities & Config links
    TrainAgents --> Utils
    TrainHS --> Utils
    EvalAgents --> Utils
    VizAgents --> Config
    VizHS --> Config

    %% Environment interactions
    EnvWrapper --> Clubs
    Clubs --> EnvWrapper

    %% Click Events
    click Clubs "https://github.com/mabanser/pokerrl/tree/master/poker/clubs"
    click EnvWrapper "https://github.com/mabanser/pokerrl/blob/master/poker/envs/clubs_env.py"
    click BaseAgent "https://github.com/mabanser/pokerrl/blob/master/poker/agents/base_agent.py"
    click CallAgent "https://github.com/mabanser/pokerrl/blob/master/poker/agents/call_agent.py"
    click RandomAgent "https://github.com/mabanser/pokerrl/blob/master/poker/agents/random_agent.py"
    click SimpleAgent "https://github.com/mabanser/pokerrl/blob/master/poker/agents/simple_agent.py"
    click LeducAgent "https://github.com/mabanser/pokerrl/blob/master/poker/agents/leduc_agent.py"
    click NFSPAgent "https://github.com/mabanser/pokerrl/blob/master/poker/agents/nfsp_agent.py"
    click HSPredictor "https://github.com/mabanser/pokerrl/blob/master/poker/models/hand_strength_predictor.py"
    click NFSPModels "https://github.com/mabanser/pokerrl/blob/master/poker/models/nfsp.py"
    click ReplayBuffer "https://github.com/mabanser/pokerrl/blob/master/poker/models/buffers.py"
    click TrainHS "https://github.com/mabanser/pokerrl/blob/master/train_handstrength.py"
    click TrainAgents "https://github.com/mabanser/pokerrl/blob/master/train_agents.py"
    click EvalAgents "https://github.com/mabanser/pokerrl/blob/master/eval_agent.py"
    click ValidateEnv "https://github.com/mabanser/pokerrl/blob/master/validate_env.py"
    click VizAgents "https://github.com/mabanser/pokerrl/blob/master/visualize_agents.py"
    click VizHS "https://github.com/mabanser/pokerrl/blob/master/visualize_hand_strength.py"
    click Utils "https://github.com/mabanser/pokerrl/blob/master/poker/utils.py"
    click VizHelpers "https://github.com/mabanser/pokerrl/blob/master/poker/visualization.py"
    click Config "https://github.com/mabanser/pokerrl/blob/master/PokerRL.yml"
    click SetupScripts "https://github.com/mabanser/pokerrl/blob/master/Install_environment.bat"

    %% Styles
    classDef environment fill:#ccffcc,stroke:#2d572c;
    classDef agent fill:#ccddff,stroke:#2c4a8a;
    classDef model fill:#ccddee,stroke:#2c4a8a;
    classDef buffer fill:#ffeeaa,stroke:#8a682c,stroke-dasharray: 5 5;
    classDef orchestration fill:#ffddcc,stroke:#8a2c2c;
    classDef utils fill:#f2f2f2,stroke:#666666;
    classDef config fill:#fef1c2,stroke:#8a552c;
```
