# digirl-agent/digirl architecture

> Architecture diagram of the GitHub repository digirl-agent/digirl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/digirl-agent/digirl
- Repository: https://github.com/digirl-agent/digirl

## Overview

##### Main Components
- Core Algorithm Modules: In the “digirl/algorithms” directory, you have submodules for different training methods (e.g., DigiRL and Filtered Behavior Cloning), as well as specialized training loops (like eval_loop and worker_collect_loop). These represent your training engines.
- Environment Component: The “digirl/environment” directory (with its “android” subfolder) contains the simulation interface for Android devices. This includes the client, evaluation routines, auto UI utilities, and environment setup for error handling.
- Data Handling and Utility Layers: The “digirl/data” package (plus related utility functions) manages how data (e.g., pre-collected trajectories) are processed and fed into training.
- Models and Agents: In “digirl/models”, you see definitions of agents (AutoUI and CogAgent), critics, and related utility functions for model inference.
- External Setup and Deployment: The “env_setup” folder contains configuration files and scripts to set up the Android environment (including screenshot tools). The “multimachine” folder and the “scripts/config” directory provide configurations for multi-machine/emulation parallel training and multi-GPU setups via accelerate.
- Experiment and Execution Layer: The “scripts” directory (and specifically the “run.py” entry point) bridges the configuration files and the launch of experiments, orchestrating training across different modes based on user configurations.

##### Relationships and Interactions Between Components
- The execution starts with scripts/run.py, which loads configuration files from scripts/config (e.g., digirl_online, digirl_offline, digirl_off2on) to define training parameters.
- Configuration files guide which algorithm component (either in digirl/algorithms/digirl or filteredbc) gets activated, how the environment is set up (using digirl/environment/android), and which data sources (from digirl/data and external checkpoints) are used.
- The environment component interacts with the RL agents (from digirl/models) through well-defined interfaces. The training loops (offline, online, offline-to-online) then continuously interact with the environment, gather trajectories, and update the agent’s policy.
- Multi-machine and multi-GPU modules ensure that the training process can be distributed; these are configured in the “multimachine” and “scripts/config/accelerate_config” sections.
- There is support for external service integrations (e.g., CogAgent server via a Gradio-based API) and cloud storage for checkpoints and data.

##### Architectural Patterns and Design Principles
- Modular Organization: Each major functionality (algorithms, environment, data handling, models) is isolated into separate directories, suggesting a clean modular design.
- Config-Driven Execution: Comprehensive configuration directories (scripts/config/*) allow flexible switching between training paradigms and deployment setups.
- Scalability: The inclusion of multi-machine emulation (multimachine/) and distributed training configurations using ‘accelerate’ indicates that the system is built to scale.

##### Technologies and Frameworks
- Python is the primary language.
- Reinforcement learning paradigms with offline, online, and hybrid training loops.
- “accelerate” for distributed multi-GPU training.
- Likely use of common deep learning libraries, though details are abstracted in model definitions.
- Gradio for serving the CogAgent API (if used).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Entry Point
    ER["Experiment Runner (scripts/run.py)"]:::experiment
    ER -->|"loads"| CM["Configuration Manager (scripts/config & env_setup)"]:::config

    %% Algorithm Engine Subgraph
    subgraph "Algorithm Engine"
        DT["DigiRL Trainer"]:::algo
        FBT["Filtered BC Trainer"]:::algo
        EL["Evaluation Loop"]:::algo
        WCL["Worker Collect Loop"]:::algo
        OT["Offpolicy Train Loop"]:::algo
    end

    %% Environment Interface Subgraph
    subgraph "Environment Interface"
        AE["Android Environment (digirl/environment/android)"]:::env
        AUT["AutoUI Utils"]:::env
        CL["Client"]:::env
        ES["Environment Setup"]:::env
        EV["Evaluate"]:::env
    end

    %% Data Pipeline
    DP["Data Pipeline (digirl/data)"]:::data

    %% Agent Models Subgraph
    subgraph "Agent Models"
        AM["Agent Models (digirl/models)"]:::model
        AA["AutoUI Agent"]:::model
        CA["CogAgent"]:::model
        CR["Critic"]:::model
        IU["Inference Utils"]:::model
        MD["Model"]:::model
    end

    %% Distributed Training Subgraph
    subgraph "Distributed Training"
        MM["Multi-machine"]:::deploy
        AC["Accelerate Config"]:::deploy
    end

    %% Connections from Configurations
    CM -->|"configures"| DT
    CM -->|"configures"| FBT
    CM -->|"configures"| EL
    CM -->|"configures"| WCL
    CM -->|"configures"| OT
    CM -->|"sets up"| MM
    CM -->|"sets up"| AC

    %% Interactions between Training and Environment
    DT -->|"trains with"| AE
    FBT -->|"trains with"| AE
    EL  -->|"evaluates"| AE
    WCL -->|"collects_data_from"| AE
    OT  -->|"trains_offline_with"| AE

    %% Environment internal connections
    AE --> AUT
    AE --> CL
    AE --> ES
    AE --> EV

    %% Data and Agent interactions
    AE -->|"provides_data"| DP
    DP -->|"supplies_data"| AM
    AM -->|"updates_policy_using"| AE
    DT -->|"updates"| AM
    FBT -->|"updates"| AM

    %% Distributed scaling influences
    MM -->|"scales"| DT
    MM -->|"scales"| FBT
    AC -->|"scales"| DT
    AC -->|"scales"| FBT

    %% Click Events
    click ER "https://github.com/digirl-agent/digirl/blob/master/scripts/run.py"
    click CM "https://github.com/digirl-agent/digirl/tree/master/scripts/config"
    click DT "https://github.com/digirl-agent/digirl/blob/master/digirl/algorithms/digirl/trainer.py"
    click FBT "https://github.com/digirl-agent/digirl/blob/master/digirl/algorithms/filteredbc/trainer.py"
    click EL "https://github.com/digirl-agent/digirl/blob/master/digirl/algorithms/eval_loop.py"
    click WCL "https://github.com/digirl-agent/digirl/blob/master/digirl/algorithms/worker_collect_loop.py"
    click OT "https://github.com/digirl-agent/digirl/blob/master/digirl/algorithms/offpolicy_train_loop.py"
    click AE "https://github.com/digirl-agent/digirl/tree/master/digirl/environment/android"
    click AUT "https://github.com/digirl-agent/digirl/blob/master/digirl/environment/android/autoui_utils.py"
    click CL "https://github.com/digirl-agent/digirl/blob/master/digirl/environment/android/client.py"
    click ES "https://github.com/digirl-agent/digirl/blob/master/digirl/environment/android/env.py"
    click EV "https://github.com/digirl-agent/digirl/blob/master/digirl/environment/android/evaluate.py"
    click DP "https://github.com/digirl-agent/digirl/tree/master/digirl/data"
    click AM "https://github.com/digirl-agent/digirl/tree/master/digirl/models"
    click AA "https://github.com/digirl-agent/digirl/blob/master/digirl/models/autoui_agent.py"
    click CA "https://github.com/digirl-agent/digirl/blob/master/digirl/models/cog_agent.py"
    click CR "https://github.com/digirl-agent/digirl/blob/master/digirl/models/critic.py"
    click IU "https://github.com/digirl-agent/digirl/blob/master/digirl/models/infer_utils.py"
    click MD "https://github.com/digirl-agent/digirl/blob/master/digirl/models/model.py"
    click MM "https://github.com/digirl-agent/digirl/tree/master/multimachine"
    click AC "https://github.com/digirl-agent/digirl/tree/master/scripts/config/accelerate_config"

    %% Styles
    classDef experiment fill:#ffcccc,stroke:#333,stroke-width:2px;
    classDef config fill:#ffffcc,stroke:#333,stroke-width:2px;
    classDef algo fill:#ccf2ff,stroke:#333,stroke-width:2px;
    classDef env fill:#ccffcc,stroke:#333,stroke-width:2px;
    classDef data fill:#e6e6fa,stroke:#333,stroke-width:2px;
    classDef model fill:#f0e68c,stroke:#333,stroke-width:2px;
    classDef deploy fill:#ffdab9,stroke:#333,stroke-width:2px;
```
