How humancompatibleai/overcooked_ai works
Project Type & Purpose
• The project is a research benchmark environment aimed at studying cooperative human-AI interaction in a simulated Overcooked kitchen task. It mixes environment simulation (the MDP), planning and agent logic, some training infrastructure (deprecated human_aware_rl modules), and a demo web server to interact with trained agents.
• The README and file structure indicate multiple “layers” of the project: a simulation core, planning/agent modules, demo web server, and auxiliary tools (data processing, tests, and experiment-running scripts).
Main Components
A. Simulation & Core Environment (overcooked_ai_py)
• The mdp submodule (overcooked_mdp.py, overcooked_env.py, layout_generator.py) is the heart of the environment and game logic.
• The agents folder, including agent implementations and benchmarking utilities, form the decision-making and evaluation layer.
• The planning folder (planners.py, search.py) provides near-optimal planning and search logic used to back the agent strategies.
• Visualization utilities (under visualization/) that support state rendering and possibly debugging the simulation.
B. Demo Server & UI (overcooked_demo)
• A Flask server (app.py, game.py) that serves as the interactive front-end.
• Static assets (JavaScript, CSS, images, templates) required for the web presentation of the game.
• Integration scripts such as move_agents.py and docker-compose/up.sh that aid deployment.
C. Training & Data Processing (human_aware_rl)
• Although now deprecated, this module contains parts for behavior cloning and PPO training. It’s organized into:
• PPO-related code with training, evaluation, and experiment scripts.
• Imitation learning for behavior cloning.
• Data processing utilities for human gameplay data.
• The static folder also includes datasets like “human_data” and testing datasets which serve as training and benchmarks.
D. Supporting Scripts & Tests
• Testing scripts in the “testing” folder and continuous integration configurations (.github, codecov, pyproject.toml, etc.) which ensure the reliability of all modules.
• Setup and installation files (setup.py, pyproject.toml) which provide dependency management and configuration.
Relationships & Data Flow
• The core simulation (mdp) underpins both the planning/agent modules and the demo server; it is effectively shared between training agents and real-time gameplay.
• The planning logic uses the simulation core to compute trajectories and strategies (data flow from simulation → planning → agents).
• Although deprecated, the human_aware_rl module shows the training loop for reinforcement learning and imitation—which involves a cyclical data flow: Raw Human Data (from static/Human Data) → Data Processing → Training (PPO / BC) → Trained Agent exported to the demo.
• Continuous Integration (CI) components and utilities enable testing flow throughout the system.
Architectural Patterns & Design Principles
• Separation of Concerns: Distinct packages separating simulation (mdp), decision-making (agents/planning), user interface (demo server), and experimental/training work (human_aware_rl).
• Modular Design: Each component is a self-contained package with dedicated responsibilities (e.g., layout generation vs. agent benchmarking).
• Integration via APIs: The demo server (Flask app) calls the simulation and agent modules as libraries. Also, planned compatibility via rllib wrappers further modularizes training interfaces.
• Extensibility: New layouts, agents, or training algorithms can be incorporated primarily by adding new modules or updating existing ones.