# andrejorsula/drl_grasping architecture

> Architecture diagram of the GitHub repository andrejorsula/drl_grasping, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/andrejorsula/drl_grasping
- Repository: https://github.com/andrejorsula/drl_grasping

## Overview

This is a ROS 2/Gazebo robotic-manipulation RL project. The principal workflow is to train an agent in registered Gym environments, using state, image, or octree observations, then evaluate policies in simulation or on real robots. Its domain branches include reach, general grasping, and lunar grasping, with simulation randomization and real-robot evaluation; an internal layer supplies octree feature extraction and replay support. Relationships beyond the README and sampled excerpts are omitted or marked unverified.

Main entry points: [`drl_grasping/envs/__init__.py`](https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/envs/__init__.py) (Gym Registration), [`drl_grasping/utils/exp_manager.py`](https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/utils/exp_manager.py) (Experiment Manager), [`drl_grasping/drl_octree/algorithms`](https://github.com/andrejorsula/drl_grasping/tree/master/drl_grasping/drl_octree/algorithms) (SAC / TD3 / TQC), [`drl_grasping/envs/perception`](https://github.com/andrejorsula/drl_grasping/tree/master/drl_grasping/envs/perception) (Camera / Octree)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_tasks["Robot Tasks"]
  node_gym["Gym Registration<br/>[__init__.py]"]
  node_reach["Reach Variants"]
  node_grasp["Grasp Variants"]
  node_lunar["Lunar Grasp"]
  node_manip["Manipulation Base<br/>[manipulation.py]"]
  node_perception["Camera / Octree"]
end

subgraph group_sim["Simulation Models"]
  node_randomizer["World Randomizer<br/>[manipulation.py]"]
  node_robot["Robot Models"]
  node_worldmodels["Scene Models"]
  node_real["Real Runtime<br/>[real_evaluation.py]"]
end

subgraph group_learning["RL Learning"]
  node_policies["SAC / TD3 / TQC"]
  node_imagefeatures["Image CNN<br/>[image_cnn.py]"]
  node_octreefeatures["Octree CNN<br/>[octree_cnn.py]"]
  node_featuremodules["CNN Modules<br/>[modules.py]"]
  node_replay[("Replay Buffer<br/>[replay_buffer.py]")]
end

subgraph group_ops["Training and Evaluation"]
  node_training["Experiment Manager<br/>[exp_manager.py]"]
  node_callbacks["Training Callbacks<br/>[callbacks.py]"]
  node_optimization["Hyperparameter Search<br/>[hyperparams_opt.py]"]
  node_dreamer["DreamerV2 Setup<br/>[train_dreamerv2.py]"]
  node_eval["Policy Evaluation<br/>[evaluate.py]"]
end

node_user(("Engineer / Operator"))

node_user -.->|"starts training"| node_training
node_gym -->|"registers"| node_reach
node_gym -->|"registers"| node_grasp
node_gym -->|"registers"| node_lunar
node_reach -.->|"extends"| node_manip
node_grasp -->|"extends"| node_manip
node_lunar -.->|"uses base"| node_manip
node_gym -->|"configures simulation"| node_randomizer
node_randomizer -->|"populates world"| node_worldmodels
node_randomizer -.->|"configures robot"| node_robot
node_perception -.->|"provides observations"| node_reach
node_perception -.->|"provides observations"| node_grasp
node_perception -.->|"provides observations"| node_lunar
node_user -->|"requests training"| node_training
node_training -.->|"creates environments"| node_gym
node_training -.->|"trains agent"| node_policies
node_policies -.->|"extracts image features"| node_imagefeatures
node_policies -->|"extracts octree features"| node_octreefeatures
node_octreefeatures -->|"uses layers"| node_featuremodules
node_policies -.->|"samples experience"| node_replay
node_training -->|"creates callbacks"| node_callbacks
node_training -->|"runs search"| node_optimization
node_user -->|"requests evaluation"| node_eval
node_eval -.->|"evaluates policy"| node_gym
node_eval -->|"evaluates on robot"| node_real
node_user -.->|"starts experiment"| node_dreamer
node_dreamer -->|"uses image environments"| node_gym

click node_gym "https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/envs/__init__.py"
click node_reach "https://github.com/andrejorsula/drl_grasping/tree/master/drl_grasping/envs/tasks/reach"
click node_grasp "https://github.com/andrejorsula/drl_grasping/tree/master/drl_grasping/envs/tasks/grasp"
click node_lunar "https://github.com/andrejorsula/drl_grasping/tree/master/drl_grasping/envs/tasks/grasp_planetary"
click node_manip "https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/envs/tasks/manipulation.py"
click node_perception "https://github.com/andrejorsula/drl_grasping/tree/master/drl_grasping/envs/perception"
click node_randomizer "https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/envs/randomizers/manipulation.py"
click node_robot "https://github.com/andrejorsula/drl_grasping/tree/master/drl_grasping/envs/models/robots"
click node_worldmodels "https://github.com/andrejorsula/drl_grasping/tree/master/drl_grasping/envs/models"
click node_real "https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/envs/runtimes/real_evaluation.py"
click node_policies "https://github.com/andrejorsula/drl_grasping/tree/master/drl_grasping/drl_octree/algorithms"
click node_imagefeatures "https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/drl_octree/features_extractor/image_cnn.py"
click node_octreefeatures "https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/drl_octree/features_extractor/octree_cnn.py"
click node_featuremodules "https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/drl_octree/features_extractor/modules.py"
click node_replay "https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/drl_octree/replay_buffer/replay_buffer.py"
click node_training "https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/utils/exp_manager.py"
click node_callbacks "https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/utils/callbacks.py"
click node_optimization "https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/utils/hyperparams_opt.py"
click node_dreamer "https://github.com/andrejorsula/drl_grasping/blob/master/scripts/train_dreamerv2.py"
click node_eval "https://github.com/andrejorsula/drl_grasping/blob/master/scripts/evaluate.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_gym,node_reach,node_grasp,node_lunar,node_manip,node_perception toneBlue
class node_randomizer,node_robot,node_worldmodels,node_real toneAmber
class node_policies,node_imagefeatures,node_octreefeatures,node_featuremodules,node_replay toneMint
class node_training,node_callbacks,node_optimization,node_dreamer,node_eval toneRose
class node_user toneIndigo
```

## Components

### Robot Tasks

- [Gym Registration](https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/envs/__init__.py): `drl_grasping/envs/__init__.py`
- [Reach Variants](https://github.com/andrejorsula/drl_grasping/tree/master/drl_grasping/envs/tasks/reach): `drl_grasping/envs/tasks/reach`
- [Grasp Variants](https://github.com/andrejorsula/drl_grasping/tree/master/drl_grasping/envs/tasks/grasp): `drl_grasping/envs/tasks/grasp`
- [Lunar Grasp](https://github.com/andrejorsula/drl_grasping/tree/master/drl_grasping/envs/tasks/grasp_planetary): `drl_grasping/envs/tasks/grasp_planetary`
- [Manipulation Base](https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/envs/tasks/manipulation.py): `drl_grasping/envs/tasks/manipulation.py`
- [Camera / Octree](https://github.com/andrejorsula/drl_grasping/tree/master/drl_grasping/envs/perception): `drl_grasping/envs/perception`

### Simulation Models

- [World Randomizer](https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/envs/randomizers/manipulation.py): `drl_grasping/envs/randomizers/manipulation.py`
- [Robot Models](https://github.com/andrejorsula/drl_grasping/tree/master/drl_grasping/envs/models/robots): `drl_grasping/envs/models/robots`
- [Scene Models](https://github.com/andrejorsula/drl_grasping/tree/master/drl_grasping/envs/models): `drl_grasping/envs/models`
- [Real Runtime](https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/envs/runtimes/real_evaluation.py): `drl_grasping/envs/runtimes/real_evaluation.py`

### RL Learning

- [SAC / TD3 / TQC](https://github.com/andrejorsula/drl_grasping/tree/master/drl_grasping/drl_octree/algorithms): `drl_grasping/drl_octree/algorithms`
- [Image CNN](https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/drl_octree/features_extractor/image_cnn.py): `drl_grasping/drl_octree/features_extractor/image_cnn.py`
- [Octree CNN](https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/drl_octree/features_extractor/octree_cnn.py): `drl_grasping/drl_octree/features_extractor/octree_cnn.py`
- [CNN Modules](https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/drl_octree/features_extractor/modules.py): `drl_grasping/drl_octree/features_extractor/modules.py`
- [Replay Buffer](https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/drl_octree/replay_buffer/replay_buffer.py): `drl_grasping/drl_octree/replay_buffer/replay_buffer.py`

### Training and Evaluation

- [Experiment Manager](https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/utils/exp_manager.py): `drl_grasping/utils/exp_manager.py`
- [Training Callbacks](https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/utils/callbacks.py): `drl_grasping/utils/callbacks.py`
- [Hyperparameter Search](https://github.com/andrejorsula/drl_grasping/blob/master/drl_grasping/utils/hyperparams_opt.py): `drl_grasping/utils/hyperparams_opt.py`
- [DreamerV2 Setup](https://github.com/andrejorsula/drl_grasping/blob/master/scripts/train_dreamerv2.py): `scripts/train_dreamerv2.py`
- [Policy Evaluation](https://github.com/andrejorsula/drl_grasping/blob/master/scripts/evaluate.py): `scripts/evaluate.py`

### Other components

- **Engineer / Operator**

## Connections

- Engineer / Operator → Experiment Manager: starts training
- Gym Registration → Reach Variants: registers (evidence: `drl_grasping/envs/__init__.py`)
- Gym Registration → Grasp Variants: registers (evidence: `drl_grasping/envs/__init__.py`)
- Gym Registration → Lunar Grasp: registers (evidence: `drl_grasping/envs/__init__.py`)
- Reach Variants → Manipulation Base: extends
- Grasp Variants → Manipulation Base: extends (evidence: `drl_grasping/envs/tasks/grasp/grasp.py`)
- Lunar Grasp → Manipulation Base: uses base
- Gym Registration → World Randomizer: configures simulation (evidence: `README.md`)
- World Randomizer → Scene Models: populates world (evidence: `drl_grasping/envs/randomizers/manipulation.py`)
- World Randomizer → Robot Models: configures robot
- Camera / Octree → Reach Variants: provides observations
- Camera / Octree → Grasp Variants: provides observations
- Camera / Octree → Lunar Grasp: provides observations
- Engineer / Operator → Experiment Manager: requests training (evidence: `README.md`)
- Experiment Manager → Gym Registration: creates environments
- Experiment Manager → SAC / TD3 / TQC: trains agent
- SAC / TD3 / TQC → Image CNN: extracts image features
- SAC / TD3 / TQC → Octree CNN: extracts octree features (evidence: `drl_grasping/drl_octree/algorithms/sac/policies.py`)
- Octree CNN → CNN Modules: uses layers (evidence: `drl_grasping/drl_octree/features_extractor/octree_cnn.py`)
- SAC / TD3 / TQC → Replay Buffer: samples experience (evidence: `drl_grasping/drl_octree/algorithms/off_policy_algorithm.py`)
- Experiment Manager → Training Callbacks: creates callbacks (evidence: `drl_grasping/utils/exp_manager.py`)
- Experiment Manager → Hyperparameter Search: runs search (evidence: `drl_grasping/utils/exp_manager.py`)
- Engineer / Operator → Policy Evaluation: requests evaluation (evidence: `README.md`)
- Policy Evaluation → Gym Registration: evaluates policy
- Policy Evaluation → Real Runtime: evaluates on robot (evidence: `README.md`)
- Engineer / Operator → DreamerV2 Setup: starts experiment
- DreamerV2 Setup → Gym Registration: uses image environments (evidence: `README.md`)