# ethz-asl/vgn architecture

> Architecture diagram of the GitHub repository ethz-asl/vgn, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ethz-asl/vgn
- Repository: https://github.com/ethz-asl/vgn

## Overview

##### Main System Components
- Data Generation & Dataset Construction:
- Components include “scripts/generate_data.py” for simulating grasp trials with pybullet (physics simulation), “scripts/construct_dataset.py” for converting raw simulation output into voxel grids/grasp targets, and associated notebooks in "notebooks/" for cleaning and visualizing data.
- Neural Network Training:
- Involve “scripts/train_vgn.py” and the core network implementation files in “src/vgn/” such as “networks.py”, “dataset.py”, and possibly “baselines.py”.
- TensorBoard logging (data/runs) should be noted as a feedback channel.
- Simulation and Evaluation:
- The system simulates grasping experiments using “scripts/sim_grasp.py” along with ROS visualization configurations from “config/sim.rviz”, “config/gpd/ros_eigen_params.cfg” and the “launch” files.
- There is also a clutter removal experiment in “src/vgn/experiments/clutter_removal.py” and the “notebooks/clutter_removal.ipynb” for evaluation.
- Robot Grasp Execution:
- The integration with hardware is shown via “scripts/panda_grasp.py”, “launch/panda_grasp.launch”, and additional ROS integration in “src/vgn/utils/ros_utils.py” and “config/panda.yaml”.
- The experimental setup involves a Franka Emika Panda robot with a Realsense sensor and the MoveIt configuration.
- Configuration & Integration:
- The project’s configuration files (YAMLs and RViz configurations in “config/”) and ROS launch files (in “launch/”) define environment, visualization, and simulation parameters.
- Build and package management are supported by CMakeLists.txt, package.xml, setup.py, and requirements.txt.

##### Out Relationships and Interactions
- Data Flow:
- Dataset Construction then processes the raw data into a format that the neural network can use.
- Next, Network Training using the processed datasets creates the trained model (e.g., vgn_conv.pth).
- Inference & Simulation:
- The trained model is then used in simulated grasping (sim_grasp.py) and is visualized using ROS RViz configurations.
- Robot Interaction:
- For real-world experiments, the trained model is deployed via scripts like “panda_grasp.py”, interfacing with ROS nodes over a distributed system (robot drivers on one machine and grasp scripts on another).
- External Dependencies:
- The system uses ROS for visualization and robot interfacing, pybullet for simulation, and optionally OpenMPI for distributed data generation.

##### Architectural Patterns
- Modularity:
- The system is organized into distinct components (data generation, training, simulation, and robot execution). Each component’s logic resides in separate directories (scripts, notebooks, src/vgn).
- Layered Approach:
- There is a clear separation between the back-end (data handling, neural network training, simulation) and the interface layer (ROS, visualization, robot control).
- Integration with ROS:
- ROS nodes, launch files, and RViz configurations indicate a service-based or event-driven communication for real-time monitoring and hardware interfacing.

## Diagram (Mermaid)

```mermaid
flowchart TB

    %% Data Generation & Dataset Construction
    subgraph "Data Generation & Dataset Construction"
        DG1["scripts/generate_data.py"]:::data
        DG2["scripts/construct_dataset.py"]:::data
        NBD["notebooks/data.ipynb"]:::notebook
    end

    %% Neural Network Training
    subgraph "Neural Network Training"
        NT1["scripts/train_vgn.py"]:::training
        NT2["src/vgn/networks.py"]:::training
        NT3["src/vgn/dataset.py"]:::training
        NT4["src/vgn/baselines.py"]:::training
        TensorBoard["TensorBoard Logging (data/runs)"]:::external
    end

    %% Simulation & Evaluation
    subgraph "Simulation & Evaluation"
        SIM1["scripts/sim_grasp.py"]:::simulation
        SIM2["src/vgn/experiments/clutter_removal.py"]:::simulation
        NBS["notebooks/clutter_removal.ipynb"]:::notebook
    end

    %% Robot Grasp Execution
    subgraph "Robot Grasp Execution"
        RG1["scripts/panda_grasp.py"]:::robot
        RG2["launch/panda_grasp.launch"]:::robot
        RG3["src/vgn/utils/ros_utils.py"]:::robot
        RG4["config/panda.yaml"]:::robot
    end

    %% Configuration & Integration
    subgraph "Configuration & Integration"
        CONF1["config/sim.rviz"]:::config
        CONF2["config/gpd/ros_eigen_params.cfg"]:::config
        CONF3["Other config files"]:::config
        LAUNCH["launch/gpd.launch"]:::config
        BUILD["CMakeLists.txt,package.xml,setup.py,requirements.txt"]:::config
    end

    %% External Dependencies
    ROS["ROS Ecosystem"]:::external
    PyBullet["PyBullet"]:::external

    %% Data Flow Connections
    DG2 -- "process_data" --> NT3
    NT1 -- "trained_model" --> SIM1
    NT1 -- "trained_model" --> RG1
    NT1 -- "logging" --> TensorBoard
    DG1 -- "simulation_output" --> PyBullet
    SIM1 -- "ROS_visualization" --> ROS
    RG1 -- "ROS_control" --> ROS
    SIM1 -- "uses_config" --> CONF1
    RG1 -- "uses_config" --> CONF1

    %% Class Definitions for styling
    classDef data fill:#AED6F1,stroke:#1B4F72,stroke-width:2px;
    classDef training fill:#A3E4D7,stroke:#0E6251,stroke-width:2px;
    classDef simulation fill:#F9E79F,stroke:#B9770E,stroke-width:2px;
    classDef robot fill:#F5B7B1,stroke:#943126,stroke-width:2px;
    classDef config fill:#D2B4DE,stroke:#6C3483,stroke-width:2px;
    classDef external fill:#FAD7A0,stroke:#CA6F1E,stroke-width:2px;
    classDef notebook fill:#FDEBD0,stroke:#D35400,stroke-width:2px;

    %% Click Events
    click DG1 "https://github.com/ethz-asl/vgn/blob/corl2020/scripts/generate_data.py"
    click DG2 "https://github.com/ethz-asl/vgn/blob/corl2020/scripts/construct_dataset.py"
    click NBD "https://github.com/ethz-asl/vgn/blob/corl2020/notebooks/data.ipynb"
    
    click NT1 "https://github.com/ethz-asl/vgn/blob/corl2020/scripts/train_vgn.py"
    click NT2 "https://github.com/ethz-asl/vgn/blob/corl2020/src/vgn/networks.py"
    click NT3 "https://github.com/ethz-asl/vgn/blob/corl2020/src/vgn/dataset.py"
    click NT4 "https://github.com/ethz-asl/vgn/blob/corl2020/src/vgn/baselines.py"
    
    click SIM1 "https://github.com/ethz-asl/vgn/blob/corl2020/scripts/sim_grasp.py"
    click SIM2 "https://github.com/ethz-asl/vgn/blob/corl2020/src/vgn/experiments/clutter_removal.py"
    click NBS "https://github.com/ethz-asl/vgn/blob/corl2020/notebooks/clutter_removal.ipynb"
    
    click RG1 "https://github.com/ethz-asl/vgn/blob/corl2020/scripts/panda_grasp.py"
    click RG2 "https://github.com/ethz-asl/vgn/blob/corl2020/launch/panda_grasp.launch"
    click RG3 "https://github.com/ethz-asl/vgn/blob/corl2020/src/vgn/utils/ros_utils.py"
    click RG4 "https://github.com/ethz-asl/vgn/blob/corl2020/config/panda.yaml"
    
    click CONF1 "https://github.com/ethz-asl/vgn/blob/corl2020/config/sim.rviz"
    click CONF2 "https://github.com/ethz-asl/vgn/blob/corl2020/config/gpd/ros_eigen_params.cfg"
    click LAUNCH "https://github.com/ethz-asl/vgn/blob/corl2020/launch/gpd.launch"
    click BUILD "https://github.com/ethz-asl/vgn/blob/corl2020/CMakeLists.txt,package.xml,setup.py,requirements.txt"
```
