# gjcliff/frankateleop architecture

> Architecture diagram of the GitHub repository gjcliff/frankateleop, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gjcliff/frankateleop
- Repository: https://github.com/gjcliff/frankateleop

## Overview

##### Project type and purpose
- This is a full-stack robotics teleoperation system built on ROS2 Iron.
- Its goal is to let a user control an Emika Franka Panda arm in real or simulated hardware via hand-tracking gestures using MediaPipe and MoveIt Servo.

##### File structure
Top-level you see five ROS2 packages plus workspace files:
- handcv — Python node for camera input, MediaPipe hand tracking & gesture recognition
- cv_franka_bridge — Python node that subscribes to handcv output and translates it into motion commands
- franka_teleop — C++ node using MoveIt Servo to drive the robot via geometry increments
- numsr_franka_moveit_config — MoveIt configuration files (URDF/xacro, controllers, planning params, RViz setups)
- hand_interfaces — custom ROS2 msgs and srvs definitions used by the other nodes
Each package has its own launch files, configs, resource files (meshes, RViz), and tests.

##### README for additional insights
- The README clarifies the data flow: RealSense Camera → handcv → cv_franka_bridge → franka_teleop → Franka Robot
- It explains external dependencies (MediaPipe, RealSense-ROS, MoveIt) and how to run on real vs. simulated hardware.
- Gestures (thumbs up/down, open/close fist) map to start/stop, shutdown, gripper open/close.

##### Elements
- Clear labels: node names, topic/service names, package names
- Blue for perception nodes
- Green for bridging
- Orange for control
- Gray for hardware
- Yellow for configuration files/params
- External dependencies: MediaPipe library, MoveIt, RealSense driver

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Launchers
    subgraph "Launch Assemblies"
        direction TB
        LaunchCamera["camera.launch.py"]:::visualization
        LaunchBridge["integrate_servo.launch.py"]:::visualization
        LaunchControl["franka_servo.launch.py"]:::visualization
        LaunchMoveIt["demo.launch.py"]:::visualization
    end

    %% Layers
    subgraph "Perception Layer"
        direction TB
        HandCV["handcv\n(Python node)\nInputs: /camera/image\nOutputs: FingerData, Pinch"]:::perception
    end

    subgraph "Bridge Layer"
        direction TB
        CVBridge["cv_franka_bridge\n(Python node)\nInputs: FingerData, Pinch\nOutputs: geometry commands, HandPose.srv"]:::bridge
    end

    subgraph "Control Layer"
        direction TB
        FrankaTeleop["franka_teleop\n(C++ MoveIt Servo)\nInputs: geometry cmds, HandPose.srv\nOutputs: hardware iface cmds, robot_state"]:::control
    end

    subgraph "Configuration"
        direction TB
        MoveItConfig["numsr_franka_moveit_config\n(URDF, params)"]:::config
    end

    subgraph "Interfaces"
        direction TB
        Interfaces["hand_interfaces\n(FingerData.msg, Pinch.msg, HandPose.srv)"]:::interfaces
    end

    subgraph "Hardware"
        direction TB
        Camera["RealSense Camera"]:::hardware
        FrankaHW["Franka Panda Robot"]:::hardware
    end

    subgraph "Visualization & Debugging"
        direction TB
        RViz["RViz2\n(subscribes to robot_state, planning_scene)"]:::visualization
    end

    %% Data Flow
    LaunchCamera --> HandCV
    LaunchBridge --> CVBridge
    LaunchControl --> FrankaTeleop
    LaunchMoveIt --> MoveItConfig

    Camera -->|"sensor_msgs/Image"| HandCV
    HandCV -->|"hand_interfaces/FingerData, Pinch"| CVBridge
    CVBridge -->|"geometry_msgs cmds\nHandPose.srv"| FrankaTeleop
    MoveItConfig -->|"URDF, params"| FrankaTeleop
    FrankaTeleop -->|"commands"| FrankaHW
    FrankaTeleop -->|"robot_state"| RViz
    MoveItConfig -->|"configs"| RViz
    CVBridge -->|"debug topics"| RViz

    %% Click Events
    click HandCV "https://github.com/gjcliff/frankateleop/blob/main/handcv/handcv/handcv.py"
    click HandCV "https://github.com/gjcliff/frankateleop/tree/main/handcv/"
    click HandCV "https://github.com/gjcliff/frankateleop/blob/main/handcv/handcv/mediapipehelper.py"
    click HandCV "https://github.com/gjcliff/frankateleop/blob/main/handcv/config/gesture_recognizer.task"
    click LaunchCamera "https://github.com/gjcliff/frankateleop/blob/main/handcv/launch/camera.launch.py"

    click CVBridge "https://github.com/gjcliff/frankateleop/blob/main/cv_franka_bridge/cv_franka_bridge/cv_franka_bridge.py"
    click CVBridge "https://github.com/gjcliff/frankateleop/tree/main/cv_franka_bridge/"
    click CVBridge "https://github.com/gjcliff/frankateleop/blob/main/cv_franka_bridge/config/integrate_servo.rviz"
    click LaunchBridge "https://github.com/gjcliff/frankateleop/blob/main/cv_franka_bridge/launch/integrate_servo.launch.py"

    click FrankaTeleop "https://github.com/gjcliff/frankateleop/blob/main/franka_teleop/src/franka_servo_real.cpp"
    click FrankaTeleop "https://github.com/gjcliff/frankateleop/blob/main/franka_teleop/src/franka_servo_fake.cpp"
    click FrankaTeleop "https://github.com/gjcliff/frankateleop/blob/main/franka_teleop/srv/PlanPath.srv"
    click FrankaTeleop "https://github.com/gjcliff/frankateleop/blob/main/franka_teleop/launch/franka_servo.launch.py"

    click MoveItConfig "https://github.com/gjcliff/frankateleop/tree/main/numsr_franka_moveit_config/"
    click MoveItConfig "https://github.com/gjcliff/frankateleop/blob/main/numsr_franka_moveit_config/config/panda_arm_real.urdf.xacro"
    click LaunchMoveIt "https://github.com/gjcliff/frankateleop/blob/main/numsr_franka_moveit_config/launch/demo.launch.py"

    click Interfaces "https://github.com/gjcliff/frankateleop/blob/main/hand_interfaces/msg/FingerData.msg"
    click Interfaces "https://github.com/gjcliff/frankateleop/blob/main/hand_interfaces/msg/Pinch.msg"
    click Interfaces "https://github.com/gjcliff/frankateleop/blob/main/hand_interfaces/srv/HandPose.srv"

    %% Styles
    classDef perception fill:#D0E8FF,stroke:#0366d6;
    classDef bridge fill:#DFFFE0,stroke:#28a745;
    classDef control fill:#FFE2B0,stroke:#d58512;
    classDef config fill:#FFF5B1,stroke:#f0ad4e;
    classDef interfaces fill:#EAD1DC,stroke:#c7437e;
    classDef hardware fill:#E0E0E0,stroke:#6c757d;
    classDef visualization fill:#FDE2EC,stroke:#d6336c;
```
