# hyuantan/lerobot_far architecture

> Architecture diagram of the GitHub repository hyuantan/lerobot_far, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hyuantan/lerobot_far
- Repository: https://github.com/hyuantan/lerobot_far

## Overview

LeRobot FAR extends LeRobot with experiments for remote VLA inference, latency assessment, and failure-aware retry on SO-101 hardware and LIBERO simulation. The principal workflow sends robot or simulation observations to a remote policy, receives actions, and executes them; with FAR enabled, runtime signals can trigger recovery and a retry from the recovered state. The README also describes data collection and model training capabilities. The graph combines those documented capabilities with the supplied tree and sampled code; unsampled server and policy internals are represented only at the documented subsystem level.

Main entry points: [`src/lerobot/async_inference/base_client.py`](https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/async_inference/base_client.py) (Async client core), [`src/lerobot/async_inference/smart_robot_client.py`](https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/async_inference/smart_robot_client.py) (SO-101 client), [`src/lerobot/async_inference/sim_test/sim_client.py`](https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/async_inference/sim_test/sim_client.py) (LIBERO client), [`src/lerobot/async_inference/sim_test/sim_smart_client.py`](https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/async_inference/sim_test/sim_smart_client.py) (Simulation recovery)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_remote["Remote inference"]
  node_robot_client["SO-101 client"]
  node_sim_client["LIBERO client<br/>[sim_client.py]"]
  node_grpc_contract["Inference RPC<br/>[services.proto]"]
  node_base_client["Async client core<br/>[base_client.py]"]
  node_policy_server["Policy server<br/>[policy_server.py]"]
end

subgraph group_recovery["Failure recovery"]
  node_far_monitor["Failure monitor"]
  node_far_recovery["Retry recovery"]
  node_sim_far["Simulation recovery"]
end

subgraph group_models["Policies and data"]
  node_policy["VLA policies"]
  node_processor["Policy processing<br/>[pipeline.py]"]
  node_recorder["Data recording<br/>[lerobot_record.py]"]
  node_datasets[("Dataset storage")]
  node_training["Model training<br/>[lerobot_train.py]"]
end

subgraph group_evaluation["Evaluation"]
  node_latency["Latency analysis<br/>[analyze_timing.py]"]
  node_libero_eval["LIBERO evaluation<br/>[eval_libero_rtc.py]"]
end

subgraph group_devices["Robot and simulation"]
  node_robot["SO-101 hardware<br/>[so_follower.py]"]
  node_camera["Camera capture"]
  node_libero_env["LIBERO environment<br/>[libero.py]"]
end

node_operator(("Operator"))
node_remote_server(("Remote server"))

node_operator -->|"starts run"| node_robot_client
node_operator -->|"starts evaluation"| node_sim_client
node_robot_client -->|"uses client core"| node_base_client
node_sim_client -->|"uses client core"| node_base_client
node_robot_client -->|"captures observations"| node_camera
node_robot_client -->|"executes actions"| node_robot
node_sim_client -->|"steps environment"| node_libero_env
node_base_client -->|"uses RPC"| node_grpc_contract
node_base_client -->|"sends observations"| node_remote_server
node_remote_server -->|"returns actions"| node_base_client
node_policy_server -->|"implements RPC"| node_grpc_contract
node_policy_server -->|"runs inference"| node_policy
node_policy -->|"uses processing"| node_processor
node_operator -->|"collects demonstrations"| node_recorder
node_recorder -->|"writes episodes"| node_datasets
node_training -->|"reads episodes"| node_datasets
node_training -->|"trains policy"| node_policy
node_robot_client -->|"checks execution"| node_far_monitor
node_far_monitor -->|"triggers retry"| node_far_recovery
node_far_recovery -->|"recovers state"| node_robot
node_far_recovery -->|"resamples inference"| node_base_client
node_sim_client -.->|"enables FAR mode"| node_sim_far
node_sim_far -->|"rewinds or resets"| node_libero_env
node_sim_far -->|"resamples inference"| node_base_client
node_robot_client -->|"records timing"| node_latency
node_sim_client -->|"evaluated by"| node_libero_eval

click node_robot_client "https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/async_inference/smart_robot_client.py"
click node_sim_client "https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/async_inference/sim_test/sim_client.py"
click node_grpc_contract "https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/transport/services.proto"
click node_base_client "https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/async_inference/base_client.py"
click node_policy_server "https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/async_inference/policy_server.py"
click node_policy "https://github.com/hyuantan/lerobot_far/tree/main/src/lerobot/policies"
click node_processor "https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/processor/pipeline.py"
click node_recorder "https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/scripts/lerobot_record.py"
click node_datasets "https://github.com/hyuantan/lerobot_far/tree/main/src/lerobot/datasets"
click node_training "https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/scripts/lerobot_train.py"
click node_far_monitor "https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/async_inference/smart_robot_client.py"
click node_far_recovery "https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/async_inference/smart_robot_client.py"
click node_sim_far "https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/async_inference/sim_test/sim_smart_client.py"
click node_latency "https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/async_inference/analyze_timing.py"
click node_libero_eval "https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/async_libero_inference/it_rtc/eval_libero_rtc.py"
click node_robot "https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/robots/so_follower/so_follower.py"
click node_camera "https://github.com/hyuantan/lerobot_far/tree/main/src/lerobot/cameras"
click node_libero_env "https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/envs/libero.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_robot_client,node_sim_client,node_grpc_contract,node_base_client,node_policy_server toneBlue
class node_far_monitor,node_far_recovery,node_sim_far toneAmber
class node_policy,node_processor,node_recorder,node_datasets,node_training,node_remote_server toneMint
class node_latency,node_libero_eval toneRose
class node_robot,node_camera,node_libero_env,node_operator toneIndigo
```

## Components

### Remote inference

- [SO-101 client](https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/async_inference/smart_robot_client.py): `src/lerobot/async_inference/smart_robot_client.py`
- [LIBERO client](https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/async_inference/sim_test/sim_client.py): `src/lerobot/async_inference/sim_test/sim_client.py`
- [Inference RPC](https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/transport/services.proto): `src/lerobot/transport/services.proto`
- [Async client core](https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/async_inference/base_client.py): `src/lerobot/async_inference/base_client.py`
- [Policy server](https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/async_inference/policy_server.py): `src/lerobot/async_inference/policy_server.py`

### Failure recovery

- [Failure monitor](https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/async_inference/smart_robot_client.py): `src/lerobot/async_inference/smart_robot_client.py`
- [Retry recovery](https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/async_inference/smart_robot_client.py): `src/lerobot/async_inference/smart_robot_client.py`
- [Simulation recovery](https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/async_inference/sim_test/sim_smart_client.py): `src/lerobot/async_inference/sim_test/sim_smart_client.py`

### Policies and data

- [VLA policies](https://github.com/hyuantan/lerobot_far/tree/main/src/lerobot/policies): `src/lerobot/policies`
- [Policy processing](https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/processor/pipeline.py): `src/lerobot/processor/pipeline.py`
- [Data recording](https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/scripts/lerobot_record.py): `src/lerobot/scripts/lerobot_record.py`
- [Dataset storage](https://github.com/hyuantan/lerobot_far/tree/main/src/lerobot/datasets): `src/lerobot/datasets`
- [Model training](https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/scripts/lerobot_train.py): `src/lerobot/scripts/lerobot_train.py`

### Evaluation

- [Latency analysis](https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/async_inference/analyze_timing.py): `src/lerobot/async_inference/analyze_timing.py`
- [LIBERO evaluation](https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/async_libero_inference/it_rtc/eval_libero_rtc.py): `src/lerobot/async_libero_inference/it_rtc/eval_libero_rtc.py`

### Robot and simulation

- [SO-101 hardware](https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/robots/so_follower/so_follower.py): `src/lerobot/robots/so_follower/so_follower.py`
- [Camera capture](https://github.com/hyuantan/lerobot_far/tree/main/src/lerobot/cameras): `src/lerobot/cameras`
- [LIBERO environment](https://github.com/hyuantan/lerobot_far/blob/main/src/lerobot/envs/libero.py): `src/lerobot/envs/libero.py`

### Other components

- **Operator**
- **Remote server**

## Connections

- Operator → SO-101 client: starts run
- Operator → LIBERO client: starts evaluation
- SO-101 client → Async client core: uses client core
- LIBERO client → Async client core: uses client core
- SO-101 client → Camera capture: captures observations
- SO-101 client → SO-101 hardware: executes actions
- LIBERO client → LIBERO environment: steps environment
- Async client core → Inference RPC: uses RPC
- Async client core → Remote server: sends observations
- Remote server → Async client core: returns actions
- Policy server → Inference RPC: implements RPC
- Policy server → VLA policies: runs inference
- VLA policies → Policy processing: uses processing
- Operator → Data recording: collects demonstrations
- Data recording → Dataset storage: writes episodes
- Model training → Dataset storage: reads episodes
- Model training → VLA policies: trains policy
- SO-101 client → Failure monitor: checks execution
- Failure monitor → Retry recovery: triggers retry
- Retry recovery → SO-101 hardware: recovers state
- Retry recovery → Async client core: resamples inference
- LIBERO client → Simulation recovery: enables FAR mode
- Simulation recovery → LIBERO environment: rewinds or resets
- Simulation recovery → Async client core: resamples inference
- SO-101 client → Latency analysis: records timing
- LIBERO client → LIBERO evaluation: evaluated by