How xtrose29/rt-safe works
RT-SAFE is a benchmark for evaluating embodied agents’ navigation safety in a live Unreal environment. The principal workflow configures a task, constructs and runs the world, has an agent perceive observations and choose actions while traffic and hazards continue evolving, then records and evaluates outcomes. The repository also includes model transports, traffic simulation, experiment runners and result analysis, plus a website presenting benchmark results and examples. The graph follows the supplied tree and README; some implementation links are necessarily unverified because the source excerpts are partial.
Main entry points: world_manager.py World manager · rt_agent.py RT agent · run_experiment_matrix.py Experiment runners · traffic_controller.py SimWorld traffic
Components 21
Runtime orchestration
- World manager
manager/world_manager.py - RT communicator
base/rt_communicator.py - UnrealCV bridge
base/rt_unrealcv.py - Scenario runner
run_scenario.py
Agent decisions
- RT agent
base/rt_agent.py - Action space
base/rt_action_space.py - Model adapters
llm/rt_llm.py
Simulation world
- RT traffic system
base/rt_traffic_system.py - Traffic controller
base/rt_traffic_controller.py - SimWorld traffic
SimWorld/simworld/traffic/controller/traffic_controller.py - Traffic actors
SimWorld/simworld/agent/vehicle.py - Signals and junctions
SimWorld/simworld/traffic/base/intersection.py
Evaluation and presentation
- Safety evaluator
base/rt_evaluator.py - Experiment runners
evaluation/run_experiment_matrix.py - Evaluation metrics
utils/evaluation_metrics.py - Result aggregation
evaluation/aggregate_results.py - Rollout replay
evaluation/replay_recorded_rollout_video.py - Benchmark website
website/app/rtsafe.tsx
Other components
- Benchmark user
- Model provider
- Unreal runtime
Connections 20
- Benchmark user → Scenario runner: starts run
- Scenario runner → World manager: sets up world
- World manager → RT agent: coordinates agent
manager/world_manager.py - RT agent → Model adapters: requests decision
base/rt_agent.py - Model adapters → Model provider: queries model
- RT agent → Action space: uses actions
base/rt_agent.py - RT agent → Safety evaluator: records outcomes
base/rt_agent.py - World manager → RT communicator: uses bridge
manager/world_manager.py - RT communicator → UnrealCV bridge: delegates transport
base/rt_communicator.py - UnrealCV bridge → Unreal runtime: exchanges state
- World manager → Traffic controller: coordinates traffic
manager/world_manager.py - Traffic controller → SimWorld traffic: extends controller
base/rt_traffic_controller.py - SimWorld traffic → Traffic actors: manages actors
SimWorld/simworld/traffic/controller/traffic_controller.py - SimWorld traffic → Signals and junctions: coordinates signals
SimWorld/simworld/traffic/controller/traffic_controller.py - Benchmark user → Experiment runners: selects evaluation
- Experiment runners → World manager: runs scenarios
- Experiment runners → Evaluation metrics: scores rollouts
- Evaluation metrics → Result aggregation: feeds summaries
- Result aggregation → Benchmark website: publishes results
- Experiment runners → Rollout replay: supports replay