How mingrui-yu/sharpa_rl_unilab works
This package provides a UniLab Manager-Based Sharpa Wave task and a policy lifecycle: configure/train a PPO, APPO, or FlashSAC teacher, optionally distill it into a student, then evaluate checkpoints or record playback. Task manager terms define physical actions, observations, reset/randomization, rewards and episode endings; training adapters connect the task to algorithm runners and shared HORA models. README documents these capabilities, but sampled excerpts do not establish every manager-term or runner call, so uncertain wiring is omitted. UniLab simulation and manager lifecycle remain external.
Main entry points: teacher_env.py Observation and reset adapter · teacher_runtime.py Teacher runtime · cli.py Task CLI · playback.py Policy playback
Components 27
Manipulation task
- Observation and reset adapter
src/sharpa_rl_unilab/tasks/sharpa_inhand/teacher_env.py - Observation protocol
src/sharpa_rl_unilab/tasks/sharpa_inhand/protocol.py - Incremental actions
src/sharpa_rl_unilab/tasks/sharpa_inhand/terms/action.py - Tactile observations
src/sharpa_rl_unilab/tasks/sharpa_inhand/terms/observation.py - Grasp reset
src/sharpa_rl_unilab/tasks/sharpa_inhand/terms/reset.py - Physical randomization
src/sharpa_rl_unilab/tasks/sharpa_inhand/terms/randomization.py - Rewards and termination
src/sharpa_rl_unilab/tasks/sharpa_inhand/terms/rewards.py - Grasp registry
src/sharpa_rl_unilab/tasks/sharpa_inhand/grasp_registry.py - Rotation registry
src/sharpa_rl_unilab/tasks/sharpa_inhand/rotation_registry.py
Policy learning
- PPO runtime
src/sharpa_rl_unilab/training/ppo_runtime.py - APPO runtime
src/sharpa_rl_unilab/training/appo_runtime.py - FlashSAC runtime
src/sharpa_rl_unilab/training/flashsac_runtime.py - Shared HORA models
src/sharpa_rl_unilab/algos/hora/models.py - Policy distribution
src/sharpa_rl_unilab/algos/hora/distribution.py - PPO/APPO adapters
src/sharpa_rl_unilab/algos/hora/on_policy.py
Training lifecycle
- Task CLI
src/sharpa_rl_unilab/cli.py - Task configuration
src/sharpa_rl_unilab/tasks/sharpa_inhand/config.py - Teacher runtime
src/sharpa_rl_unilab/training/teacher_runtime.py - Student distillation
src/sharpa_rl_unilab/training/student_runtime.py - Checkpoint handling
src/sharpa_rl_unilab/training/checkpoints.py - Algorithm comparison
src/sharpa_rl_unilab/tools/compare_rl_algo.py
Evaluation and playback
- Checkpoint evaluation
src/sharpa_rl_unilab/training/evaluation.py - Policy playback
src/sharpa_rl_unilab/training/playback.py - Video rendering
src/sharpa_rl_unilab/training/rendering.py
Other components
- Engineer
- UniLab manager and simulator
- MuJoCo backend
Connections 39
- Engineer → Task CLI: selects run
- Task CLI → Task configuration: composes config
- Task CLI → Teacher runtime: starts training
- Task CLI → Checkpoint evaluation: evaluates checkpoint
- Task CLI → Policy playback: plays checkpoint
- Task configuration → UniLab manager and simulator: configures task
- UniLab manager and simulator → Observation and reset adapter: creates environment
- Observation and reset adapter → Observation protocol: validates layout
- Observation and reset adapter → Tactile observations: reads observations
- Tactile observations → Incremental actions: resolves action term
- UniLab manager and simulator → Incremental actions: dispatches actions
- UniLab manager and simulator → Grasp reset: dispatches resets
- UniLab manager and simulator → Physical randomization: dispatches events
- UniLab manager and simulator → Rewards and termination: computes episode outcome
- Grasp reset → Grasp registry: uses grasp cache
- Task configuration → Rotation registry: selects rotation task
- Teacher runtime → Observation and reset adapter: constructs environment
- Teacher runtime → PPO runtime: runs PPO
- Teacher runtime → APPO runtime: runs APPO
- Teacher runtime → FlashSAC runtime: runs FlashSAC
- PPO runtime → Shared HORA models: uses policy
- APPO runtime → Shared HORA models: uses policy
- FlashSAC runtime → Shared HORA models: uses policy
- PPO runtime → PPO/APPO adapters: uses adapter
- APPO runtime → PPO/APPO adapters: uses adapter
- PPO/APPO adapters → Policy distribution: computes distribution
- Teacher runtime → Checkpoint handling: saves teacher
- Student distillation → Checkpoint handling: loads and saves
- Student distillation → Observation and reset adapter: collects rollouts
- Student distillation → Shared HORA models: builds student actor
- Checkpoint evaluation → Checkpoint handling: loads checkpoint
- Checkpoint evaluation → Observation and reset adapter: runs episodes
- Policy playback → Checkpoint handling: loads checkpoint
- Policy playback → Observation and reset adapter: runs policy
- Policy playback → Video rendering: records video
- Algorithm comparison → Task CLI: launches runs
- Algorithm comparison → Student distillation: launches distillation
- Algorithm comparison → Checkpoint evaluation: launches evaluation
- UniLab manager and simulator → MuJoCo backend: uses backend