How lecar-lab/16831-f26-hw works
This repository contains three iterations of robot-learning coursework code: behavioral cloning, policy-gradient learning, and actor-critic/DQN methods. The central runtime pattern is an RL trainer configured with an agent, which gathers experience, stores rollouts, and trains a policy or critic; HW3 adds frame-based DQN replay and environment monitoring. The README identifies the course but does not document product workflows. The graph therefore uses explicit source excerpts and SOURCE INDEX references for relationships and omits unsampled script wiring.
Main entry points: replay_buffer.py Rollout replay · monitor.py Environment monitor · rl_trainer.py HW1 trainer · dqn_agent.py DQN agent
Components 20
Training loops
- HW1 trainer
hw1/rob831/infrastructure/rl_trainer.py - HW2 trainer
hw2/rob831/infrastructure/rl_trainer.py
Learning agents
- Behavior cloning
hw1/rob831/agents/bc_agent.py - Policy gradient
hw2/rob831/agents/pg_agent.py - Actor critic
hw3/rob831/agents/ac_agent.py - DQN agent
hw3/rob831/agents/dqn_agent.py
Policies and critics
- MLP policy
hw1/rob831/policies/MLP_policy.py - Continuous critic
hw3/rob831/critics/bootstrapped_continuous_critic.py - DQN critic
hw3/rob831/critics/dqn_critic.py - Argmax policy
hw3/rob831/policies/argmax_policy.py
Experience and monitoring
- Rollout replay
hw1/rob831/infrastructure/replay_buffer.py - Trajectory utilities
hw1/rob831/infrastructure/utils.py - PyTorch utilities
hw1/rob831/infrastructure/pytorch_util.py - Training logger
hw1/rob831/infrastructure/logger.py - DQN replay utilities
hw3/rob831/infrastructure/dqn_utils.py - Environment monitor
hw3/rob831/infrastructure/monitor.py - Episode statistics
hw3/rob831/infrastructure/monitoring/stats_recorder.py - Video recorder
hw3/rob831/infrastructure/monitoring/video_recorder.py - Atomic file writes
hw3/rob831/infrastructure/monitoring/utils/atomic_write.py - JSON encoding
hw3/rob831/infrastructure/monitoring/utils/json_utils.py
Connections 22
- HW1 trainer → Training logger: creates
hw1/rob831/infrastructure/rl_trainer.py - HW1 trainer → Behavior cloning: instantiates configured agent
hw1/rob831/infrastructure/rl_trainer.py - HW2 trainer → Policy gradient: instantiates configured agent
hw2/rob831/infrastructure/rl_trainer.py - HW1 trainer → Rollout replay: adds collected rollouts
hw1/rob831/infrastructure/rl_trainer.py - HW2 trainer → Rollout replay: adds collected rollouts
hw2/rob831/infrastructure/rl_trainer.py - Behavior cloning → Rollout replay: uses
hw1/rob831/agents/bc_agent.py - Behavior cloning → MLP policy: uses
hw1/rob831/agents/bc_agent.py - Policy gradient → MLP policy: creates actor
hw2/rob831/agents/pg_agent.py - Policy gradient → Rollout replay: creates buffer
hw2/rob831/agents/pg_agent.py - Actor critic → Continuous critic: uses critic
hw3/rob831/agents/ac_agent.py - DQN agent → DQN critic: creates critic
hw3/rob831/agents/dqn_agent.py - DQN agent → Argmax policy: creates policy
hw3/rob831/agents/dqn_agent.py - DQN agent → DQN replay utilities: uses replay and schedule
hw3/rob831/agents/dqn_agent.py - MLP policy → PyTorch utilities: builds networks with
hw1/rob831/policies/MLP_policy.py - Rollout replay → Trajectory utilities: converts rollouts with
hw1/rob831/infrastructure/replay_buffer.py - DQN critic → PyTorch utilities: uses tensor utilities
hw3/rob831/critics/dqn_critic.py - Environment monitor → Episode statistics: uses
hw3/rob831/infrastructure/monitor.py - Environment monitor → Video recorder: uses
hw3/rob831/infrastructure/monitor.py - Environment monitor → Atomic file writes: uses
hw3/rob831/infrastructure/monitor.py - Environment monitor → JSON encoding: encodes with
hw3/rob831/infrastructure/monitor.py - Episode statistics → Atomic file writes: writes through
hw3/rob831/infrastructure/monitoring/stats_recorder.py - Episode statistics → JSON encoding: encodes with
hw3/rob831/infrastructure/monitoring/stats_recorder.py