How larocs/sac_uav works
1. This repository implements a Soft Actor-Critic (SAC) reinforcement-learning pipeline for low-level UAV control. The core goal is to train a policy that drives a drone from a randomized старт state back to a fixed target at the origin.
2. The main architectural boundary is between the Python RL code and the external robotics simulation stack. The repo’s code talks to PyRep, which in turn drives CoppeliaSim. That simulator is the environment backend; this repo does not embed physics itself.
3. The training entry point is main.py. It is the central orchestration script for the SAC loop: initialize the environment, collect transitions from the simulator, update the agent, and periodically persist policies. The presence of training.sh suggests the canonical training path is shell-driven rather than a library API.
4. The evaluation path is separated from training. evaluate.py and evaluate_container.py are dedicated inference/rollout entry points, with matching shell wrappers evaluate.sh and evaluate_container.sh. This separation implies the trained policy is loaded from disk and executed without learning updates.
5. The model definitions live under networks/, especially networks/structures.py. This is the policy/value-function subsystem: actor and critic network structure, and likely SAC-specific modules used by the training loop. This is the main “learned function approximation” boundary in the repo.
6. Shared support code is in common/, with common/utils.py acting as a utility layer. This is where reusable helpers, bookkeeping, tensor handling, or environment/action utilities likely sit, keeping the main loop and network code cleaner.
7. The trained artifacts are stored in saved_policies/, which contains .pt files for pretrained SAC policies. This directory is the handoff point between training and evaluation: training produces policy checkpoints, evaluation consumes them.
8. Containerization is a first-class operational concern. Dockerfile, Makefile, and the *_container.sh scripts indicate a reproducible execution path for both training and evaluation inside a Docker image, useful for cluster runs or environments where CoppeliaSim/PyRep dependencies are hard to install manually.
9. The repo also includes visualization and analysis outputs rather than runtime code. notebooks/ contains notebooks for reproducing figures and trajectory plots; assets/ stores static diagrams used by the README. These are downstream analysis/communication artifacts, not part of the control loop.
10. The overall data flow is: simulator state/observations from CoppeliaSim/PyRep → SAC agent code in main.py → actor/critic modules in networks/ → actions back to the simulator → logged trajectories/rewards → checkpoints in saved_policies/ and figures in notebooks/. That loop is the central architecture to capture in a diagram.
11. The repo is intentionally lightweight in Python package structure: requirements.txt declares dependencies, while README instructions mention external prerequisites such as CoppeliaSim and PyRep. Those are material to the architecture because the project depends on an external simulator runtime rather than a pure-Python environment.
12. In short, the repository is organized around one learning system with three clear layers: simulation backend, SAC training/evaluation logic, and artifact/reproducibility tooling. The key diagram should show the simulator as an external service, the Python agent as the control and learning core, and checkpoints/notebooks/Docker as supporting outputs and execution paths.
Main entry points: main.py Train loop · evaluate.py Evaluate · Dockerfile Dockerfile · evaluate_container.py Eval container
Components 14
SAC pipeline
Training and evaluation entry points plus the learning modules they orchestrate.
- Train loop · Central SAC training entry point: environment setup, rollout collection, updates, and checkpointing.
main.py - Evaluate · Standalone rollout/inference entry point for loaded policies.
evaluate.py - Networks · Actor/critic and SAC-specific network structure definitions.
networks/structures.py - Common utils · Reusable helper layer for bookkeeping and environment/action support.
common/utils.py
Ops and artifacts
Reproducibility scripts, container execution, and generated outputs.
- Eval container · Container-oriented evaluation entry point.
evaluate_container.py - Train script · Canonical shell wrapper for launching training.
training.sh - Eval script · Shell wrapper for standard evaluation runs.
evaluate.sh - Eval container · Shell wrapper for containerized evaluation runs.
evaluate_container.sh - Checkpoints · Persisted SAC policy weights consumed by evaluation.
saved_policies - Notebooks · Trajectory analysis and figure reproduction notebooks.
notebooks - Figures · Static plots exported by notebooks for documentation and analysis.
notebooks/figures - Dockerfile · Container image definition for reproducible execution.
Dockerfile - Makefile · Build and execution convenience targets.
Makefile
Simulator backend
External robotics stack driven through PyRep and CoppeliaSim.
- PyRep/CoppeliaSim · External robotics simulation backend providing state and physics.
Connections 17
- Train script → Train loop: launches · Shell wrapper starts the training orchestrator.
- Train loop → PyRep/CoppeliaSim: steps env · Training collects transitions by interacting with the simulator backend.
- Train loop → Networks: updates · Training uses actor/critic networks for policy and value learning.
- Train loop → Common utils: uses · Shared helper functions support rollout, logging, and training bookkeeping.
- Train loop → Checkpoints: saves · Training periodically writes policy checkpoints.
- Eval script → Evaluate: launches · Shell wrapper starts standard evaluation.
- Eval container → Eval container: launches · Shell wrapper starts containerized evaluation.
- Eval container → Evaluate: delegates · Container entry point reuses the evaluation flow.
- Evaluate → PyRep/CoppeliaSim: rolls out · Evaluation runs the trained policy in the simulator backend.
- Evaluate → Networks: loads · Evaluation instantiates the learned policy/value structure.
- Evaluate → Checkpoints: reads · Evaluation loads saved policy checkpoints from disk.
- Evaluate → Common utils: uses · Shared helpers support inference and result handling.
- Dockerfile → Train loop: packages · Container image supports reproducible training runs.
- Dockerfile → Eval container: packages · Container image supports reproducible evaluation runs.
- Makefile → Dockerfile: drives · Make targets likely wrap image/build/runtime commands.
- Notebooks → Figures: produces · Notebooks generate analysis figures and plots.
- Notebooks → Checkpoints: analyzes · Notebook workflows consume trained checkpoints for trajectory studies.