How sunnywist/dpct works
Project Type and Purpose
• DPCT is a research-oriented reinforcement-learning pipeline built on PRIMAL2.
• It consists of three phases: environment generation, distributed training, and distributed simulation.
• It integrates Python controllers, C/C++/Cython path-planning libraries, and leverages CUDA/GPU and Ray/Horovod for distribution.
File Structure
Top-level folders and files reveal three major subsystems:
• DPCT/
• Core Python modules: Runner.py, TrainDriver.py, SimulateWithRayDriver.py, Worker.py
• Env generators: Env_Builder.py, Map_Generator.py, TestGenerator.py, TestingEnv.py
• RL components: ACNet.py, Observer_Builder.py, PRIMAL2_Env.py, PRIMAL2_Observer*.py
• Config abstractions under PyConfigs/ and configs/
• Bash scripts (train.sh, simulate.sh, both_train_simulate.sh, generate_envs.sh)
• DPCT/astarlib3 and DPCT/od_mstar3
• Native extensions for A* and OD-M* planning (C, Cython, C++).
• envs/
• Raw map generation tools (Generate_Raw_Map, Envs_Reseter, etc.)
• scripts and config directories: requirements.txt, configs/*.yaml, cpp_setup.sh.
README Insights
• Describes a conda/CUDA setup, installation of TensorFlow with Horovod, and bash-driven entry points: generate_envs.sh, train.sh, simulate.sh.
• References PRIMAL2, confirming a multi-agent RL use case.
Main Components
a. Environment Generation Layer
• Raw Map Generators (envs/*.py)
• DPCT/Map_Generator.py and Env_Builder.py
• Configs: generator.yaml, GeneratorConfig.py
• Outputs: Serialized environment definitions
b. Training Layer
• TrainDriver.py orchestrates multiple Worker.py processes (Ray/Horovod)
• ACNet.py (neural network model)
• Observer modules (PRIMAL2_Observer*)
• Configs: train.yaml, TrainConfig.py
• Underlying GPU cluster (CUDA) and distributed framework (Ray, Horovod)
c. Simulation Layer
• SimulateWithRayDriver.py and Runner.py
• Worker.py re-used for rollout collection
• Configs: simulate.yaml, SimulateConfig.py
d. Native Path-Planning Services
• astarlib3 (A* via Cython)
• od_mstar3 (OD-M* via C++/Cython)
• interface.py in od_mstar3 used by Python controllers
e. Configuration and Orchestration
• Centralized config directory (configs/) plus PyConfigs/ abstraction
• Bash scripts as high-level entry points
Relationships and Data Flow
• Start: User invokes bash script → loads configs → calls Python entry point
• Env generation writes map files → consumed by Training layer
• Training driver dispatches Workers → each calls Cython planners via astarlib3/od_mstar3 for step simulation → collects rollouts → updates ACNet model
• Simulation driver replays trained model in new environments → aggregates metrics
Architectural Patterns & Design Principles
• Pipeline pattern: sequential phases (Generate → Train → Simulate)
• Modular design: clear separation of concerns (env, model, planner, orchestration)
• Distributed actor model: Ray + Horovod for parallel workers
• Native acceleration: C/C++ extensions for compute-intensive planning
Final Advice
• Be exhaustive: represent every DPCT/*.py and envs/*.py cluster by function rather than listing each file.
• Orchestration and data flow clearly.