# mczhi/dtpp architecture

> Architecture diagram of the GitHub repository mczhi/dtpp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mczhi/dtpp
- Repository: https://github.com/mczhi/dtpp

## Overview

##### Main Components
- Data Processing & Ingestion:
- “data_process.py” module along with “data_utils.py” and “data_process.py”. These scripts process the raw nuPlan dataset (NuScenes-like) and maps, preparing the data for both training and testing.
- Data input points (e.g., paths to nuPlan dataset & maps as documented in the README).

- Model Training:
- “train.py” and “train_utils.py”, which handle the training of the DTPP model using PyTorch.
- Loading or saving of the pre-trained model (base_model.pth).

- Prediction and Planning Subsystems:
- Modules such as “planner.py”, “planner_utils.py”, “path_planner.py”, “trajectory_tree_planner.py”, “scenario_tree_prediction.py”, and “prediction_modules.py”. These components:
° Handle the tree-based planning logic and prediction function.
° Include optimization or evaluation strategies (joint conditional prediction and cost evaluation).
- Specialized modules like “spline_planner.py” and “bezier_path.py” for generating and smoothing trajectories.

- Observation Processing:
- “obs_adapter.py”, which likely serves as an adapter for fusing sensor data or external observations into the planning workflow.

- Common and Utility Functions:
- “common_utils.py” as responsible for shared utilities used throughout the project.

- Testing and Simulation:
- “test.py” and include “test_scenario.yaml” for testing the planning framework, particularly in nuPlan simulation scenarios.

##### External Dependencies and Environment Setup
- The nuPlan dataset and devkit (external dependency modules) which feed into the data processing and testing modules.
- The use of PyTorch (with version specifics) which is central in the training and inference pipeline.
- Environment variables (e.g., NUPLAN_EXP_ROOT) can be annotated in a “Configuration/Environment” section.

- Data flow from the nuPlan dataset/maps into the Data Processing module.
- Processed data moving into the Training module, where the PyTorch-based DTPP model is trained.
- The trained model (stored as base_model.pth) being utilized by the Prediction and Planning modules.
- Data flow from these modules into the Testing/Simulation module, which interacts with nuPlan scenarios.

##### Architectural Patterns
- Modular Decomposition:
- Distinct modules (data, training, planning, prediction) to clearly delineate responsibilities.
- Integration with External Systems:
- Separate components that interface with the nuPlan devkit and dataset.
- End-to-End Pipeline:

##### Putting It All Together
° Input Data (nuPlan dataset, map files) → Data Processing modules → Training module (with PyTorch and environment setup) → Model storage (base_model.pth) → Prediction & Planning modules (including separate nodes for bezier path, spline, and tree-based planning) → Testing and simulation scenarios.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Input Data
    extInput["\"nuPlan Dataset & Maps\""]:::external

    %% Data Processing & Ingestion
    subgraph "Data Processing & Ingestion"
        DP1["\"data_process.py\""]:::processing
        DP2["\"data_utils.py\""]:::processing
        READ["\"README.md\""]:::config
    end

    %% Model Training
    subgraph "Model Training"
        T1["\"train.py\""]:::training
        T2["\"train_utils.py\""]:::training
        BASE["\"base_model.pth\""]:::storage
        PT["\"PyTorch\""]:::external
    end

    %% Prediction & Planning Subsystems
    subgraph "Prediction & Planning"
        PP1["\"planner.py\""]:::planning
        PP2["\"planner_utils.py\""]:::planning
        PP3["\"path_planner.py\""]:::planning
        PP4["\"trajectory_tree_planner.py\""]:::planning
        PP5["\"scenario_tree_prediction.py\""]:::planning
        PP6["\"prediction_modules.py\""]:::planning
        PP7["\"bezier_path.py\""]:::planning
        PP8["\"spline_planner.py\""]:::planning
    end

    %% Observation Processing
    subgraph "Observation Processing"
        OBS["\"obs_adapter.py\""]:::processing
    end

    %% Common and Utility Functions
    subgraph "Common Utilities"
        CU["\"common_utils.py\""]:::utility
    end

    %% Testing and Simulation
    subgraph "Testing & Simulation"
        TS1["\"test.py\""]:::testing
        TS2["\"test_scenario.yaml\""]:::testing
    end

    %% Configuration / Environment
    subgraph "Configuration/Environment"
        ENV["\"NUPLAN_EXP_ROOT\""]:::config
    end

    %% Data Flow
    extInput -->|"feeds"| DP1
    extInput -->|"feeds"| DP2
    DP1 -->|"readsConfig"| READ
    DP2 -->|"readsConfig"| READ

    %% Data Processing to Training
    DP1 -->|"processedData"| T1
    DP2 -->|"processedData"| T1
    READ -->|"configures"| T1
    ENV -->|"setsEnv"| T1

    %% Training Process
    T1 -->|"trainsModel"| T2
    T2 -->|"savesModel"| BASE
    T1 ---|"uses"| PT
    T2 ---|"uses"| PT

    %% Training to Prediction & Planning
    BASE -->|"suppliesModel"| PP1

    %% Prediction & Planning internal calls
    PP1 -->|"utilizes"| PP2
    PP1 -->|"calls"| PP3
    PP1 -->|"calls"| PP4
    PP1 -->|"calls"| PP5
    PP1 -->|"calls"| PP6
    PP1 -->|"calls"| PP7
    PP1 -->|"calls"| PP8

    %% Observation Processing feeds Prediction
    OBS -->|"suppliesObs"| PP1

    %% Prediction & Planning to Testing & Simulation
    PP1 -->|"validatedBy"| TS1
    TS1 -->|"usesScenario"| TS2
    extInput -->|"usedInTests"| TS1

    %% Common Utilities used by multiple modules (dashed connections)
    CU -.-> DP1
    CU -.-> T1
    CU -.-> PP1
    CU -.-> TS1

    %% Click Events
    click DP1 "https://github.com/mczhi/dtpp/blob/main/data_process.py"
    click DP2 "https://github.com/mczhi/dtpp/blob/main/data_utils.py"
    click READ "https://github.com/mczhi/dtpp/blob/main/README.md"
    click T1 "https://github.com/mczhi/dtpp/blob/main/train.py"
    click T2 "https://github.com/mczhi/dtpp/blob/main/train_utils.py"
    click BASE "https://github.com/mczhi/dtpp/blob/main/base_model.pth"
    click PP1 "https://github.com/mczhi/dtpp/blob/main/planner.py"
    click PP2 "https://github.com/mczhi/dtpp/blob/main/planner_utils.py"
    click PP3 "https://github.com/mczhi/dtpp/blob/main/path_planner.py"
    click PP4 "https://github.com/mczhi/dtpp/blob/main/trajectory_tree_planner.py"
    click PP5 "https://github.com/mczhi/dtpp/blob/main/scenario_tree_prediction.py"
    click PP6 "https://github.com/mczhi/dtpp/blob/main/prediction_modules.py"
    click PP7 "https://github.com/mczhi/dtpp/blob/main/bezier_path.py"
    click PP8 "https://github.com/mczhi/dtpp/blob/main/spline_planner.py"
    click OBS "https://github.com/mczhi/dtpp/blob/main/obs_adapter.py"
    click CU "https://github.com/mczhi/dtpp/blob/main/common_utils.py"
    click TS1 "https://github.com/mczhi/dtpp/blob/main/test.py"
    click TS2 "https://github.com/mczhi/dtpp/blob/main/test_scenario.yaml"

    %% Styling Classes
    classDef external fill:#d4f0ff,stroke:#333,stroke-width:2px;
    classDef processing fill:#ffefc7,stroke:#333,stroke-width:2px;
    classDef training fill:#c7ffc7,stroke:#333,stroke-width:2px;
    classDef planning fill:#ffbdbd,stroke:#333,stroke-width:2px;
    classDef storage fill:#f7c6c7,stroke:#333,stroke-width:2px;
    classDef testing fill:#e2caff,stroke:#333,stroke-width:2px;
    classDef config fill:#ffe3e3,stroke:#333,stroke-width:2px;
    classDef utility fill:#e0e0e0,stroke:#333,stroke-width:2px;
```
