# rainmaker22/smart architecture

> Architecture diagram of the GitHub repository rainmaker22/smart, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rainmaker22/smart
- Repository: https://github.com/rainmaker22/smart

## Overview

##### Project Type and Purpose
- This is not a traditional full-stack web application but instead an advanced deep learning framework for autonomous driving motion generation. The project implements SMART – a scalable multi-agent, real-time motion generation system based on next-token predictions. It covers data preprocessing, model definition, training/evaluation, and integration of various modules (e.g., layers, metrics, tokens, and transforms).

##### Break Down the File Structure
- Top-level directories/files:
- configs: Contains YAML configuration files for training (train_scalable.yaml) and validation (validation_scalable.yaml), illustrating configurable hyperparameters and possibly training/evaluation settings.
- data: Includes raw or demo data in pickle format and a directory structure that hints to how data is organized.
- scripts: Contains supplementary scripts such as installation helpers for PyG and specific tools like traj_clustering.
- smart: This is the main package and is split into several sub-modules
- datamodules and datasets for handling data input and serving batches;
- layers include implementations of attention layers, Fourier embeddings, and MLP layers for building the neural network;
- model contains the main SMART model implementation;
- modules potentially orchestrate the decoding for agents and maps;
- tokens house pre-generated tokens (likely from clustering or quantization) used in the discrete sequence modeling;
- transforms, metrics, and utils provide support for data transformation, performance measurement, and general utility functions.
- train.py and val.py: Orchestrate the training and evaluation loops using the architectural components.
- data_preprocess.py: An independent script for converting raw Waymo data to preprocessed tokens.
- Environment and dependency files (environment.yml, requirements.txt) ensure the right runtime and package versions.

##### Main Components to Capture
a. Data Input & Preprocessing:
- Raw Waymo Open Motion Dataset (organized externally as described in README under data/waymo).
- Data Preprocessing module, triggered via data_preprocess.py or smart/preprocess/preprocess.py, converts raw data into preprocessed tokens & structured formats.
- Preprocessed output stored under data/ (or data/waymo_processed once generated).

b. Data Management & Loading:
- DataModules (smart/datamodules) and Dataset classes (smart/datasets) load and serve data for training and evaluation.
- Configuration specified by YAML files in the configs directory directs data handling (e.g., scalable settings).

c. Network Architecture:
- Model Core: Located in smart/model (smart.py) aggregates different components for motion generation.
- Layers: smart/layers (attention_layer, fourier_embedding, mlp_layer) form the building blocks of the model.
- Modules: smart/modules (agent_decoder, map_decoder, smart_decoder) handle specific decoding tasks from the model’s output.
- Tokens: smart/tokens represent how map and trajectory data are discretized into sequence tokens.
- Transforms: smart/transforms (target_builder) supports target preparation for training.

d. Training and Evaluation Pipeline:
- Training script (train.py) integrates the model, data modules, and configuration to run the training loop.
- Evaluation script (val.py) uses the trained model and involves metrics (found in smart/metrics) such as min_ade, min_fde, next_token_cls, average_meter, etc.
- Additional scripts and utilities (e.g., smart/utils) provide logging, weight initialization, cluster reading, and geometry processing to support the core pipeline.

e. Configuration and Environment Management:
- The configs directory provides specific settings for training/validation.
- Environment files ensure that dependencies (e.g., PyTorch, PyG, PyTorch Lightning) are properly installed.

##### Relationships and Data Flow
- Then, branch the flow into “Data Management” or “Data Loading” (smart/datamodules and smart/datasets), which uses configuration from the configs folder.
– “Layers” (attention, MLP, Fourier embeddings)
– “Modules/Decoders” (agent, map, overall SMART decoder)
– “Tokens” (handling discretized trajectory and map data)
- Link the “Data Loading” step to the “Training Pipeline” that invokes train.py.
- External scripts/utilities as supplementary components that support installation (scripts/install_pyg.sh) and specialized processes (scripts/traj_clstering.py).

##### Architectural Patterns and Guiding Principles
- The system uses modular design principles – clear separation of data handling, model computation, and evaluation.
- Dependency injection via YAML configs allows ease of scalability and adjustments according to model and dataset variations.
- The discrete token-based representation (a unique pattern) is central to converting continuous map/trajectory data into sequence tokens, which is reflected in the specialized directories (smart/tokens, transforms).

## Diagram (Mermaid)

```mermaid
graph TD
    %% Data Layer
    DS["Data Source (Waymo Dataset)"]:::data
    DP["Data Preprocessing"]:::data
    DS --> DP

    %% Data Management & Loading
    subgraph "Data Management & Loading"
        DM1["Data Modules (smart/datamodules)"]:::data
        DM2["Datasets (smart/datasets)"]:::data
    end
    DP --> DM1
    DP --> DM2

    %% Configuration & Environment Support for Data Management
    subgraph "Configuration Management"
        CONF_train["Train Config (configs/train_scalable.yaml)"]:::config
        CONF_val["Validation Config (configs/validation_scalable.yaml)"]:::config
    end
    CONF_train --> DM1
    CONF_val --> DM2

    subgraph "Environment & Dependencies"
        ENV_yml["environment.yml"]:::config
        ENV_req["requirements.txt"]:::config
    end
    ENV_yml --> DM1
    ENV_req --> DM2

    %% Training Pipeline
    TP["Training Pipeline (train.py)"]:::model
    DM1 --> TP
    DM2 --> TP

    %% Model Architecture
    subgraph "Model Architecture"
        MC["Model Core (smart/model)"]:::model
        %% Layers Subgraph
        subgraph "Layers"
            AL["Attention Layer (smart/layers)"]:::model
            FE["Fourier Embedding (smart/layers)"]:::model
            MLP["MLP Layer (smart/layers)"]:::model
        end
        %% Modules/Decoders Subgraph
        subgraph "Modules/Decoders"
            AD["Agent Decoder (smart/modules)"]:::model
            MAPD["Map Decoder (smart/modules)"]:::model
            SD["SMART Decoder (smart/modules)"]:::model
        end
        %% Tokens Handling Subgraph
        subgraph "Tokens Handling"
            CT["Cluster Tokens (smart/tokens)"]:::model
            MT["Map Tokens (smart/tokens)"]:::model
        end
        TR["Transforms (smart/transforms)"]:::model
    end
    TP --> MC
    TP --> AL
    TP --> AD
    TP --> CT
    TP --> TR

    %% Metrics and Evaluation Pipeline
    subgraph "Metrics"
        mADE["min_ade (smart/metrics)"]:::model
        mFDE["min_fde (smart/metrics)"]:::model
        NT["next_token_cls (smart/metrics)"]:::model
        AM["average_meter (smart/metrics)"]:::model
        MUtils["metrics utils (smart/metrics)"]:::model
    end
    MC --> mADE
    MC --> mFDE
    MC --> NT
    MC --> AM
    MC --> MUtils

    EP["Evaluation Pipeline (val.py)"]:::model
    mADE --> EP
    mFDE --> EP
    NT --> EP
    AM --> EP
    MUtils --> EP

    %% Supplementary Scripts/Utilities
    subgraph "Supplementary Scripts/Utilities"
        SS_IPYG["Install PyG (scripts/install_pyg.sh)"]:::util
        SS_TC["Trajectory Clustering (scripts/traj_clustering.py)"]:::util
        SS_UTILS["Utilities (smart/utils)"]:::util
    end
    SS_IPYG --> DP
    SS_TC --> DP
    SS_UTILS --> MC

    %% Click Events for Component Mapping
    click DP "https://github.com/rainmaker22/smart/blob/main/data_preprocess.py"
    click DM1 "https://github.com/rainmaker22/smart/blob/main/smart/datamodules/scalable_datamodule.py"
    click DM2 "https://github.com/rainmaker22/smart/blob/main/smart/datasets/scalable_dataset.py"
    click MC "https://github.com/rainmaker22/smart/blob/main/smart/model/smart.py"
    click AL "https://github.com/rainmaker22/smart/blob/main/smart/layers/attention_layer.py"
    click FE "https://github.com/rainmaker22/smart/blob/main/smart/layers/fourier_embedding.py"
    click MLP "https://github.com/rainmaker22/smart/blob/main/smart/layers/mlp_layer.py"
    click AD "https://github.com/rainmaker22/smart/blob/main/smart/modules/agent_decoder.py"
    click MAPD "https://github.com/rainmaker22/smart/blob/main/smart/modules/map_decoder.py"
    click SD "https://github.com/rainmaker22/smart/blob/main/smart/modules/smart_decoder.py"
    click CT "https://github.com/rainmaker22/smart/blob/main/smart/tokens/cluster_frame_5_2048.pkl"
    click MT "https://github.com/rainmaker22/smart/blob/main/smart/tokens/map_traj_token5.pkl"
    click TR "https://github.com/rainmaker22/smart/blob/main/smart/transforms/target_builder.py"
    click mADE "https://github.com/rainmaker22/smart/blob/main/smart/metrics/min_ade.py"
    click mFDE "https://github.com/rainmaker22/smart/blob/main/smart/metrics/min_fde.py"
    click NT "https://github.com/rainmaker22/smart/blob/main/smart/metrics/next_token_cls.py"
    click AM "https://github.com/rainmaker22/smart/blob/main/smart/metrics/average_meter.py"
    click MUtils "https://github.com/rainmaker22/smart/blob/main/smart/metrics/utils.py"
    click TP "https://github.com/rainmaker22/smart/blob/main/train.py"
    click EP "https://github.com/rainmaker22/smart/blob/main/val.py"
    click CONF_train "https://github.com/rainmaker22/smart/blob/main/configs/train/train_scalable.yaml"
    click CONF_val "https://github.com/rainmaker22/smart/blob/main/configs/validation/validation_scalable.yaml"
    click ENV_yml "https://github.com/rainmaker22/smart/blob/main/environment.yml"
    click ENV_req "https://github.com/rainmaker22/smart/blob/main/requirements.txt"
    click SS_IPYG "https://github.com/rainmaker22/smart/blob/main/scripts/install_pyg.sh"
    click SS_TC "https://github.com/rainmaker22/smart/blob/main/scripts/traj_clustering.py"
    click SS_UTILS "https://github.com/rainmaker22/smart/tree/main/smart/utils"

    %% Styling
    classDef data fill:#AED6F1,stroke:#1B4F72,stroke-width:2px;
    classDef model fill:#F9E79F,stroke:#B9770E,stroke-width:2px;
    classDef config fill:#ABEBC6,stroke:#145A32,stroke-width:2px;
    classDef util fill:#E6B0AA,stroke:#922B21,stroke-width:2px;
```
