# nvidia/dl4agx architecture

> Architecture diagram of the GitHub repository nvidia/dl4agx, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nvidia/dl4agx
- Repository: https://github.com/nvidia/dl4agx

## Overview

DL4AGX is a collection of autonomous-vehicle model designs, TensorRT deployment solutions, and inference samples for NVIDIA DRIVE—not a single application. Its user-facing workflow is generally to prepare or export a model, deploy it with a platform-specific inference solution, and inspect or visualize results. The README also names quantization, sparsity, plugin, and LLM deployment capabilities, which are preserved as separate branches. Most source excerpts cover selected inference apps; dashed, uncited links mark broader workflows that the available evidence does not verify in detail.

Main entry points: [`AV-Solutions/uniad-trt/inference_app/enqueueV2/include/uniad.hpp`](https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/uniad-trt/inference_app/enqueueV2/include/uniad.hpp) (UniAD Kernel), [`AV-Solutions/far3d-trt/tools/export_onnx.py`](https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/far3d-trt/tools/export_onnx.py) (ONNX Export), [`AV-Solutions/vad-trt/export_eval/bev_deploy/trt/inference.py`](https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/vad-trt/export_eval/bev_deploy/trt/inference.py) (TensorRT Engine), [`AV-Solutions/dcnv4-trt`](https://github.com/nvidia/dl4agx/tree/master/AV-Solutions/dcnv4-trt) (DCNv4 TensorRT)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_model["Model Designs"]
  node_dest["DEST Models<br/>[DESTNet.py]"]
  node_reduceformer["ReduceFormer<br/>[backbone.py]"]
end

subgraph group_deploy["AV Deployment"]
  node_dcnv4["DCNv4 TensorRT"]
  node_far3d["Far3D TensorRT<br/>[far3d.cpp]"]
  node_mtmi["MTMI TensorRT<br/>[main.cpp]"]
  node_petr["PETR TensorRT<br/>[petrv2.cpp]"]
  node_streampetr["StreamPETR TensorRT<br/>[main.cpp]"]
  node_uniad["UniAD TensorRT<br/>[main.cpp]"]
  node_vad["VAD TensorRT<br/>[main.cpp]"]
  node_bevformer["BEVFormer INT8"]
  node_sparsity["Sparsity INT8"]
  node_llm["TensorRT-LLM<br/>[json_modifier.py]"]
  node_diffusion["Diffusion Planner"]
end

subgraph group_infer["Inference Apps"]
  node_far3d_runtime["Far3D Runtime<br/>[far3d.cpp]"]
  node_far3d_tensor["Tensor Utilities<br/>[tensor.hpp]"]
  node_uniad_kernel["UniAD Kernel<br/>[uniad.hpp]"]
  node_uniad_preprocess["Input Preparation<br/>[pre_process.hpp]"]
  node_uniad_postprocess["Output Processing<br/>[post_process.hpp]"]
  node_visualization["AV Visualization<br/>[visualize.hpp]"]
  node_cudla["cuDLA Context<br/>[cudla_context.cpp]"]
end

subgraph group_prep["Model Preparation"]
  node_export["ONNX Export<br/>[export_onnx.py]"]
  node_quantize["ONNX Quantization<br/>[quantize_onnx.py]"]
  node_calibration["Calibration Data"]
  node_engine["TensorRT Engine<br/>[inference.py]"]
end

subgraph group_support["Deployment Extensions"]
  node_plugins["TensorRT Plugins"]
  node_plugin_guide["PTX Plugin Guidance"]
  node_export_guide["ONNX Export Guidance"]
end

node_user(("AV Engineer"))

node_user -.->|"selects model"| node_dest
node_user -.->|"selects model"| node_reduceformer
node_user -.->|"prepares model"| node_export
node_export -.->|"exports ONNX"| node_engine
node_quantize -.->|"quantizes model"| node_engine
node_calibration -.->|"supplies calibration"| node_engine
node_user -.->|"deploys model"| node_dcnv4
node_user -.->|"runs inference"| node_far3d
node_user -.->|"runs inference"| node_mtmi
node_user -.->|"runs inference"| node_petr
node_user -.->|"runs inference"| node_streampetr
node_user -.->|"runs inference"| node_uniad
node_user -.->|"runs inference"| node_vad
node_user -.->|"quantizes model"| node_bevformer
node_user -.->|"trains model"| node_sparsity
node_user -.->|"deploys LLM"| node_llm
node_user -.->|"deploys planner"| node_diffusion
node_far3d -.->|"runs inference"| node_far3d_runtime
node_far3d_runtime -->|"uses tensors"| node_far3d_tensor
node_user -.->|"runs inference"| node_uniad_kernel
node_uniad_kernel -.->|"prepares inputs"| node_uniad_preprocess
node_uniad_kernel -->|"processes outputs"| node_uniad_postprocess
node_user -.->|"views results"| node_visualization
node_mtmi -->|"uses context"| node_cudla
node_vad -->|"renders results"| node_visualization
node_dcnv4 -.->|"uses plugins"| node_plugins
node_plugins -.->|"integration guidance"| node_plugin_guide
node_export -.->|"export guidance"| node_export_guide

click node_dest "https://github.com/nvidia/dl4agx/blob/master/Models/DEST/networks/DESTNet.py"
click node_reduceformer "https://github.com/nvidia/dl4agx/blob/master/Models/ReduceFormer/reduceformer/backbone.py"
click node_dcnv4 "https://github.com/nvidia/dl4agx/tree/master/AV-Solutions/dcnv4-trt"
click node_far3d "https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/far3d-trt/inference_app/lib/far3d.cpp"
click node_mtmi "https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/mtmi/inference_app/main.cpp"
click node_petr "https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/petr-trt/app/petrv2.cpp"
click node_streampetr "https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/streampetr-trt/inference_app/main.cpp"
click node_uniad "https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/uniad-trt/inference_app/enqueueV2/src/main.cpp"
click node_vad "https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/vad-trt/app/main.cpp"
click node_bevformer "https://github.com/nvidia/dl4agx/tree/master/AV-Solutions/bevformer-int8-eq"
click node_sparsity "https://github.com/nvidia/dl4agx/tree/master/AV-Solutions/SparsityINT8"
click node_llm "https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/llms-trtllm/json_modifier.py"
click node_diffusion "https://github.com/nvidia/dl4agx/tree/master/AV-Solutions/diffusion_planner-trt"
click node_export "https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/far3d-trt/tools/export_onnx.py"
click node_quantize "https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/far3d-trt/tools/quantize_onnx.py"
click node_calibration "https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/uniad-trt/tools/prepare_calib_data.py"
click node_engine "https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/vad-trt/export_eval/bev_deploy/trt/inference.py"
click node_far3d_runtime "https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/far3d-trt/inference_app/lib/far3d.cpp"
click node_far3d_tensor "https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/far3d-trt/inference_app/include/tensor.hpp"
click node_uniad_kernel "https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/uniad-trt/inference_app/enqueueV2/include/uniad.hpp"
click node_uniad_preprocess "https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/uniad-trt/inference_app/enqueueV2/include/pre_process.hpp"
click node_uniad_postprocess "https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/uniad-trt/inference_app/enqueueV2/include/post_process.hpp"
click node_visualization "https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/vad-trt/app/lib/visualize.hpp"
click node_cudla "https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/mtmi/inference_app/src/cudla_context.cpp"
click node_plugins "https://github.com/nvidia/dl4agx/tree/master/AV-Solutions/vad-trt/plugins"
click node_plugin_guide "https://github.com/nvidia/dl4agx/tree/master/AV-Solutions/triton2cubin-plugin-guidance"
click node_export_guide "https://github.com/nvidia/dl4agx/tree/master/AV-Solutions/onnx-export-guidance"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dest,node_reduceformer toneBlue
class node_dcnv4,node_far3d,node_mtmi,node_petr,node_streampetr,node_uniad,node_vad,node_bevformer,node_sparsity,node_llm,node_diffusion toneAmber
class node_far3d_runtime,node_far3d_tensor,node_uniad_kernel,node_uniad_preprocess,node_uniad_postprocess,node_visualization,node_cudla toneMint
class node_export,node_quantize,node_calibration,node_engine toneRose
class node_plugins,node_plugin_guide,node_export_guide,node_user toneIndigo
```

## Components

### Model Designs

- [DEST Models](https://github.com/nvidia/dl4agx/blob/master/Models/DEST/networks/DESTNet.py): `Models/DEST/networks/DESTNet.py`
- [ReduceFormer](https://github.com/nvidia/dl4agx/blob/master/Models/ReduceFormer/reduceformer/backbone.py): `Models/ReduceFormer/reduceformer/backbone.py`

### AV Deployment

- [DCNv4 TensorRT](https://github.com/nvidia/dl4agx/tree/master/AV-Solutions/dcnv4-trt): `AV-Solutions/dcnv4-trt`
- [Far3D TensorRT](https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/far3d-trt/inference_app/lib/far3d.cpp): `AV-Solutions/far3d-trt/inference_app/lib/far3d.cpp`
- [MTMI TensorRT](https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/mtmi/inference_app/main.cpp): `AV-Solutions/mtmi/inference_app/main.cpp`
- [PETR TensorRT](https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/petr-trt/app/petrv2.cpp): `AV-Solutions/petr-trt/app/petrv2.cpp`
- [StreamPETR TensorRT](https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/streampetr-trt/inference_app/main.cpp): `AV-Solutions/streampetr-trt/inference_app/main.cpp`
- [UniAD TensorRT](https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/uniad-trt/inference_app/enqueueV2/src/main.cpp): `AV-Solutions/uniad-trt/inference_app/enqueueV2/src/main.cpp`
- [VAD TensorRT](https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/vad-trt/app/main.cpp): `AV-Solutions/vad-trt/app/main.cpp`
- [BEVFormer INT8](https://github.com/nvidia/dl4agx/tree/master/AV-Solutions/bevformer-int8-eq): `AV-Solutions/bevformer-int8-eq`
- [Sparsity INT8](https://github.com/nvidia/dl4agx/tree/master/AV-Solutions/SparsityINT8): `AV-Solutions/SparsityINT8`
- [TensorRT-LLM](https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/llms-trtllm/json_modifier.py): `AV-Solutions/llms-trtllm/json_modifier.py`
- [Diffusion Planner](https://github.com/nvidia/dl4agx/tree/master/AV-Solutions/diffusion_planner-trt): `AV-Solutions/diffusion_planner-trt`

### Inference Apps

- [Far3D Runtime](https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/far3d-trt/inference_app/lib/far3d.cpp): `AV-Solutions/far3d-trt/inference_app/lib/far3d.cpp`
- [Tensor Utilities](https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/far3d-trt/inference_app/include/tensor.hpp): `AV-Solutions/far3d-trt/inference_app/include/tensor.hpp`
- [UniAD Kernel](https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/uniad-trt/inference_app/enqueueV2/include/uniad.hpp): `AV-Solutions/uniad-trt/inference_app/enqueueV2/include/uniad.hpp`
- [Input Preparation](https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/uniad-trt/inference_app/enqueueV2/include/pre_process.hpp): `AV-Solutions/uniad-trt/inference_app/enqueueV2/include/pre_process.hpp`
- [Output Processing](https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/uniad-trt/inference_app/enqueueV2/include/post_process.hpp): `AV-Solutions/uniad-trt/inference_app/enqueueV2/include/post_process.hpp`
- [AV Visualization](https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/vad-trt/app/lib/visualize.hpp): `AV-Solutions/vad-trt/app/lib/visualize.hpp`
- [cuDLA Context](https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/mtmi/inference_app/src/cudla_context.cpp): `AV-Solutions/mtmi/inference_app/src/cudla_context.cpp`

### Model Preparation

- [ONNX Export](https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/far3d-trt/tools/export_onnx.py): `AV-Solutions/far3d-trt/tools/export_onnx.py`
- [ONNX Quantization](https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/far3d-trt/tools/quantize_onnx.py): `AV-Solutions/far3d-trt/tools/quantize_onnx.py`
- [Calibration Data](https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/uniad-trt/tools/prepare_calib_data.py): `AV-Solutions/uniad-trt/tools/prepare_calib_data.py`
- [TensorRT Engine](https://github.com/nvidia/dl4agx/blob/master/AV-Solutions/vad-trt/export_eval/bev_deploy/trt/inference.py): `AV-Solutions/vad-trt/export_eval/bev_deploy/trt/inference.py`

### Deployment Extensions

- [TensorRT Plugins](https://github.com/nvidia/dl4agx/tree/master/AV-Solutions/vad-trt/plugins): `AV-Solutions/vad-trt/plugins`
- [PTX Plugin Guidance](https://github.com/nvidia/dl4agx/tree/master/AV-Solutions/triton2cubin-plugin-guidance): `AV-Solutions/triton2cubin-plugin-guidance`
- [ONNX Export Guidance](https://github.com/nvidia/dl4agx/tree/master/AV-Solutions/onnx-export-guidance): `AV-Solutions/onnx-export-guidance`

### Other components

- **AV Engineer**

## Connections

- AV Engineer → DEST Models: selects model
- AV Engineer → ReduceFormer: selects model
- AV Engineer → ONNX Export: prepares model
- ONNX Export → TensorRT Engine: exports ONNX
- ONNX Quantization → TensorRT Engine: quantizes model
- Calibration Data → TensorRT Engine: supplies calibration
- AV Engineer → DCNv4 TensorRT: deploys model
- AV Engineer → Far3D TensorRT: runs inference
- AV Engineer → MTMI TensorRT: runs inference
- AV Engineer → PETR TensorRT: runs inference
- AV Engineer → StreamPETR TensorRT: runs inference
- AV Engineer → UniAD TensorRT: runs inference
- AV Engineer → VAD TensorRT: runs inference
- AV Engineer → BEVFormer INT8: quantizes model
- AV Engineer → Sparsity INT8: trains model
- AV Engineer → TensorRT-LLM: deploys LLM
- AV Engineer → Diffusion Planner: deploys planner
- Far3D TensorRT → Far3D Runtime: runs inference (evidence: `AV-Solutions/far3d-trt/inference_app/lib/far3d.cpp`)
- Far3D Runtime → Tensor Utilities: uses tensors
- AV Engineer → UniAD Kernel: runs inference
- UniAD Kernel → Input Preparation: prepares inputs (evidence: `AV-Solutions/uniad-trt/inference_app/enqueueV2/include/pre_process.hpp`)
- UniAD Kernel → Output Processing: processes outputs (evidence: `AV-Solutions/uniad-trt/inference_app/enqueueV2/include/post_process.hpp`)
- AV Engineer → AV Visualization: views results
- MTMI TensorRT → cuDLA Context: uses context (evidence: `AV-Solutions/mtmi/inference_app/main.cpp`)
- VAD TensorRT → AV Visualization: renders results (evidence: `AV-Solutions/vad-trt/app/main.cpp`)
- DCNv4 TensorRT → TensorRT Plugins: uses plugins
- TensorRT Plugins → PTX Plugin Guidance: integration guidance
- ONNX Export → ONNX Export Guidance: export guidance