# dusty-nv/jetson-inference architecture

> Architecture diagram of the GitHub repository dusty-nv/jetson-inference, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dusty-nv/jetson-inference
- Repository: https://github.com/dusty-nv/jetson-inference

## Overview

##### High-Level Architecture Overview

a) Core Deep Learning Components:
- TensorRT Engine Layer
- Neural Network Models Layer
- Vision Primitives Layer (imageNet, detectNet, segNet, poseNet, actionNet, etc.)

b) Utility Components:
- CUDA Utilities
- Image Processing
- I/O and Streaming
- Camera Interfaces

c) API Interfaces:
- C++ API
- Python Bindings
- WebRTC Interface

##### Component Relationships

**[Input Sources] → [Image Processing] → [Neural Network Models] → [Output Handlers]**
- ↑                   ↑                      ↑                      ↑

**[Camera APIs]    [CUDA Utilities]        [TensorRT Engine]     [Display/Network]**

##### Detailed Component Breakdown

a) Input Sources (Blue):
- Camera Feeds (V4L2, GStreamer)
- Image Files
- Video Streams
- Network Streams

b) Processing Layer (Green):
- CUDA Image Processing
- Tensor Conversion
- Memory Management
- GPU Acceleration

c) Neural Network Layer (Red):
- Model Loading
- TensorRT Optimization
- Inference Pipeline
- Network Types (Classification, Detection, Segmentation, etc.)

d) Output Layer (Purple):
- Visualization
- WebRTC Streaming
- Network Output
- File Output

##### Integration Points

Connections for:
- ROS/ROS2 Integration
- WebRTC Server Interface
- Python Bindings
- External Tool Integration

##### Additional Elements

Include:
- Version compatibility notes
- Hardware requirements
- Key file locations
- Configuration points

##### Implementation Notes

- Real-time processing capabilities
- GPU acceleration points
- Model optimization workflow
- Extensibility points
- Multi-platform support

##### Specific Project Elements

Highlight:
- Pre-trained model support
- Custom model integration
- Training pipeline integration
- WebRTC streaming capabilities
- Camera interface options

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Legend
        A[Core Component]:::core
        B([API/Interface]):::api
        C[(Storage/Models)]:::storage
        D{{Processing Unit}}:::processing
        E[/Input/Output/]:::io
    end

    subgraph "Input Sources"
        V4L2([V4L2 Camera]):::api
        GSTR([GStreamer]):::api
        IMG[/Image Files/]:::io
        VID[/Video Streams/]:::io
        NET[/Network Streams/]:::io
    end

    subgraph "Core Processing"
        CUDA{{CUDA Utilities}}:::processing
        IMGP{{Image Processing}}:::processing
        TENC{{Tensor Conversion}}:::processing
    end

    subgraph "Deep Learning Engine"
        TRT[TensorRT Engine]:::core
        NNB[Neural Network Base]:::core
    end

    subgraph "Vision Primitives"
        IMGN[ImageNet]:::core
        DETN[DetectNet]:::core
        SEGN[SegNet]:::core
        POSN[PoseNet]:::core
        ACTN[ActionNet]:::core
    end

    subgraph "Model Management"
        MDOWN[Model Downloader]:::core
        MCONF[(Network Configs)]:::storage
    end

    subgraph "Object Tracking"
        TRACK[Tracking Core]:::core
        IOU[IOU Tracker]:::core
    end

    subgraph "API Layer"
        CPP([C++ API]):::api
        PY([Python Bindings]):::api
        WEB([WebRTC Interface]):::api
    end

    V4L2 & GSTR & IMG & VID & NET --> IMGP
    IMGP --> TENC
    CUDA --> IMGP
    CUDA --> TENC
    TENC --> TRT
    TRT --> NNB
    NNB --> IMGN & DETN & SEGN & POSN & ACTN
    MDOWN --> MCONF
    MCONF --> TRT
    DETN --> TRACK
    TRACK --> IOU
    IMGN & DETN & SEGN & POSN & ACTN --> CPP & PY & WEB

    %% Click events for component mapping
    click TRT "https://github.com/dusty-nv/jetson-inference/blob/master/c/tensorNet.h"
    click NNB "https://github.com/dusty-nv/jetson-inference/blob/master/c/tensorConvert.h"
    click IMGN "https://github.com/dusty-nv/jetson-inference/blob/master/c/imageNet.h"
    click DETN "https://github.com/dusty-nv/jetson-inference/blob/master/c/detectNet.h"
    click SEGN "https://github.com/dusty-nv/jetson-inference/blob/master/c/segNet.h"
    click POSN "https://github.com/dusty-nv/jetson-inference/blob/master/c/poseNet.h"
    click ACTN "https://github.com/dusty-nv/jetson-inference/blob/master/c/actionNet.h"
    click CUDA "https://github.com/dusty-nv/jetson-inference/tree/master/c/plugins/"
    click IMGP "https://github.com/dusty-nv/jetson-inference/blob/master/c/cudaColorspace.h"
    click V4L2 "https://github.com/dusty-nv/jetson-inference/blob/master/c/v4l2Camera.h"
    click GSTR "https://github.com/dusty-nv/jetson-inference/blob/master/c/gstCamera.h"
    click PY "https://github.com/dusty-nv/jetson-inference/tree/master/python/bindings/"
    click WEB "https://github.com/dusty-nv/jetson-inference/tree/master/python/www/"
    click CPP "https://github.com/dusty-nv/jetson-inference/tree/master/examples/"
    click MDOWN "https://github.com/dusty-nv/jetson-inference/blob/master/c/modelDownloader.h"
    click MCONF "https://github.com/dusty-nv/jetson-inference/tree/master/data/networks/"
    click TRACK "https://github.com/dusty-nv/jetson-inference/blob/master/c/tracking/objectTracker.h"
    click IOU "https://github.com/dusty-nv/jetson-inference/blob/master/c/tracking/objectTrackerIOU.h"

    classDef core fill:#f9f,stroke:#333,stroke-width:2px
    classDef api fill:#69f,stroke:#333,stroke-width:2px
    classDef storage fill:#fc3,stroke:#333,stroke-width:2px
    classDef processing fill:#3c9,stroke:#333,stroke-width:2px
    classDef io fill:#ff9,stroke:#333,stroke-width:2px
```
