# nvidia/cuopt architecture

> Architecture diagram of the GitHub repository nvidia/cuopt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nvidia/cuopt
- Repository: https://github.com/nvidia/cuopt

## Overview

##### Project Type and Purpose
- GPU-accelerated optimization engine (cuOpt) implemented in C++/CUDA.
- Solves linear programming (LP), mixed-integer linear programming (MILP), and vehicle routing problems (VRP).
- Exposes functionality via multiple APIs: C API (libcuopt), Python API (Cython), Server API (HTTP), plus CLI tools.
- Distributed as pip/conda packages and Docker containers.

##### File Structure Breakdown
- cpp/
- src/: core solver modules (linear_programming, mip, routing, utilities)
- /: public C++ and C API headers
- libmps_parser/: MPS file parsing submodule
- tests/benchmarks: C++ unit tests and performance benchmarks
- python/
- libcuopt/: low-level C API loader and CLI wrapper
- cuopt/: high-level Python package (Cython bindings for distance_engine, linear_programming, routing)
- cuopt_server/: microservice implementation (HTTP endpoints, job queue, data validation, result store, webserver.py)
- cuopt_self_hosted/: thin client for self-hosted server with secure communication
- docs/
- docs/cuopt/: Sphinx source for C API, Python API, CLI, server quickstarts
- benchmarks/, datasets/: performance and sample data for LP, MILP, routing
- ci/, conda/, cmake/: build scripts, CI/CD workflows, packaging recipes

##### Insights from README
- Multi-API support: C, C++, Python, Server
- Installation via pip (cuopt-server, cuopt-sh-client), conda, or Docker image
- Build‐from‐source via CMake and conda recipes
- Requires CUDA ≥12.0, Linux (WSL2 supported)

##### Main Components & Their Relationships
a. Core Compute Layer
- CUDA kernels and C++ host code (cpp/src)
- MPS Parser (cpp/libmps_parser)
b. C API Layer
- libcuopt: C bindings wrapping core C++ functionality
c. Python Binding Layer
- Cython modules in python/cuopt/* invoking C API
d. High-Level Python & CLI Layer
- python/cuopt package for end-users
- C++ cuopt_cli tool and python/libcuopt CLI wrapper
e. Service Layer
- python/cuopt_server: HTTP webserver, request validation, job queue, solver invocation, result storage
f. Client Layer
- python/cuopt_self_hosted: thin client calling server via secure RPC/HTTP
- Interactive notebooks and example scripts
g. Packaging & Deployment
- pip and conda packages, Docker image (nvidia/cuopt:…)
- CI/CD pipelines (GitHub Actions) produce build artifacts

##### Data and Control Flow
- Client (CLI, notebook, thin client) → HTTP or function call → cuOpt Server
- Server validates input → enqueues job → calls high-level Python API
- Python Cython binding → C API → Core Compute Engine on GPU
- Compute result → propagate back through C API → Python wrapper → Server → Client response
- Packaging workflows consume compiled libraries and Python wheels to publish pip/conda artifacts and Docker images

##### Architectural Patterns
- Layered architecture: Compute → C API → Python API → Service → Client
- Adapter pattern: Cython modules adapt Python data models to C API calls
- Microservice pattern: stateless HTTP server with job queue and result store
- Strategy modularity in core engine: separate modules for LP, MILP, VRP

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Clients Layer
    subgraph "Clients"
        style Clients fill:#ffeb99,stroke:#333,stroke-width:1px
        CLI_NATIVE["Native CLI"]:::client
        CLI_PYWRAP["Python CLI Wrapper"]:::client
        NOTEBOOKS["Notebooks & Scripts"]:::client
        THIN_CLIENT["Self-Hosted Client"]:::client
    end

    %% Service Layer
    subgraph "Service Layer"
        style Service Layer fill:#ffcc99,stroke:#333,stroke-width:1px
        SERVER["cuOpt Server\n(webserver.py)"]:::service
        VALIDATION["Input Validation"]:::service
        JOB_QUEUE[(Job Queue)]:::data
        RESULT_STORE[(Result Store)]:::data
    end

    %% API Bindings
    subgraph "API Bindings"
        style API Bindings fill:#ccff99,stroke:#333,stroke-width:1px
        C_API["libcuopt C API"]:::api
        CYTHON["Cython Modules"]:::api
    end

    %% Core Compute Layer
    subgraph "Core Compute"
        style Core Compute fill:#99ccff,stroke:#333,stroke-width:1px
        CUDA_HOST["C++ Host Code"]:::core
        CUDA_KERNELS["CUDA Kernels"]:::core
        MPS_PARSER["MPS Parser"]:::core
    end

    %% Data & Benchmarks
    subgraph "Datasets & Benchmarks"
        style Datasets fill:#eeeeee,stroke:#333,stroke-width:1px
        DATASETS["Datasets & Benchmarks"]:::data
    end

    %% Packaging & Deployment
    subgraph "Packaging & Deployment"
        style Packaging fill:#e0b3ff,stroke:#333,stroke-width:1px
        CI["CI/CD Workflows"]:::deploy
        CONDA["Conda Recipes"]:::deploy
        DOCKER["Docker Image Build"]:::deploy
        PIP["pip/conda Packages"]:::deploy
    end

    %% Connections: Clients to Service
    CLI_NATIVE -->|"HTTP Request"| SERVER
    NOTEBOOKS -->|"HTTP Request"| SERVER
    THIN_CLIENT -->|"HTTP/S RPC"| SERVER
    CLI_PYWRAP -->|"Function Call"| C_API

    %% Service internals
    SERVER -->|validate| VALIDATION
    VALIDATION -->|enqueue| JOB_QUEUE
    JOB_QUEUE -->|job dispatch| CYTHON
    CYTHON -->|"calls"| C_API
    C_API -->|"invokes"| CUDA_HOST
    CUDA_HOST -->|"launches"| CUDA_KERNELS
    C_API -->|"parses"| MPS_PARSER
    DATASETS -->|"input files"| MPS_PARSER
    CUDA_KERNELS -->|results| CUDA_HOST
    CUDA_HOST -->|return data| C_API
    C_API -->|wrapped by| CYTHON
    CYTHON -->|response| SERVER
    SERVER -->|HTTP Response| CLI_NATIVE
    SERVER -->|HTTP Response| NOTEBOOKS
    SERVER -->|HTTP Response| THIN_CLIENT

    %% Packaging Flow
    CI --> CONDA
    CI --> DOCKER
    CONDA --> PIP
    DOCKER --> PIP
    PIP --> CLI_PYWRAP
    PIP --> NOTEBOOKS
    PIP --> THIN_CLIENT

    %% Click Events
    click CLI_NATIVE "https://github.com/nvidia/cuopt/blob/branch-25.05/cpp/cuopt_cli.cpp"
    click CLI_PYWRAP "https://github.com/nvidia/cuopt/blob/branch-25.05/python/libcuopt/libcuopt/_cli_wrapper.py"
    click THIN_CLIENT "https://github.com/nvidia/cuopt/blob/branch-25.05/python/cuopt_self_hosted/cuopt_sh_client/cuopt_self_host_client.py"
    click SERVER "https://github.com/nvidia/cuopt/blob/branch-25.05/python/cuopt_server/cuopt_server/webserver.py"
    click C_API "https://github.com/nvidia/cuopt/tree/branch-25.05/cpp/include/cuopt"
    click CYTHON "https://github.com/nvidia/cuopt/tree/branch-25.05/python/cuopt/cuopt"
    click CUDA_HOST "https://github.com/nvidia/cuopt/tree/branch-25.05/cpp/src"
    click MPS_PARSER "https://github.com/nvidia/cuopt/tree/branch-25.05/cpp/libmps_parser"
    click CI "https://github.com/nvidia/cuopt/tree/branch-25.05/.github/workflows"
    click CONDA "https://github.com/nvidia/cuopt/tree/branch-25.05/conda/recipes"
    click DOCKER "https://github.com/nvidia/cuopt/blob/branch-25.05/ci/build-service.Dockerfile"

    %% Styles
    classDef core fill:#99ccff,stroke:#333,stroke-width:1px
    classDef api fill:#ccff99,stroke:#333,stroke-width:1px
    classDef service fill:#ffcc99,stroke:#333,stroke-width:1px
    classDef client fill:#ffeb99,stroke:#333,stroke-width:1px
    classDef data fill:#eeeeee,stroke:#333,stroke-width:1px,stroke-dasharray: 5 5
    classDef deploy fill:#e0b3ff,stroke:#333,stroke-width:1px,stroke-dasharray: 2 2
```
