# rapidsai/rmm architecture

> Architecture diagram of the GitHub repository rapidsai/rmm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rapidsai/rmm
- Repository: https://github.com/rapidsai/rmm

## Overview

##### Core Components (Top Level)
a. Language Interfaces (C++ and Python)
b. Memory Resource Management Layer
c. CUDA Memory Layer

##### Memory Resource Management Layer (Central)
a. Base Memory Resources
- device_memory_resource (abstract interface)
- host_memory_resource (abstract interface)
b. Concrete Memory Resources
- cuda_memory_resource
- managed_memory_resource
- pool_memory_resource
- fixed_size_memory_resource
- binning_memory_resource
c. Resource Adaptors
- logging_resource_adaptor
- thread_safe_resource_adaptor
- statistics_resource_adaptor

##### Data Structures Layer
- Device data structures:
- device_buffer
- device_uvector<T>
- device_scalar

##### Integration Layer
- CuPy Allocator
- Numba Allocator
- PyTorch Allocator
- Thrust Integration

##### Stream Management
- Stream handling components:
- cuda_stream_view
- cuda_stream
- cuda_stream_pool

Use these visual elements:

##### Connection Types
- Double lines: Interface implementations

##### Resource Management Pattern
- How memory resources can be composed and wrapped
- Default resource chain

##### Stream-Ordered Memory Management
- How streams are associated with allocations
- Stream synchronization points

##### Python/C++ Interoperability
- Bridging between Python and C++ components
- Ownership transfer points

##### Memory Pool Management
- How pool resources manage memory blocks
- Sizing and growth strategies

- Special annotations or markers
- Component type categories

Remember to emphasize the stream-ordered nature of the memory management, as this is a key distinguishing feature of RMM's architecture.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Define styles
    classDef core fill:#b3d9ff,stroke:#333,stroke-width:2px
    classDef python fill:#90EE90,stroke:#333,stroke-width:2px
    classDef cpp fill:#FFB366,stroke:#333,stroke-width:2px
    classDef external fill:#FFE5CC,stroke:#333,stroke-width:2px
    classDef utility fill:#E6E6E6,stroke:#333,stroke-width:2px
    classDef abstract fill:#fff,stroke:#333,stroke-width:2px,stroke-dasharray: 5 5
    
    subgraph Language_Interfaces
        CPP["C++ Interface"]:::cpp
        PY["Python Interface"]:::python
        click PY "https://github.com/rapidsai/rmm/blob/branch-25.02/python/rmm/rmm.py"
    end

    subgraph Memory_Resource_Management
        subgraph Base_Resources
            DMR["Device Memory Resource"]:::abstract
            HMR["Host Memory Resource"]:::abstract
            click DMR "https://github.com/rapidsai/rmm/blob/branch-25.02/include/rmm/mr/device/device_memory_resource.hpp"
            click HMR "https://github.com/rapidsai/rmm/blob/branch-25.02/include/rmm/mr/host/host_memory_resource.hpp"
        end

        subgraph Concrete_Resources
            CMR["CUDA Memory Resource"]:::core
            MMR["Managed Memory Resource"]:::core
            PMR["Pool Memory Resource"]:::core
            FSMR["Fixed Size Memory Resource"]:::core
            BMR["Binning Memory Resource"]:::core
            click CMR "https://github.com/rapidsai/rmm/blob/branch-25.02/include/rmm/mr/device/cuda_memory_resource.hpp"
            click MMR "https://github.com/rapidsai/rmm/blob/branch-25.02/include/rmm/mr/device/managed_memory_resource.hpp"
            click PMR "https://github.com/rapidsai/rmm/blob/branch-25.02/include/rmm/mr/device/pool_memory_resource.hpp"
            click FSMR "https://github.com/rapidsai/rmm/blob/branch-25.02/include/rmm/mr/device/fixed_size_memory_resource.hpp"
            click BMR "https://github.com/rapidsai/rmm/blob/branch-25.02/include/rmm/mr/device/binning_memory_resource.hpp"
        end

        subgraph Resource_Adaptors
            LRA["Logging Resource Adaptor"]:::utility
            TSRA["Thread Safe Resource Adaptor"]:::utility
            SRA["Statistics Resource Adaptor"]:::utility
            click LRA "https://github.com/rapidsai/rmm/blob/branch-25.02/include/rmm/mr/device/logging_resource_adaptor.hpp"
            click TSRA "https://github.com/rapidsai/rmm/blob/branch-25.02/include/rmm/mr/device/thread_safe_resource_adaptor.hpp"
            click SRA "https://github.com/rapidsai/rmm/blob/branch-25.02/include/rmm/mr/device/statistics_resource_adaptor.hpp"
        end
    end

    subgraph Data_Structures
        DB["Device Buffer"]:::core
        DUV["Device UVector"]:::core
        DS["Device Scalar"]:::core
        click DB "https://github.com/rapidsai/rmm/blob/branch-25.02/include/rmm/device_buffer.hpp"
        click DUV "https://github.com/rapidsai/rmm/blob/branch-25.02/include/rmm/device_uvector.hpp"
        click DS "https://github.com/rapidsai/rmm/blob/branch-25.02/include/rmm/device_scalar.hpp"
    end

    subgraph Stream_Management
        CS["CUDA Stream"]:::core
        CSV["CUDA Stream View"]:::core
        CSP["CUDA Stream Pool"]:::core
        click CS "https://github.com/rapidsai/rmm/blob/branch-25.02/include/rmm/cuda_stream.hpp"
        click CSV "https://github.com/rapidsai/rmm/blob/branch-25.02/include/rmm/cuda_stream_view.hpp"
        click CSP "https://github.com/rapidsai/rmm/blob/branch-25.02/include/rmm/cuda_stream_pool.hpp"
    end

    subgraph External_Integration
        CUPY["CuPy Allocator"]:::external
        NUMBA["Numba Allocator"]:::external
        TORCH["PyTorch Allocator"]:::external
        THRUST["Thrust Integration"]:::external
        click CUPY "https://github.com/rapidsai/rmm/blob/branch-25.02/python/rmm/allocators/cupy.py"
        click NUMBA "https://github.com/rapidsai/rmm/blob/branch-25.02/python/rmm/allocators/numba.py"
        click TORCH "https://github.com/rapidsai/rmm/blob/branch-25.02/python/rmm/allocators/torch.py"
        click THRUST "https://github.com/rapidsai/rmm/blob/branch-25.02/include/rmm/mr/device/thrust_allocator_adaptor.hpp"
    end

    %% Connections
    DMR --> CMR
    DMR --> MMR
    DMR --> PMR
    DMR --> FSMR
    DMR --> BMR
    
    CMR & MMR & PMR & FSMR & BMR --> LRA
    CMR & MMR & PMR & FSMR & BMR --> TSRA
    CMR & MMR & PMR & FSMR & BMR --> SRA
    
    LRA & TSRA & SRA --> DB
    DB --> DUV
    DB --> DS
    
    CS --> CSV
    CS --> CSP
    
    PY --> CUPY
    PY --> NUMBA
    PY --> TORCH
    CPP --> THRUST
    
    %% Legend
    subgraph Legend
        L1["Core Components"]:::core
        L2["Python Components"]:::python
        L3["C++ Components"]:::cpp
        L4["External Integration"]:::external
        L5["Utility Components"]:::utility
        L6["Abstract Interfaces"]:::abstract
    end
```
