# vivek5200/ouroboros-triton architecture

> Architecture diagram of the GitHub repository vivek5200/ouroboros-triton, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vivek5200/ouroboros-triton
- Repository: https://github.com/vivek5200/ouroboros-triton

## Overview

Ouroboros Triton combines fixed-size block memory, attention computation, and a supervisor/worker serving layer. The principal documented request workflow is supervisor allocation, worker dispatch, and crash recovery through a safe queue; the attention kernel is a separate engine capability. The sampled excerpts establish the memory and serving ownership model, but do not show attention dispatch from serving, so that connection is omitted.

Main entry points: [`src/serving.py`](https://github.com/vivek5200/ouroboros-triton/blob/main/src/serving.py) (Supervisor), [`src/block_table.py`](https://github.com/vivek5200/ouroboros-triton/blob/main/src/block_table.py) (Python Block Table), [`src/kernels/block_attention.py`](https://github.com/vivek5200/ouroboros-triton/blob/main/src/kernels/block_attention.py) (Triton Attention), [`src/scoping_mask.py`](https://github.com/vivek5200/ouroboros-triton/blob/main/src/scoping_mask.py) (Scoping Mask)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_serving["HA Serving"]
  node_supervisor["Supervisor<br/>[serving.py]"]
  node_worker["Stateless Worker<br/>[serving.py]"]
  node_safe_queue["Safe Queue<br/>[serving.py]"]
  node_kv_cache[("KV Cache<br/>[serving.py]")]
end

subgraph group_memory["Block Memory"]
  node_py_table["Python Block Table<br/>[block_table.py]"]
  node_page_row["Page Table Row<br/>[block_table.py]"]
  node_cpp_binding["C++ Bindings<br/>[bindings.cpp]"]
  node_cpp_table["C++ Block Table<br/>[block_table.cpp]"]
end

subgraph group_attention["Attention"]
  node_scope_mask["Scoping Mask<br/>[scoping_mask.py]"]
  node_triton_attention["Triton Attention<br/>[block_attention.py]"]
  node_golden_attention["Reference Attention"]
end

node_client(("Request Client"))
node_model_caller(("Model Caller"))

node_client -->|"submits request"| node_supervisor
node_supervisor -->|"allocates chain"| node_py_table
node_supervisor -->|"records head"| node_kv_cache
node_supervisor -->|"dispatches request"| node_worker
node_worker -->|"returns result"| node_supervisor
node_supervisor -->|"queues on crash"| node_safe_queue
node_supervisor -->|"expands or releases"| node_py_table
node_py_table -->|"builds row"| node_page_row
node_cpp_binding -->|"calls methods"| node_cpp_table
node_model_caller -->|"provides inputs"| node_triton_attention
node_scope_mask -->|"provides mask"| node_triton_attention
node_triton_attention -->|"returns output"| node_model_caller
node_scope_mask -->|"provides scope mask"| node_golden_attention

click node_supervisor "https://github.com/vivek5200/ouroboros-triton/blob/main/src/serving.py"
click node_worker "https://github.com/vivek5200/ouroboros-triton/blob/main/src/serving.py"
click node_safe_queue "https://github.com/vivek5200/ouroboros-triton/blob/main/src/serving.py"
click node_kv_cache "https://github.com/vivek5200/ouroboros-triton/blob/main/src/serving.py"
click node_py_table "https://github.com/vivek5200/ouroboros-triton/blob/main/src/block_table.py"
click node_page_row "https://github.com/vivek5200/ouroboros-triton/blob/main/src/block_table.py"
click node_cpp_binding "https://github.com/vivek5200/ouroboros-triton/blob/main/src/cpp/bindings.cpp"
click node_cpp_table "https://github.com/vivek5200/ouroboros-triton/blob/main/src/cpp/block_table.cpp"
click node_scope_mask "https://github.com/vivek5200/ouroboros-triton/blob/main/src/scoping_mask.py"
click node_triton_attention "https://github.com/vivek5200/ouroboros-triton/blob/main/src/kernels/block_attention.py"
click node_golden_attention "https://github.com/vivek5200/ouroboros-triton/blob/main/src/kernels/reference_attention.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_supervisor,node_worker,node_safe_queue,node_kv_cache,node_client toneBlue
class node_py_table,node_page_row,node_cpp_binding,node_cpp_table toneAmber
class node_scope_mask,node_triton_attention,node_golden_attention toneMint
class node_model_caller toneIndigo
```

## Components

### HA Serving

- [Supervisor](https://github.com/vivek5200/ouroboros-triton/blob/main/src/serving.py): `src/serving.py`
- [Stateless Worker](https://github.com/vivek5200/ouroboros-triton/blob/main/src/serving.py): `src/serving.py`
- [Safe Queue](https://github.com/vivek5200/ouroboros-triton/blob/main/src/serving.py): `src/serving.py`
- [KV Cache](https://github.com/vivek5200/ouroboros-triton/blob/main/src/serving.py): `src/serving.py`

### Block Memory

- [Python Block Table](https://github.com/vivek5200/ouroboros-triton/blob/main/src/block_table.py): `src/block_table.py`
- [Page Table Row](https://github.com/vivek5200/ouroboros-triton/blob/main/src/block_table.py): `src/block_table.py`
- [C++ Bindings](https://github.com/vivek5200/ouroboros-triton/blob/main/src/cpp/bindings.cpp): `src/cpp/bindings.cpp`
- [C++ Block Table](https://github.com/vivek5200/ouroboros-triton/blob/main/src/cpp/block_table.cpp): `src/cpp/block_table.cpp`

### Attention

- [Scoping Mask](https://github.com/vivek5200/ouroboros-triton/blob/main/src/scoping_mask.py): `src/scoping_mask.py`
- [Triton Attention](https://github.com/vivek5200/ouroboros-triton/blob/main/src/kernels/block_attention.py): `src/kernels/block_attention.py`
- [Reference Attention](https://github.com/vivek5200/ouroboros-triton/blob/main/src/kernels/reference_attention.py): `src/kernels/reference_attention.py`

### Other components

- **Request Client**
- **Model Caller**

## Connections

- Request Client → Supervisor: submits request
- Supervisor → Python Block Table: allocates chain
- Supervisor → KV Cache: records head
- Supervisor → Stateless Worker: dispatches request
- Stateless Worker → Supervisor: returns result
- Supervisor → Safe Queue: queues on crash
- Supervisor → Python Block Table: expands or releases
- Python Block Table → Page Table Row: builds row
- C++ Bindings → C++ Block Table: calls methods
- Model Caller → Triton Attention: provides inputs
- Scoping Mask → Triton Attention: provides mask
- Triton Attention → Model Caller: returns output
- Scoping Mask → Reference Attention: provides scope mask