How vivek5200/ouroboros-triton works
Ouroboros Triton combines fixed-size block memory, attention computation, and a supervisor/worker serving layer. The principal documented request workflow is supervisor allocation, worker dispatch, and crash recovery through a safe queue; the attention kernel is a separate engine capability. The sampled excerpts establish the memory and serving ownership model, but do not show attention dispatch from serving, so that connection is omitted.
Main entry points: serving.py Supervisor · block_table.py Python Block Table · block_attention.py Triton Attention · scoping_mask.py Scoping Mask
Components 13
HA Serving
- Supervisor
src/serving.py - Stateless Worker
src/serving.py - Safe Queue
src/serving.py - KV Cache
src/serving.py
Block Memory
- Python Block Table
src/block_table.py - Page Table Row
src/block_table.py - C++ Bindings
src/cpp/bindings.cpp - C++ Block Table
src/cpp/block_table.cpp
Attention
- Scoping Mask
src/scoping_mask.py - Triton Attention
src/kernels/block_attention.py - Reference Attention
src/kernels/reference_attention.py
Other components
- Request Client
- Model Caller
Connections 13
- Request Client → Supervisor: submits request
- Supervisor → Python Block Table: allocates chain
- Supervisor → KV Cache: records head
- Supervisor → Stateless Worker: dispatches request
- Stateless Worker → Supervisor: returns result
- Supervisor → Safe Queue: queues on crash
- Supervisor → Python Block Table: expands or releases
- Python Block Table → Page Table Row: builds row
- C++ Bindings → C++ Block Table: calls methods
- Model Caller → Triton Attention: provides inputs
- Scoping Mask → Triton Attention: provides mask
- Triton Attention → Model Caller: returns output
- Scoping Mask → Reference Attention: provides scope mask