How jeremyfrc/operationniobium_lab works
OperationNiobium_Lab is a mixed repository of transformer and paged-KV inference kernels alongside CS149 coursework and kernel-challenge materials; its README does not describe a single end-user product. The clearest runtime workflow is a caller providing prompt token IDs to transformer generation, which runs decoder layers and returns generated IDs. Transformer internals and paged-cache management are represented separately where integration is not evidenced. The coursework branches are independent exercises, not stages in that workflow. Connections are limited to relationships visible in excerpts or the source index; unsampled implementation details are not inferred.
Main entry points: transformer.cpp Decoder Stack · attention.cpp Attention · main.rs Challenge Application · sequence_manager.cpp Sequence Capacity
Components 23
Transformer Inference
- Autoregressive Generation
kernels/models/transformer/src/transformer.cpp - Decoder Stack
kernels/models/transformer/src/transformer.cpp - Attention
kernels/models/transformer/src/attention.cpp - Tensor Storage
kernels/models/transformer/src/tensor.cpp - Tensor Operations
kernels/models/transformer/src/ops.cpp - RoPE Encoding
kernels/models/transformer/src/rope.cpp - SwiGLU Feed-Forward
kernels/models/transformer/src/swiglu.cpp - RMS Normalization
kernels/models/transformer/src/rmsnorm.cpp
Paged KV Cache
- Paged Attention
kernels/models/transformer/src/attention_paged.cpp - Paged Forward Pass
kernels/models/transformer/src/forward_paged.cpp - Sequence Capacity
kernels/paged_kv/src/sequence_manager.cpp - Block Table
kernels/paged_kv/src/block_table.cpp - Paged KV Store
kernels/paged_kv/src/paged_kv_cache.cpp
CS149 Coursework
- Vector Intrinsics
lectures/cs149/asst1/prog2_vecintrin/main.cpp - ISPC Mandelbrot
lectures/cs149/asst1/prog3_mandelbrot_ispc/main.cpp - Circle Renderer
lectures/cs149/asst3/render/main.cpp - Parallel Scan
lectures/cs149/asst3/scan/main.cpp
Kernel Challenges
- Kernel Problems
lectures/cs149/asst5-kernels/problems - Challenge Application
lectures/cs149/asst5-kernels/src/main.rs - Challenge Commands
lectures/cs149/asst5-kernels/src/cmd - Challenge Service
lectures/cs149/asst5-kernels/src/service - Challenge Views
lectures/cs149/asst5-kernels/src/views
Other components
- Model Caller
Connections 16
- Model Caller → Autoregressive Generation: provides prompt
- Autoregressive Generation → Decoder Stack: runs decoding
kernels/models/transformer/src/transformer.cpp - Decoder Stack → Attention: calls attention
kernels/models/transformer/src/transformer.cpp - Decoder Stack → SwiGLU Feed-Forward: calls feed-forward
kernels/models/transformer/src/transformer.cpp - Decoder Stack → RMS Normalization: calls normalization
kernels/models/transformer/src/transformer.cpp - Attention → Tensor Operations: calls operations
kernels/models/transformer/src/attention.cpp - Attention → RoPE Encoding: applies encoding
kernels/models/transformer/src/attention.cpp - Attention → Tensor Storage: uses tensors
kernels/models/transformer/src/attention.cpp - Sequence Capacity → Block Table: reserves capacity
kernels/paged_kv/src/sequence_manager.cpp - Sequence Capacity → Block Table: advances length
kernels/paged_kv/src/sequence_manager.cpp - Paged Attention → Paged KV Store: uses KV cache
- Paged Forward Pass → Paged Attention: dispatches attention
- Vector Intrinsics → ISPC Mandelbrot: independent exercise
- Challenge Application → Challenge Commands: imports commands
lectures/cs149/asst5-kernels/src/main.rs - Challenge Application → Challenge Service: imports service
lectures/cs149/asst5-kernels/src/main.rs - Challenge Application → Challenge Views: imports views
lectures/cs149/asst5-kernels/src/main.rs