How hec-ovi/llama-vulkan-strix works
This repository packages llama.cpp inference for AMD Strix Halo rather than implementing a model server. The main workflow is to configure model paths and context in .env, select a Compose package, start the chosen stack, and send requests to its API. Documented branches include stock Vulkan, ROCmFP4+MTP, and Laguna ROCmFPX; supporting tools validate context, inspect GTT placement, and benchmark served models. The Compose files and custom entrypoint were not sampled, so their internal wiring is represented only where the README documents it.
Main entry points: .env.example Model settings · docker-compose.laguna-rocmfpx.yml Laguna ROCmFPX · bench_server.py Depth benchmark · bench_full.py Full benchmark
Components 15
Inference deployment
- Model settings
.env.example - Vulkan stack
docker-compose.yml - Model packages
compose/models - ROCmFP4 stack
docker-compose.rocmfp4.yml - Laguna ROCmFPX
docker-compose.laguna-rocmfpx.yml - Laguna entrypoint
tools/entrypoint.laguna-rocmfpx.sh
Runtime operations
- Context validator
scripts/check_context_config.py - GPU memory report
scripts/gpu_mem.py - GTT verification
scripts/verify-gtt.sh
Performance measurement
- Depth benchmark
scripts/bench_server.py - Full benchmark
scripts/bench_full.py - Benchmark results
docs/bench-results
Other components
- Operator
- llama.cpp server
- API client
Connections 20
- Operator → Model settings: sets options
README.md - Operator → Model packages: selects package
README.md - Model settings → Vulkan stack: configures
README.md - Model packages → Vulkan stack: extends stack
README.md - Model settings → ROCmFP4 stack: configures
README.md - Model settings → Laguna ROCmFPX: configures
README.md - Laguna ROCmFPX → Laguna entrypoint: uses entrypoint
- Vulkan stack → llama.cpp server: starts server
README.md - ROCmFP4 stack → llama.cpp server: starts server
README.md - Laguna ROCmFPX → llama.cpp server: starts server
README.md - API client → llama.cpp server: sends requests
README.md - Operator → Context validator: validates settings
README.md - Operator → GPU memory report: checks memory
README.md - Operator → GTT verification: verifies placement
README.md - Operator → Depth benchmark: runs benchmark
scripts/bench_server.py - Depth benchmark → llama.cpp server: requests completion
scripts/bench_server.py - Operator → Full benchmark: runs benchmark
scripts/bench_full.py - Full benchmark → llama.cpp server: requests completion
scripts/bench_full.py - Depth benchmark → Benchmark results: records metrics
- Full benchmark → Benchmark results: records metrics