How sybil-solutions/inference-optimisation-skill works
This repository is an agent skill and lightweight harness for optimizing local LLM inference, not an inference server. Its main workflow estimates performance ceilings, benchmarks a baseline, iterates on one tuning lever using interleaved controls, records results, then validates stability and qualifies a pinned build before publishing. The README also describes memory monitoring and engine-specific guidance. Only ceiling.py was excerpted; other module relationships below are limited to capabilities and workflow links explicitly documented in the README.
Main entry points: SKILL.md Optimization procedure · bench.py Server benchmark · examples Benchmark configurations · queue.sh Serialized run queue
Components 21
Procedure and guidance
- Optimization procedure
skills/inference-optimisation/SKILL.md - Agent task prompt
PROMPT.md - Ceiling methodology
skills/inference-optimisation/references/ceilings.md - Measurement methodology
skills/inference-optimisation/references/methodology.md - Tuning levers
skills/inference-optimisation/references/levers.md - Engine gotchas
skills/inference-optimisation/references/gotchas.md - Publishing guidance
skills/inference-optimisation/references/publishing.md
Measurement tools
- Ceiling calculator
skills/inference-optimisation/scripts/ceiling.py - Server benchmark
skills/inference-optimisation/scripts/bench.py - Memory monitor
skills/inference-optimisation/scripts/mem.sh
Experiment workflow
- Benchmark configurations
skills/inference-optimisation/examples - Serialized run queue
skills/inference-optimisation/scripts/queue.sh - Result ledger
- Tuning plan
Validation and release
- Long-context retrieval gate
skills/inference-optimisation/scripts/bench.py - Full-KV concurrency soak
skills/inference-optimisation/SKILL.md - Pinned clean build
skills/inference-optimisation/references/publishing.md - Acceptance gates
skills/inference-optimisation/references/publishing.md - Published results
skills/inference-optimisation/references/publishing.md
Other components
- Engineer or agent
- OpenAI-compatible server
Connections 23
- Engineer or agent → Optimization procedure: follows procedure
README.md - Engineer or agent → Agent task prompt: uses prompt
README.md - Optimization procedure → Ceiling methodology: consults
README.md - Optimization procedure → Measurement methodology: consults
README.md - Optimization procedure → Tuning levers: consults
README.md - Optimization procedure → Engine gotchas: consults
README.md - Optimization procedure → Publishing guidance: consults
README.md - Engineer or agent → Ceiling calculator: computes ceilings
README.md - Benchmark configurations → Ceiling calculator: provides inputs
README.md - Engineer or agent → Server benchmark: runs benchmark
README.md - Benchmark configurations → Server benchmark: configures run
README.md - Server benchmark → OpenAI-compatible server: sends requests
README.md - Server benchmark → Result ledger: logs results
README.md - Engineer or agent → Memory monitor: checks footprint
README.md - Engineer or agent → Serialized run queue: queues runs
README.md - Serialized run queue → Server benchmark: runs controls
README.md - Result ledger → Tuning plan: informs tuning
README.md - Tuning plan → Engineer or agent: guides next change
README.md - Server benchmark → Long-context retrieval gate: checks retrieval
README.md - Long-context retrieval gate → Full-KV concurrency soak: precedes soak
README.md - Full-KV concurrency soak → Pinned clean build: precedes qualification
README.md - Pinned clean build → Acceptance gates: runs gates
README.md - Acceptance gates → Published results: qualifies results
README.md