How tuttlebr/genai-perf-k6 works
Project Purpose and Type
• The system is not a traditional full-stack web application but instead a specialized service for LLM inference deployed in a cloud-native/Kubernetes environment.
• Its primary goals include deploying an LLM service (using NVIDIA’s model containers), configuring resource constraints (especially for GPU-enabled nodes), and carrying out performance and load testing using Triton Inference Server SDK and Grafana K6.
• The environment variables outline the parameters for request payloads, token statistics, and performance metrics, while the README includes detailed Kubernetes/Helm instructions for deployment and performance test scripts.
Analysis of the File Structure
• Top-level directories and files such as .gitignore, .pre-commit-config.yaml, and README.md are standard; however, the key directories/files are:
• The “app” folder with the script.js, which is the k6 load testing script.
• docker-compose.yaml, which is used to run the performance tests (both the “triton-genai-perf” and “k6” tasks).
• The “nginx/templates” directory containing the default configuration template for Nginx, indicating that a reverse proxy or ingress controller is involved.
• The presence of a shell script (triton-genai-perf.sh) hints at performance testing automation with Triton’s inference benchmarks.