How rayonlabs/chutes works
Project Type and Purpose
• This is a Python-based SDK and CLI tool (“chutes”) for building, deploying, and running containerized AI applications (“chutes”) on the chutes.ai platform.
• It provides:
• A local CLI client for user workflows (register, build, deploy, run)
• A library for programmatically constructing Docker images (chutes.image)
• A library for defining FastAPI-style applications (“chutes”) via decorators (chutes.chute)
• Integration points for authentication (Bittensor hotkeys) and GPU validation (GraVal)
Main Components
1) Developer / Client Layer
• “chutes” CLI (chutes/cli.py + entrypoint/* subcommands)
• SDK modules (config, constants, util, crud)
• Local Docker context (–local builds, examples/)
2) Control-Plane / Platform Layer
• Chutes API server (external microservice, not in this repo)
• Remote build service (receives Docker context, builds images)
• Container registry (stores built images)
• Orchestrator / scheduler (schedules containers on GPU hosts)
3) Data-Plane / Compute Layer
• GPU Miner Nodes
• GraVal middleware inside each running chute for GPU authenticity
• FastAPI application processes (cord endpoints)
Relationships and Data Flows
• Developer invokes chutes <command> → local CLI parses command via entrypoint modules →
– HTTP call to Chutes API for auth, registration, key management
– HTTP call to remote build service (if build --wait or remote build) with Docker directives from chutes.image.Image
– HTTP call to deploy endpoint with metadata from chutes.chute (template, NodeSelector, concurrency)
• Remote build service → builds Docker image → pushes to container registry → notifies API server
• Deploy API → orchestrator schedules container on a miner → miner pulls image → starts FastAPI server
• Client applications or end users → HTTP calls to deployed chute’s public_api_path endpoints → requests handled by FastAPI methods decorated via @chute.cord → compute on GPU
• At container startup, GraVal middleware decrypts/validates traffic with validator service
Architectural Patterns & Design Principles
• Microservices: separate API server, build service, orchestrator
• Command Pattern: CLI entrypoints modularized under chutes/entrypoint
• Plugin/Template System: chutes.chute.template.* and chutes.image.directive.*
• Decorator-driven routing: FastAPI-like @chute.cord for automatic OpenAPI schema generation
• Containerization: Docker image DSL built in Python, remote vs local execution contexts
• Security: Bittensor hotkey auth, GraVal GPU validation