How rktim/infra works
infra is a local model-planning and serving tool. The main user workflow is to inspect hardware, discover and rank models that fit a workload, download a selection, then serve it through vLLM or run in-process inference. The graph also preserves the interactive dashboard and model/API capabilities documented in the README. Internal edges are limited to relationships visible in excerpts or the source index; unsampled implementation details are not inferred.
Main entry points: cli.py CLI · recommender.py Recommendations · tui.py Terminal dashboard · server.py Server manager
Components 21
User interfaces
- CLI
infra/cli.py - Terminal dashboard
infra/tui.py
Model planning
- Hardware detection
infra/hardware.py - Hub metadata
infra/metadata.py - Model discovery
infra/discovery.py - Memory estimation
infra/estimator.py - Fit checking
infra/compatibility.py - Serving optimizer
infra/optimizer.py - Recommendations
infra/recommender.py - Model suggestions
infra/suggestor.py
Model access
- Model inspection
infra/models.py - Model downloader
infra/downloader.py
Inference and serving
- Inference backend
infra/backend.py - vLLM runtime
infra/vllm.py - Server manager
infra/server.py - API support
infra/api.py - Runtime profiling
infra/runtime.py
Local state
- Metadata cache
infra/cache.py
Other components
- User
- Hugging Face Hub
- Local machine
Connections 30
- User → CLI: runs commands
README.md - User → Terminal dashboard: opens dashboard
README.md - CLI → Hardware detection: inspects hardware
README.md - CLI → Recommendations: requests recommendations
README.md - CLI → Memory estimation: checks memory
README.md - CLI → Model downloader: downloads model
README.md - CLI → Server manager: manages server
README.md - CLI → vLLM runtime: runs inference
README.md - Recommendations → Model discovery: shortlists models
infra/recommender.py - Recommendations → Hub metadata: gets metadata
infra/recommender.py - Recommendations → Memory estimation: estimates memory
infra/recommender.py - Recommendations → Hardware detection: builds runtime budget
infra/recommender.py - Recommendations → Fit checking: checks fit
infra/recommender.py - Recommendations → Serving optimizer: tunes serving
infra/recommender.py - Hub metadata → Metadata cache: reads and writes
infra/metadata.py - Model discovery → Metadata cache: uses cache
infra/discovery.py - Hub metadata → Hugging Face Hub: fetches model data
infra/metadata.py - Model inspection → Hugging Face Hub: reads model files
infra/models.py - Model downloader → Hugging Face Hub: downloads weights
README.md - Terminal dashboard → Hardware detection: shows telemetry
README.md - Terminal dashboard → Recommendations: shows recommendations
README.md - Terminal dashboard → Model downloader: starts downloads
infra/tui.py - Terminal dashboard → Server manager: manages daemons
README.md - Terminal dashboard → API support: shows endpoint status
README.md - Terminal dashboard → Model suggestions: uses suggestions
infra/tui.py - vLLM runtime → Inference backend: implements backend
infra/vllm.py - Server manager → Local machine: starts daemon
README.md - Server manager → Local machine: checks process
infra/server.py - vLLM runtime → Local machine: loads model
infra/vllm.py - CLI → Runtime profiling: profiles inference