# rktim/infra architecture

> Architecture diagram of the GitHub repository rktim/infra, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rktim/infra
- Repository: https://github.com/rktim/infra

## Overview

infra is a local model-planning and serving tool. The main user workflow is to inspect hardware, discover and rank models that fit a workload, download a selection, then serve it through vLLM or run in-process inference. The graph also preserves the interactive dashboard and model/API capabilities documented in the README. Internal edges are limited to relationships visible in excerpts or the source index; unsampled implementation details are not inferred.

Main entry points: [`infra/cli.py`](https://github.com/rktim/infra/blob/main/infra/cli.py) (CLI), [`infra/recommender.py`](https://github.com/rktim/infra/blob/main/infra/recommender.py) (Recommendations), [`infra/tui.py`](https://github.com/rktim/infra/blob/main/infra/tui.py) (Terminal dashboard), [`infra/server.py`](https://github.com/rktim/infra/blob/main/infra/server.py) (Server manager)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["User interfaces"]
  node_cli["CLI<br/>[cli.py]"]
  node_dashboard["Terminal dashboard<br/>[tui.py]"]
end

subgraph group_planning["Model planning"]
  node_hardware["Hardware detection<br/>[hardware.py]"]
  node_metadata["Hub metadata<br/>[metadata.py]"]
  node_discovery["Model discovery<br/>[discovery.py]"]
  node_estimate["Memory estimation<br/>[estimator.py]"]
  node_compatibility["Fit checking<br/>[compatibility.py]"]
  node_optimizer["Serving optimizer<br/>[optimizer.py]"]
  node_recommend["Recommendations<br/>[recommender.py]"]
  node_suggestor["Model suggestions<br/>[suggestor.py]"]
end

subgraph group_assets["Model access"]
  node_modelinfo["Model inspection<br/>[models.py]"]
  node_download["Model downloader<br/>[downloader.py]"]
end

subgraph group_serving["Inference and serving"]
  node_backend["Inference backend<br/>[backend.py]"]
  node_vllm["vLLM runtime<br/>[vllm.py]"]
  node_server["Server manager<br/>[server.py]"]
  node_api["API support<br/>[api.py]"]
  node_runtime["Runtime profiling<br/>[runtime.py]"]
end

subgraph group_support["Local state"]
  node_cache[("Metadata cache<br/>[cache.py]")]
end

node_user(("User"))
node_hub["Hugging Face Hub"]
node_local["Local machine"]

node_user -->|"runs commands"| node_cli
node_user -->|"opens dashboard"| node_dashboard
node_cli -->|"inspects hardware"| node_hardware
node_cli -->|"requests recommendations"| node_recommend
node_cli -->|"checks memory"| node_estimate
node_cli -->|"downloads model"| node_download
node_cli -->|"manages server"| node_server
node_cli -->|"runs inference"| node_vllm
node_recommend -->|"shortlists models"| node_discovery
node_recommend -->|"gets metadata"| node_metadata
node_recommend -->|"estimates memory"| node_estimate
node_recommend -->|"builds runtime budget"| node_hardware
node_recommend -->|"checks fit"| node_compatibility
node_recommend -.->|"tunes serving"| node_optimizer
node_metadata -->|"reads and writes"| node_cache
node_discovery -.->|"uses cache"| node_cache
node_metadata -->|"fetches model data"| node_hub
node_modelinfo -->|"reads model files"| node_hub
node_download -->|"downloads weights"| node_hub
node_dashboard -->|"shows telemetry"| node_hardware
node_dashboard -->|"shows recommendations"| node_recommend
node_dashboard -->|"starts downloads"| node_download
node_dashboard -->|"manages daemons"| node_server
node_dashboard -->|"shows endpoint status"| node_api
node_dashboard -.->|"uses suggestions"| node_suggestor
node_vllm -->|"implements backend"| node_backend
node_server -->|"starts daemon"| node_local
node_server -->|"checks process"| node_local
node_vllm -->|"loads model"| node_local
node_cli -.->|"profiles inference"| node_runtime

click node_cli "https://github.com/rktim/infra/blob/main/infra/cli.py"
click node_dashboard "https://github.com/rktim/infra/blob/main/infra/tui.py"
click node_hardware "https://github.com/rktim/infra/blob/main/infra/hardware.py"
click node_metadata "https://github.com/rktim/infra/blob/main/infra/metadata.py"
click node_discovery "https://github.com/rktim/infra/blob/main/infra/discovery.py"
click node_estimate "https://github.com/rktim/infra/blob/main/infra/estimator.py"
click node_compatibility "https://github.com/rktim/infra/blob/main/infra/compatibility.py"
click node_optimizer "https://github.com/rktim/infra/blob/main/infra/optimizer.py"
click node_recommend "https://github.com/rktim/infra/blob/main/infra/recommender.py"
click node_modelinfo "https://github.com/rktim/infra/blob/main/infra/models.py"
click node_download "https://github.com/rktim/infra/blob/main/infra/downloader.py"
click node_cache "https://github.com/rktim/infra/blob/main/infra/cache.py"
click node_backend "https://github.com/rktim/infra/blob/main/infra/backend.py"
click node_vllm "https://github.com/rktim/infra/blob/main/infra/vllm.py"
click node_server "https://github.com/rktim/infra/blob/main/infra/server.py"
click node_api "https://github.com/rktim/infra/blob/main/infra/api.py"
click node_suggestor "https://github.com/rktim/infra/blob/main/infra/suggestor.py"
click node_runtime "https://github.com/rktim/infra/blob/main/infra/runtime.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_dashboard,node_user toneBlue
class node_hardware,node_metadata,node_discovery,node_estimate,node_compatibility,node_optimizer,node_recommend,node_suggestor toneAmber
class node_modelinfo,node_download toneMint
class node_backend,node_vllm,node_server,node_api,node_runtime toneRose
class node_cache,node_hub,node_local toneIndigo
```

## Components

### User interfaces

- [CLI](https://github.com/rktim/infra/blob/main/infra/cli.py): `infra/cli.py`
- [Terminal dashboard](https://github.com/rktim/infra/blob/main/infra/tui.py): `infra/tui.py`

### Model planning

- [Hardware detection](https://github.com/rktim/infra/blob/main/infra/hardware.py): `infra/hardware.py`
- [Hub metadata](https://github.com/rktim/infra/blob/main/infra/metadata.py): `infra/metadata.py`
- [Model discovery](https://github.com/rktim/infra/blob/main/infra/discovery.py): `infra/discovery.py`
- [Memory estimation](https://github.com/rktim/infra/blob/main/infra/estimator.py): `infra/estimator.py`
- [Fit checking](https://github.com/rktim/infra/blob/main/infra/compatibility.py): `infra/compatibility.py`
- [Serving optimizer](https://github.com/rktim/infra/blob/main/infra/optimizer.py): `infra/optimizer.py`
- [Recommendations](https://github.com/rktim/infra/blob/main/infra/recommender.py): `infra/recommender.py`
- [Model suggestions](https://github.com/rktim/infra/blob/main/infra/suggestor.py): `infra/suggestor.py`

### Model access

- [Model inspection](https://github.com/rktim/infra/blob/main/infra/models.py): `infra/models.py`
- [Model downloader](https://github.com/rktim/infra/blob/main/infra/downloader.py): `infra/downloader.py`

### Inference and serving

- [Inference backend](https://github.com/rktim/infra/blob/main/infra/backend.py): `infra/backend.py`
- [vLLM runtime](https://github.com/rktim/infra/blob/main/infra/vllm.py): `infra/vllm.py`
- [Server manager](https://github.com/rktim/infra/blob/main/infra/server.py): `infra/server.py`
- [API support](https://github.com/rktim/infra/blob/main/infra/api.py): `infra/api.py`
- [Runtime profiling](https://github.com/rktim/infra/blob/main/infra/runtime.py): `infra/runtime.py`

### Local state

- [Metadata cache](https://github.com/rktim/infra/blob/main/infra/cache.py): `infra/cache.py`

### Other components

- **User**
- **Hugging Face Hub**
- **Local machine**

## Connections

- User → CLI: runs commands (evidence: `README.md`)
- User → Terminal dashboard: opens dashboard (evidence: `README.md`)
- CLI → Hardware detection: inspects hardware (evidence: `README.md`)
- CLI → Recommendations: requests recommendations (evidence: `README.md`)
- CLI → Memory estimation: checks memory (evidence: `README.md`)
- CLI → Model downloader: downloads model (evidence: `README.md`)
- CLI → Server manager: manages server (evidence: `README.md`)
- CLI → vLLM runtime: runs inference (evidence: `README.md`)
- Recommendations → Model discovery: shortlists models (evidence: `infra/recommender.py`)
- Recommendations → Hub metadata: gets metadata (evidence: `infra/recommender.py`)
- Recommendations → Memory estimation: estimates memory (evidence: `infra/recommender.py`)
- Recommendations → Hardware detection: builds runtime budget (evidence: `infra/recommender.py`)
- Recommendations → Fit checking: checks fit (evidence: `infra/recommender.py`)
- Recommendations → Serving optimizer: tunes serving (evidence: `infra/recommender.py`)
- Hub metadata → Metadata cache: reads and writes (evidence: `infra/metadata.py`)
- Model discovery → Metadata cache: uses cache (evidence: `infra/discovery.py`)
- Hub metadata → Hugging Face Hub: fetches model data (evidence: `infra/metadata.py`)
- Model inspection → Hugging Face Hub: reads model files (evidence: `infra/models.py`)
- Model downloader → Hugging Face Hub: downloads weights (evidence: `README.md`)
- Terminal dashboard → Hardware detection: shows telemetry (evidence: `README.md`)
- Terminal dashboard → Recommendations: shows recommendations (evidence: `README.md`)
- Terminal dashboard → Model downloader: starts downloads (evidence: `infra/tui.py`)
- Terminal dashboard → Server manager: manages daemons (evidence: `README.md`)
- Terminal dashboard → API support: shows endpoint status (evidence: `README.md`)
- Terminal dashboard → Model suggestions: uses suggestions (evidence: `infra/tui.py`)
- vLLM runtime → Inference backend: implements backend (evidence: `infra/vllm.py`)
- Server manager → Local machine: starts daemon (evidence: `README.md`)
- Server manager → Local machine: checks process (evidence: `infra/server.py`)
- vLLM runtime → Local machine: loads model (evidence: `infra/vllm.py`)
- CLI → Runtime profiling: profiles inference
