# freim32/vllmops architecture

> Architecture diagram of the GitHub repository freim32/vllmops, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/freim32/vllmops
- Repository: https://github.com/freim32/vllmops

## Overview

vllmops is a local control plane for bare-metal vLLM servers. Users declare models and profiles in YAML, then use the CLI or TUI to create and manage model processes, inspect health/logs/metrics, and optionally expose running models through a LiteLLM gateway. The graph highlights the shared service layer that backs both interfaces, plus diagnostics and TUI views. Some source was not sampled; edges are limited to README statements, shown calls, and SOURCE INDEX references.

Main entry points: [`src/vllmops/tui/app.py`](https://github.com/freim32/vllmops/blob/main/src/vllmops/tui/app.py) (Live TUI), [`src/vllmops/service.py`](https://github.com/freim32/vllmops/blob/main/src/vllmops/service.py) (Shared service), [`src/vllmops/proxy.py`](https://github.com/freim32/vllmops/blob/main/src/vllmops/proxy.py) (Gateway management), [`src/vllmops/cli.py`](https://github.com/freim32/vllmops/blob/main/src/vllmops/cli.py) (CLI commands)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interfaces["User interfaces"]
  node_cli["CLI commands<br/>[cli.py]"]
  node_tui["Live TUI<br/>[app.py]"]
  node_gpu_panel["GPU panel<br/>[gpu_panel.py]"]
  node_metrics_panel["Metrics panel<br/>[metrics_panel.py]"]
  node_log_viewer["Log viewer<br/>[log_viewer.py]"]
  node_highlighter["Log highlighter<br/>[highlighter.py]"]
  node_models_tree["Models tree<br/>[models_tree.py]"]
  node_format["Metric formatting<br/>[_format.py]"]
  node_errors_panel["Errors panel<br/>[errors_panel.py]"]
  node_progress_bar["Progress bar<br/>[progress_bar.py]"]
end

subgraph group_catalog["Project and catalog"]
  node_project["Project discovery<br/>[project.py]"]
  node_project_config["Profiles and proxy settings<br/>[project.py]"]
  node_model_config["Model YAML schema<br/>[config.py]"]
  node_config_errors["Config error summaries<br/>[config_errors.py]"]
end

subgraph group_runtime["Model runtime"]
  node_service["Shared service<br/>[service.py]"]
  node_lifecycle["Process lifecycle<br/>[lifecycle.py]"]
end

subgraph group_observability["Monitoring and diagnostics"]
  node_gpu["GPU stats reader<br/>[gpu.py]"]
  node_metrics["Metrics scraper and history<br/>[metrics.py]"]
  node_doctor["Environment doctor<br/>[doctor.py]"]
end

subgraph group_gateway["Proxy gateway"]
  node_proxy["Gateway management<br/>[proxy.py]"]
end

node_user(("Operator"))
node_vllm["vLLM server"]
node_litellm["LiteLLM gateway"]

node_user -->|"runs commands"| node_cli
node_user -->|"opens dashboard"| node_tui
node_cli -->|"invokes actions"| node_service
node_cli -->|"launches"| node_tui
node_cli -->|"runs checks"| node_doctor
node_cli -->|"manages gateway"| node_proxy
node_tui -->|"reads and acts"| node_service
node_tui -->|"queries stats"| node_gpu
node_tui -->|"scrapes metrics"| node_metrics
node_tui -->|"checks gateway"| node_proxy
node_project -->|"loads"| node_project_config
node_project -->|"formats failures"| node_config_errors
node_model_config -->|"uses project names"| node_project
node_service -->|"uses project paths"| node_project
node_service -->|"loads model definitions"| node_model_config
node_service -->|"manages processes"| node_lifecycle
node_service -->|"spawns servers"| node_vllm
node_vllm -->|"serves metrics"| node_metrics
node_doctor -->|"checks catalog"| node_service
node_doctor -->|"checks GPU"| node_gpu
node_doctor -->|"checks executable"| node_proxy
node_proxy -->|"reads model catalog"| node_service
node_proxy -->|"reads settings"| node_project
node_proxy -->|"manages gateway process"| node_lifecycle
node_project_config -->|"configures gateway"| node_proxy
node_proxy -->|"starts gateway"| node_litellm
node_litellm -->|"routes requests"| node_vllm
node_tui -->|"renders"| node_gpu_panel
node_tui -->|"renders"| node_metrics_panel
node_tui -->|"renders"| node_log_viewer
node_tui -->|"renders"| node_models_tree
node_tui -->|"renders"| node_errors_panel
node_gpu_panel -->|"displays snapshots"| node_gpu
node_gpu_panel -->|"formats values"| node_format
node_metrics_panel -->|"renders history"| node_metrics
node_metrics_panel -->|"formats values"| node_format
node_log_viewer -->|"highlights output"| node_highlighter
node_models_tree -->|"uses catalog views"| node_service

click node_cli "https://github.com/freim32/vllmops/blob/main/src/vllmops/cli.py"
click node_tui "https://github.com/freim32/vllmops/blob/main/src/vllmops/tui/app.py"
click node_service "https://github.com/freim32/vllmops/blob/main/src/vllmops/service.py"
click node_project "https://github.com/freim32/vllmops/blob/main/src/vllmops/project.py"
click node_project_config "https://github.com/freim32/vllmops/blob/main/src/vllmops/project.py"
click node_model_config "https://github.com/freim32/vllmops/blob/main/src/vllmops/config.py"
click node_config_errors "https://github.com/freim32/vllmops/blob/main/src/vllmops/config_errors.py"
click node_lifecycle "https://github.com/freim32/vllmops/blob/main/src/vllmops/lifecycle.py"
click node_gpu "https://github.com/freim32/vllmops/blob/main/src/vllmops/gpu.py"
click node_metrics "https://github.com/freim32/vllmops/blob/main/src/vllmops/metrics.py"
click node_doctor "https://github.com/freim32/vllmops/blob/main/src/vllmops/doctor.py"
click node_proxy "https://github.com/freim32/vllmops/blob/main/src/vllmops/proxy.py"
click node_gpu_panel "https://github.com/freim32/vllmops/blob/main/src/vllmops/tui/widgets/gpu_panel.py"
click node_metrics_panel "https://github.com/freim32/vllmops/blob/main/src/vllmops/tui/widgets/metrics_panel.py"
click node_log_viewer "https://github.com/freim32/vllmops/blob/main/src/vllmops/tui/widgets/log_viewer.py"
click node_highlighter "https://github.com/freim32/vllmops/blob/main/src/vllmops/tui/widgets/highlighter.py"
click node_models_tree "https://github.com/freim32/vllmops/blob/main/src/vllmops/tui/widgets/models_tree.py"
click node_format "https://github.com/freim32/vllmops/blob/main/src/vllmops/tui/widgets/_format.py"
click node_errors_panel "https://github.com/freim32/vllmops/blob/main/src/vllmops/tui/widgets/errors_panel.py"
click node_progress_bar "https://github.com/freim32/vllmops/blob/main/src/vllmops/tui/widgets/progress_bar.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_tui,node_gpu_panel,node_metrics_panel,node_log_viewer,node_highlighter,node_models_tree,node_format,node_errors_panel,node_progress_bar toneBlue
class node_project,node_project_config,node_model_config,node_config_errors toneAmber
class node_service,node_lifecycle,node_vllm toneMint
class node_gpu,node_metrics,node_doctor toneRose
class node_proxy,node_user,node_litellm toneIndigo
```

## Components

### User interfaces

- [CLI commands](https://github.com/freim32/vllmops/blob/main/src/vllmops/cli.py): `src/vllmops/cli.py`
- [Live TUI](https://github.com/freim32/vllmops/blob/main/src/vllmops/tui/app.py): `src/vllmops/tui/app.py`
- [GPU panel](https://github.com/freim32/vllmops/blob/main/src/vllmops/tui/widgets/gpu_panel.py): `src/vllmops/tui/widgets/gpu_panel.py`
- [Metrics panel](https://github.com/freim32/vllmops/blob/main/src/vllmops/tui/widgets/metrics_panel.py): `src/vllmops/tui/widgets/metrics_panel.py`
- [Log viewer](https://github.com/freim32/vllmops/blob/main/src/vllmops/tui/widgets/log_viewer.py): `src/vllmops/tui/widgets/log_viewer.py`
- [Log highlighter](https://github.com/freim32/vllmops/blob/main/src/vllmops/tui/widgets/highlighter.py): `src/vllmops/tui/widgets/highlighter.py`
- [Models tree](https://github.com/freim32/vllmops/blob/main/src/vllmops/tui/widgets/models_tree.py): `src/vllmops/tui/widgets/models_tree.py`
- [Metric formatting](https://github.com/freim32/vllmops/blob/main/src/vllmops/tui/widgets/_format.py): `src/vllmops/tui/widgets/_format.py`
- [Errors panel](https://github.com/freim32/vllmops/blob/main/src/vllmops/tui/widgets/errors_panel.py): `src/vllmops/tui/widgets/errors_panel.py`
- [Progress bar](https://github.com/freim32/vllmops/blob/main/src/vllmops/tui/widgets/progress_bar.py): `src/vllmops/tui/widgets/progress_bar.py`

### Project and catalog

- [Project discovery](https://github.com/freim32/vllmops/blob/main/src/vllmops/project.py): `src/vllmops/project.py`
- [Profiles and proxy settings](https://github.com/freim32/vllmops/blob/main/src/vllmops/project.py): `src/vllmops/project.py`
- [Model YAML schema](https://github.com/freim32/vllmops/blob/main/src/vllmops/config.py): `src/vllmops/config.py`
- [Config error summaries](https://github.com/freim32/vllmops/blob/main/src/vllmops/config_errors.py): `src/vllmops/config_errors.py`

### Model runtime

- [Shared service](https://github.com/freim32/vllmops/blob/main/src/vllmops/service.py): `src/vllmops/service.py`
- [Process lifecycle](https://github.com/freim32/vllmops/blob/main/src/vllmops/lifecycle.py): `src/vllmops/lifecycle.py`

### Monitoring and diagnostics

- [GPU stats reader](https://github.com/freim32/vllmops/blob/main/src/vllmops/gpu.py): `src/vllmops/gpu.py`
- [Metrics scraper and history](https://github.com/freim32/vllmops/blob/main/src/vllmops/metrics.py): `src/vllmops/metrics.py`
- [Environment doctor](https://github.com/freim32/vllmops/blob/main/src/vllmops/doctor.py): `src/vllmops/doctor.py`

### Proxy gateway

- [Gateway management](https://github.com/freim32/vllmops/blob/main/src/vllmops/proxy.py): `src/vllmops/proxy.py`

### Other components

- **Operator**
- **vLLM server**
- **LiteLLM gateway**

## Connections

- Operator → CLI commands: runs commands (evidence: `README.md`)
- Operator → Live TUI: opens dashboard (evidence: `README.md`)
- CLI commands → Shared service: invokes actions (evidence: `src/vllmops/cli.py`)
- CLI commands → Live TUI: launches (evidence: `src/vllmops/cli.py`)
- CLI commands → Environment doctor: runs checks (evidence: `src/vllmops/cli.py`)
- CLI commands → Gateway management: manages gateway (evidence: `src/vllmops/cli.py`)
- Live TUI → Shared service: reads and acts (evidence: `src/vllmops/tui/app.py`)
- Live TUI → GPU stats reader: queries stats (evidence: `src/vllmops/tui/app.py`)
- Live TUI → Metrics scraper and history: scrapes metrics (evidence: `src/vllmops/tui/app.py`)
- Live TUI → Gateway management: checks gateway (evidence: `src/vllmops/tui/app.py`)
- Project discovery → Profiles and proxy settings: loads (evidence: `src/vllmops/project.py`)
- Project discovery → Config error summaries: formats failures (evidence: `src/vllmops/project.py`)
- Model YAML schema → Project discovery: uses project names (evidence: `src/vllmops/config.py`)
- Shared service → Project discovery: uses project paths (evidence: `src/vllmops/service.py`)
- Shared service → Model YAML schema: loads model definitions (evidence: `src/vllmops/service.py`)
- Shared service → Process lifecycle: manages processes (evidence: `src/vllmops/service.py`)
- Shared service → vLLM server: spawns servers (evidence: `README.md`)
- vLLM server → Metrics scraper and history: serves metrics (evidence: `README.md`)
- Environment doctor → Shared service: checks catalog (evidence: `src/vllmops/doctor.py`)
- Environment doctor → GPU stats reader: checks GPU (evidence: `src/vllmops/doctor.py`)
- Environment doctor → Gateway management: checks executable (evidence: `src/vllmops/doctor.py`)
- Gateway management → Shared service: reads model catalog (evidence: `src/vllmops/proxy.py`)
- Gateway management → Project discovery: reads settings (evidence: `src/vllmops/proxy.py`)
- Gateway management → Process lifecycle: manages gateway process (evidence: `src/vllmops/proxy.py`)
- Profiles and proxy settings → Gateway management: configures gateway (evidence: `README.md`)
- Gateway management → LiteLLM gateway: starts gateway (evidence: `README.md`)
- LiteLLM gateway → vLLM server: routes requests (evidence: `README.md`)
- Live TUI → GPU panel: renders
- Live TUI → Metrics panel: renders
- Live TUI → Log viewer: renders
- Live TUI → Models tree: renders
- Live TUI → Errors panel: renders
- GPU panel → GPU stats reader: displays snapshots (evidence: `src/vllmops/tui/widgets/gpu_panel.py`)
- GPU panel → Metric formatting: formats values (evidence: `src/vllmops/tui/widgets/gpu_panel.py`)
- Metrics panel → Metrics scraper and history: renders history (evidence: `src/vllmops/tui/widgets/metrics_panel.py`)
- Metrics panel → Metric formatting: formats values (evidence: `src/vllmops/tui/widgets/metrics_panel.py`)
- Log viewer → Log highlighter: highlights output (evidence: `src/vllmops/tui/widgets/log_viewer.py`)
- Models tree → Shared service: uses catalog views (evidence: `src/vllmops/tui/widgets/models_tree.py`)
