# valentinfrlch/ha-llmvision architecture

> Architecture diagram of the GitHub repository valentinfrlch/ha-llmvision, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/valentinfrlch/ha-llmvision
- Repository: https://github.com/valentinfrlch/ha-llmvision

## Overview

#### Purpose and Host Runtime
LLM Vision is a Python custom integration for Home Assistant, installed through HACS. It analyzes camera images, videos, live feeds, and Frigate events with multimodal LLMs; it can retain recognized entities, update Home Assistant state, and publish event history. The integration boundary and Home Assistant lifecycle entry point are `custom_components/llmvision/__init__.py` and `custom_components/llmvision/manifest.json`.

#### Setup, Configuration, and Service Surface
Home Assistant config entries drive initial setup and provider creation through `custom_components/llmvision/config_flow.py`. The integration exposes its user-facing automation/service contract in `custom_components/llmvision/services.yaml`; constants and identifiers shared across modules reside in `custom_components/llmvision/const.py`. Blueprints provide a ready-made event-summary automation: `blueprints/event_summary.yaml` and `blueprints/event_summary_beta.yaml`.

#### Request Orchestration
`custom_components/llmvision/api.py` is the likely central application API: it accepts analysis requests, coordinates media preparation, provider invocation, and downstream result handling. This is the principal control boundary between Home Assistant automations/services and LLM-specific implementation. `custom_components/llmvision/__init__.py` registers the integration-level facilities that make this API available at runtime.

#### Media Acquisition and Normalization
`custom_components/llmvision/media_handlers.py` forms the media boundary for camera feeds, images, video files, snapshots, and Frigate-originated material. It isolates format/source handling before payloads reach model providers. Persistent snapshot/media storage is expected to use Home Assistant’s `/media` area, an operational requirement noted by the project setup guidance.

#### Multi-Provider LLM Adapter Layer
`custom_components/llmvision/providers.py` abstracts calls to cloud, local, and OpenAI-compatible multimodal backends. Supported external services include OpenRouter, OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure, Groq, Ollama, Open WebUI, and LocalAI. Configuration selects credentials/endpoints and model behavior, while the API layer should remain provider-neutral.

#### Memory and Derived Home Automation Data
`custom_components/llmvision/memory.py` manages retained knowledge about people, pets, and objects across analyses. Analysis responses can therefore serve both immediate prompts and longer-lived contextual recognition. The service definitions in `custom_components/llmvision/services.yaml` are the key contract for passing extracted results into Home Assistant automations and sensor/state updates.

#### Timeline, Calendar, and Consumer Interfaces
`custom_components/llmvision/timeline.py` persists or exposes analyzed camera-event history for dashboard and Assist-style queries. `custom_components/llmvision/calendar.py` provides a Home Assistant calendar-facing projection of those events. The external timeline API contract is captured in `timeline_openapi_specs.yaml`; localized timeline UI text lives under `custom_components/llmvision/timeline_strings/`.

#### Delivery and Validation
The integration is distributed as a HACS repository (`hacs.json`) and relies on Home Assistant’s Python runtime and configuration-entry framework. CI validates and tests the package through `.github/workflows/tests.yaml` and `.github/workflows/validate.yaml`; the test suite is structured around the major architectural modules in `tests/`.

Main entry points: [`custom_components/llmvision/api.py`](https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/api.py) (Analysis API), [`custom_components/llmvision/__init__.py`](https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/__init__.py) (Integration Entry), [`custom_components/llmvision/timeline.py`](https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/timeline.py) (Event Timeline), [`custom_components/llmvision/const.py`](https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/const.py) (Shared Constants)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_integration["Home Assistant Integration"]
  node_integration_entry["Integration Entry<br/>lifecycle<br/>[__init__.py]"]
  node_manifest["Integration Manifest<br/>metadata<br/>[manifest.json]"]
  node_config_flow["Config Flow<br/>setup<br/>[config_flow.py]"]
  node_services["Automation Services<br/>service contract<br/>[services.yaml]"]
  node_constants["Shared Constants<br/>identifiers<br/>[const.py]"]
  node_event_blueprints["Event Summary Blueprints<br/>automation templates<br/>[event_summary.yaml]"]
end

subgraph group_analysis["Analysis Pipeline"]
  node_analysis_api["Analysis API<br/>orchestrator<br/>[api.py]"]
  node_media_handlers["Media Handlers<br/>media boundary<br/>[media_handlers.py]"]
  node_providers["Provider Adapters<br/>LLM adapter layer<br/>[providers.py]"]
  node_memory[("Recognition Memory<br/>retained knowledge<br/>[memory.py]")]
end

subgraph group_history["Event History"]
  node_timeline[("Event Timeline<br/>event history<br/>[timeline.py]")]
  node_calendar["Calendar Projection<br/>HA calendar adapter<br/>[calendar.py]"]
  node_timeline_api_spec["Timeline API Spec<br/>API contract"]
  node_timeline_ui_strings["Timeline UI Text<br/>localized resources"]
end

subgraph group_external["External Systems"]
  node_ha_runtime{{"Home Assistant Runtime<br/>host runtime"}}
  node_media_storage[("HA Media Storage<br/>persistent media")]
  node_llm_backends{{"Multimodal LLM Backends<br/>external AI services"}}
  node_ci["CI Validation<br/>delivery pipeline<br/>[tests.yaml]"]
end

node_ha_runtime -->|"loads"| node_integration_entry
node_manifest -->|"declares"| node_integration_entry
node_integration_entry -->|"starts setup"| node_config_flow
node_integration_entry -->|"registers"| node_analysis_api
node_services -->|"invokes"| node_analysis_api
node_event_blueprints -->|"uses"| node_services
node_constants -.->|"shared by"| node_integration_entry
node_constants -.->|"shared by"| node_analysis_api
node_analysis_api -->|"prepares media"| node_media_handlers
node_media_handlers -->|"reads and writes"| node_media_storage
node_analysis_api -->|"requests analysis"| node_providers
node_providers -->|"calls"| node_llm_backends
node_analysis_api -->|"reads and updates"| node_memory
node_analysis_api -->|"records results"| node_timeline
node_timeline -->|"projects events"| node_calendar
node_timeline_api_spec -.->|"defines API"| node_timeline
node_timeline_ui_strings -.->|"localizes UI"| node_timeline
node_calendar -->|"publishes calendar"| node_ha_runtime
node_analysis_api -->|"updates state for automations"| node_ha_runtime
node_ci -.->|"tests and validates"| node_integration_entry

click node_integration_entry "https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/__init__.py"
click node_manifest "https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/manifest.json"
click node_config_flow "https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/config_flow.py"
click node_services "https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/services.yaml"
click node_constants "https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/const.py"
click node_analysis_api "https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/api.py"
click node_media_handlers "https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/media_handlers.py"
click node_providers "https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/providers.py"
click node_memory "https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/memory.py"
click node_timeline "https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/timeline.py"
click node_calendar "https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/calendar.py"
click node_timeline_api_spec "https://github.com/valentinfrlch/ha-llmvision/blob/main/timeline_openapi_specs.yaml"
click node_event_blueprints "https://github.com/valentinfrlch/ha-llmvision/blob/main/blueprints/event_summary.yaml"
click node_ci "https://github.com/valentinfrlch/ha-llmvision/blob/main/.github/workflows/tests.yaml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_integration_entry,node_manifest,node_config_flow,node_services,node_constants,node_event_blueprints toneBlue
class node_analysis_api,node_media_handlers,node_providers,node_memory toneAmber
class node_timeline,node_calendar,node_timeline_api_spec,node_timeline_ui_strings toneMint
class node_ha_runtime,node_media_storage,node_llm_backends,node_ci toneRose
```

## Components

### Home Assistant Integration

- [Integration Entry](https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/__init__.py): lifecycle · `custom_components/llmvision/__init__.py`
- [Integration Manifest](https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/manifest.json): metadata · `custom_components/llmvision/manifest.json`
- [Config Flow](https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/config_flow.py): setup · `custom_components/llmvision/config_flow.py`
- [Automation Services](https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/services.yaml): service contract · `custom_components/llmvision/services.yaml`
- [Shared Constants](https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/const.py): identifiers · `custom_components/llmvision/const.py`
- [Event Summary Blueprints](https://github.com/valentinfrlch/ha-llmvision/blob/main/blueprints/event_summary.yaml): automation templates · Ready-made event-summary automations. · `blueprints/event_summary.yaml`

### Analysis Pipeline

- [Analysis API](https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/api.py): orchestrator · Coordinates media preparation, model calls, and result handling. · `custom_components/llmvision/api.py`
- [Media Handlers](https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/media_handlers.py): media boundary · Normalizes camera, image, video, snapshot, and Frigate inputs. · `custom_components/llmvision/media_handlers.py`
- [Provider Adapters](https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/providers.py): LLM adapter layer · Provider-neutral adapters for cloud, local, and compatible multimodal models. · `custom_components/llmvision/providers.py`
- [Recognition Memory](https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/memory.py): retained knowledge · Retains recognized people, pets, and objects across analyses. · `custom_components/llmvision/memory.py`

### Event History

- [Event Timeline](https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/timeline.py): event history · Persists and exposes analyzed camera-event history. · `custom_components/llmvision/timeline.py`
- [Calendar Projection](https://github.com/valentinfrlch/ha-llmvision/blob/main/custom_components/llmvision/calendar.py): HA calendar adapter · `custom_components/llmvision/calendar.py`
- [Timeline API Spec](https://github.com/valentinfrlch/ha-llmvision/blob/main/timeline_openapi_specs.yaml): API contract · `timeline_openapi_specs.yaml`
- **Timeline UI Text**: localized resources

### External Systems

- **Home Assistant Runtime**: host runtime · Hosts configuration entries, automations, state, and calendar entities.
- **HA Media Storage**: persistent media · Home Assistant /media area for snapshots and retained media.
- **Multimodal LLM Backends**: external AI services · OpenAI, Anthropic, Gemini, Bedrock, OpenRouter, Ollama, and compatible endpoints.
- [CI Validation](https://github.com/valentinfrlch/ha-llmvision/blob/main/.github/workflows/tests.yaml): delivery pipeline · Tests and validates the integration package. · `.github/workflows/tests.yaml`

## Connections

- Home Assistant Runtime → Integration Entry: loads
- Integration Manifest → Integration Entry: declares
- Integration Entry → Config Flow: starts setup
- Integration Entry → Analysis API: registers
- Automation Services → Analysis API: invokes
- Event Summary Blueprints → Automation Services: uses
- Shared Constants → Integration Entry: shared by
- Shared Constants → Analysis API: shared by
- Analysis API → Media Handlers: prepares media
- Media Handlers → HA Media Storage: reads and writes
- Analysis API → Provider Adapters: requests analysis
- Provider Adapters → Multimodal LLM Backends: calls
- Analysis API → Recognition Memory: reads and updates
- Analysis API → Event Timeline: records results
- Event Timeline → Calendar Projection: projects events
- Timeline API Spec → Event Timeline: defines API
- Timeline UI Text → Event Timeline: localizes UI
- Calendar Projection → Home Assistant Runtime: publishes calendar
- Analysis API → Home Assistant Runtime: updates state for automations
- CI Validation → Integration Entry: tests and validates