# messkan/prompt-cache architecture

> Architecture diagram of the GitHub repository messkan/prompt-cache, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/messkan/prompt-cache
- Repository: https://github.com/messkan/prompt-cache

## Overview

PromptCache is a self-hosted semantic cache exposing an OpenAI-compatible chat-completions API. A request is checked against stored embeddings: strong matches return cached responses, misses are forwarded to a configured model provider, and gray-zone matches may be verified before deciding. Responses can be streamed, and cache warming and runtime configuration are management capabilities. The graph also shows persistence, request middleware, and operational metrics. Core wiring is supported by the README and source index; some handler details are absent from the excerpts, so uncertain links are omitted.

Main entry points: [`cmd/api/main.go`](https://github.com/messkan/prompt-cache/blob/main/cmd/api/main.go) (HTTP API), [`internal/semantic/semantic.go`](https://github.com/messkan/prompt-cache/blob/main/internal/semantic/semantic.go) (Semantic engine), [`internal/semantic/providers.go`](https://github.com/messkan/prompt-cache/blob/main/internal/semantic/providers.go) (Model providers), [`internal/cache/cache.go`](https://github.com/messkan/prompt-cache/blob/main/internal/cache/cache.go) (Response cache)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["API and management"]
  node_api["HTTP API<br/>[main.go]"]
  node_middleware["Request middleware<br/>[middleware.go]"]
  node_management["Management endpoints<br/>[main.go]"]
  node_warming["Cache warming<br/>[main.go]"]
end

subgraph group_matching["Semantic matching"]
  node_engine["Semantic engine<br/>[semantic.go]"]
  node_providers["Model providers<br/>[providers.go]"]
  node_ann["Vector index<br/>[ann.go]"]
end

subgraph group_persistence["Cache and persistence"]
  node_cache["Response cache<br/>[cache.go]"]
  node_namespace["Cache namespaces<br/>[namespace.go]"]
  node_badger[("Persistent store<br/>[badger.go]")]
end

subgraph group_operations["Operations and integrations"]
  node_config["Runtime configuration<br/>[config.go]"]
  node_metrics["Metrics<br/>[metrics.go]"]
  node_httpclient["Retryable HTTP<br/>[client.go]"]
end

node_client(("OpenAI client"))
node_llm{{"LLM providers"}}
node_result["Chat response"]

node_client -->|"sends requests"| node_api
node_api -->|"uses"| node_middleware
node_api -->|"checks similarity"| node_engine
node_engine -->|"embeds and verifies"| node_providers
node_engine -->|"searches vectors"| node_ann
node_engine -->|"loads embeddings"| node_badger
node_api -->|"reads and writes"| node_cache
node_cache -->|"persists entries"| node_badger
node_cache -->|"scopes keys"| node_namespace
node_middleware -->|"sets request scope"| node_namespace
node_api -->|"forwards misses"| node_providers
node_providers -->|"calls provider"| node_llm
node_providers -->|"uses retries"| node_httpclient
node_api -->|"returns response"| node_result
node_api -->|"routes controls"| node_management
node_management -->|"updates thresholds"| node_engine
node_management -->|"routes warming"| node_warming
node_warming -->|"embeds prompts"| node_engine
node_warming -->|"stores responses"| node_cache
node_api -->|"serves metrics"| node_metrics
node_middleware -->|"records requests"| node_metrics
node_providers -->|"records calls"| node_metrics
node_api -->|"loads settings"| node_config

click node_api "https://github.com/messkan/prompt-cache/blob/main/cmd/api/main.go"
click node_middleware "https://github.com/messkan/prompt-cache/blob/main/internal/middleware/middleware.go"
click node_management "https://github.com/messkan/prompt-cache/blob/main/cmd/api/main.go"
click node_warming "https://github.com/messkan/prompt-cache/blob/main/cmd/api/main.go"
click node_engine "https://github.com/messkan/prompt-cache/blob/main/internal/semantic/semantic.go"
click node_providers "https://github.com/messkan/prompt-cache/blob/main/internal/semantic/providers.go"
click node_ann "https://github.com/messkan/prompt-cache/blob/main/internal/ann/ann.go"
click node_cache "https://github.com/messkan/prompt-cache/blob/main/internal/cache/cache.go"
click node_namespace "https://github.com/messkan/prompt-cache/blob/main/internal/cachecontext/namespace.go"
click node_badger "https://github.com/messkan/prompt-cache/blob/main/internal/storage/badger.go"
click node_config "https://github.com/messkan/prompt-cache/blob/main/internal/config/config.go"
click node_metrics "https://github.com/messkan/prompt-cache/blob/main/internal/metrics/metrics.go"
click node_httpclient "https://github.com/messkan/prompt-cache/blob/main/internal/http/client.go"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_api,node_middleware,node_management,node_warming,node_client toneBlue
class node_engine,node_providers,node_ann toneAmber
class node_cache,node_namespace,node_badger toneMint
class node_config,node_metrics,node_httpclient toneRose
class node_llm,node_result toneIndigo
```

## Components

### API and management

- [HTTP API](https://github.com/messkan/prompt-cache/blob/main/cmd/api/main.go): `cmd/api/main.go`
- [Request middleware](https://github.com/messkan/prompt-cache/blob/main/internal/middleware/middleware.go): `internal/middleware/middleware.go`
- [Management endpoints](https://github.com/messkan/prompt-cache/blob/main/cmd/api/main.go): `cmd/api/main.go`
- [Cache warming](https://github.com/messkan/prompt-cache/blob/main/cmd/api/main.go): `cmd/api/main.go`

### Semantic matching

- [Semantic engine](https://github.com/messkan/prompt-cache/blob/main/internal/semantic/semantic.go): `internal/semantic/semantic.go`
- [Model providers](https://github.com/messkan/prompt-cache/blob/main/internal/semantic/providers.go): `internal/semantic/providers.go`
- [Vector index](https://github.com/messkan/prompt-cache/blob/main/internal/ann/ann.go): `internal/ann/ann.go`

### Cache and persistence

- [Response cache](https://github.com/messkan/prompt-cache/blob/main/internal/cache/cache.go): `internal/cache/cache.go`
- [Cache namespaces](https://github.com/messkan/prompt-cache/blob/main/internal/cachecontext/namespace.go): `internal/cachecontext/namespace.go`
- [Persistent store](https://github.com/messkan/prompt-cache/blob/main/internal/storage/badger.go): `internal/storage/badger.go`

### Operations and integrations

- [Runtime configuration](https://github.com/messkan/prompt-cache/blob/main/internal/config/config.go): `internal/config/config.go`
- [Metrics](https://github.com/messkan/prompt-cache/blob/main/internal/metrics/metrics.go): `internal/metrics/metrics.go`
- [Retryable HTTP](https://github.com/messkan/prompt-cache/blob/main/internal/http/client.go): `internal/http/client.go`

### Other components

- **OpenAI client**
- **LLM providers**
- **Chat response**

## Connections

- OpenAI client → HTTP API: sends requests (evidence: `README.md`)
- HTTP API → Request middleware: uses (evidence: `cmd/api/main.go`)
- HTTP API → Semantic engine: checks similarity (evidence: `README.md`)
- Semantic engine → Model providers: embeds and verifies (evidence: `internal/semantic/semantic.go`)
- Semantic engine → Vector index: searches vectors (evidence: `internal/semantic/semantic.go`)
- Semantic engine → Persistent store: loads embeddings (evidence: `internal/semantic/semantic.go`)
- HTTP API → Response cache: reads and writes (evidence: `cmd/api/main.go`)
- Response cache → Persistent store: persists entries (evidence: `internal/cache/cache.go`)
- Response cache → Cache namespaces: scopes keys (evidence: `internal/cache/cache.go`)
- Request middleware → Cache namespaces: sets request scope (evidence: `internal/middleware/middleware.go`)
- HTTP API → Model providers: forwards misses (evidence: `README.md`)
- Model providers → LLM providers: calls provider (evidence: `README.md`)
- Model providers → Retryable HTTP: uses retries (evidence: `internal/semantic/providers.go`)
- HTTP API → Chat response: returns response (evidence: `README.md`)
- HTTP API → Management endpoints: routes controls (evidence: `cmd/api/main.go`)
- Management endpoints → Semantic engine: updates thresholds (evidence: `cmd/api/main.go`)
- Management endpoints → Cache warming: routes warming (evidence: `README.md`)
- Cache warming → Semantic engine: embeds prompts (evidence: `cmd/api/main.go`)
- Cache warming → Response cache: stores responses (evidence: `cmd/api/main.go`)
- HTTP API → Metrics: serves metrics (evidence: `cmd/api/main.go`)
- Request middleware → Metrics: records requests (evidence: `internal/middleware/middleware.go`)
- Model providers → Metrics: records calls (evidence: `internal/semantic/providers.go`)
- HTTP API → Runtime configuration: loads settings (evidence: `cmd/api/main.go`)
