How messkan/prompt-cache works
PromptCache is a self-hosted semantic cache exposing an OpenAI-compatible chat-completions API. A request is checked against stored embeddings: strong matches return cached responses, misses are forwarded to a configured model provider, and gray-zone matches may be verified before deciding. Responses can be streamed, and cache warming and runtime configuration are management capabilities. The graph also shows persistence, request middleware, and operational metrics. Core wiring is supported by the README and source index; some handler details are absent from the excerpts, so uncertain links are omitted.
Main entry points: main.go HTTP API · semantic.go Semantic engine · providers.go Model providers · cache.go Response cache
Components 16
API and management
- HTTP API
cmd/api/main.go - Request middleware
internal/middleware/middleware.go - Management endpoints
cmd/api/main.go - Cache warming
cmd/api/main.go
Semantic matching
- Semantic engine
internal/semantic/semantic.go - Model providers
internal/semantic/providers.go - Vector index
internal/ann/ann.go
Cache and persistence
- Response cache
internal/cache/cache.go - Cache namespaces
internal/cachecontext/namespace.go - Persistent store
internal/storage/badger.go
Operations and integrations
- Runtime configuration
internal/config/config.go - Metrics
internal/metrics/metrics.go - Retryable HTTP
internal/http/client.go
Other components
- OpenAI client
- LLM providers
- Chat response
Connections 23
- OpenAI client → HTTP API: sends requests
README.md - HTTP API → Request middleware: uses
cmd/api/main.go - HTTP API → Semantic engine: checks similarity
README.md - Semantic engine → Model providers: embeds and verifies
internal/semantic/semantic.go - Semantic engine → Vector index: searches vectors
internal/semantic/semantic.go - Semantic engine → Persistent store: loads embeddings
internal/semantic/semantic.go - HTTP API → Response cache: reads and writes
cmd/api/main.go - Response cache → Persistent store: persists entries
internal/cache/cache.go - Response cache → Cache namespaces: scopes keys
internal/cache/cache.go - Request middleware → Cache namespaces: sets request scope
internal/middleware/middleware.go - HTTP API → Model providers: forwards misses
README.md - Model providers → LLM providers: calls provider
README.md - Model providers → Retryable HTTP: uses retries
internal/semantic/providers.go - HTTP API → Chat response: returns response
README.md - HTTP API → Management endpoints: routes controls
cmd/api/main.go - Management endpoints → Semantic engine: updates thresholds
cmd/api/main.go - Management endpoints → Cache warming: routes warming
README.md - Cache warming → Semantic engine: embeds prompts
cmd/api/main.go - Cache warming → Response cache: stores responses
cmd/api/main.go - HTTP API → Metrics: serves metrics
cmd/api/main.go - Request middleware → Metrics: records requests
internal/middleware/middleware.go - Model providers → Metrics: records calls
internal/semantic/providers.go - HTTP API → Runtime configuration: loads settings
cmd/api/main.go