How axm1647/fastapi-semcache works
fastapi-semcache is a semantic response-caching library for FastAPI/ASGI APIs, with an optional reverse-proxy mode. A request enters the middleware, which extracts query and scope information and asks SemanticCache for a match; on a hit it replays the stored response, and on a miss it invokes the downstream app and may store the response. SemanticCache coordinates embedding, pgvector similarity search, and optional Redis response blobs. The README documents multiple embedding providers and proxy support; sampled source confirms the main middleware, cache, and proxy wiring, while some provider and storage details are represented at their tree-supported directory paths.
Main entry points: middleware.py ASGI middleware · cache.py SemanticCache · proxy.py Proxy app · embedders Embedder backends
Components 18
Request handling
- ASGI middleware
semanticcache/middleware/adapters/fastapi/middleware.py - Request extraction
semanticcache/middleware/core/extractors.py - Request coordination
semanticcache/middleware/core/coordination.py - Response replay
semanticcache/middleware/core/replay.py - Proxy app
semanticcache/proxy.py
Cache core
- SemanticCache
semanticcache/cache.py - Cache settings
semanticcache/config.py
Embedding
- Embedder interface
semanticcache/embedders/_base.py - Embedder backends
semanticcache/embedders
Persistence
- Vector store
semanticcache/stores/vector/pgvector.py - Postgres pgvector
- Redis response store
semanticcache/stores/response/redis_store.py - Redis
- Storage identifiers
semanticcache/stores/vector/storage_ids.py
Other components
- API client
- Downstream app
- Upstream API
- Embedding providers
Connections 20
- API client → ASGI middleware: sends request
- ASGI middleware → Request extraction: extracts fields
- ASGI middleware → Request coordination: coordinates request
- ASGI middleware → SemanticCache: looks up
- SemanticCache → Embedder interface: embeds query
- Embedder backends → Embedder interface: implements
- Embedder backends → Embedding providers: requests embeddings
- SemanticCache → Vector store: searches and writes
- Vector store → Postgres pgvector: queries vectors
- SemanticCache → Redis response store: reads and writes blobs
- Redis response store → Redis: stores JSON
- SemanticCache → Cache settings: uses configuration
- SemanticCache → Storage identifiers: derives identifiers
- ASGI middleware → Response replay: replays hit
- Response replay → API client: sends response
- ASGI middleware → Downstream app: dispatches miss
- Downstream app → ASGI middleware: returns response
- ASGI middleware → Proxy app: dispatches proxy request
- Proxy app → Upstream API: forwards HTTP
- Upstream API → Proxy app: returns HTTP response