How chroma-core/chroma works
1. Repository shape
- This is a monorepo for Chroma, an AI vector database/search system.
- It contains three major product surfaces: the Python client/server package, the JavaScript client packages, and the Rust backend/runtime that powers local, server, and distributed deployments.
2. Public API boundary
- The top-level user-facing API is the collection-oriented CRUD/search interface shown in the README: create collection, add/update/delete documents, and query/get results.
- Python exposes this through chromadb, with both in-process and client-server modes.
- JavaScript exposes the same concepts through generated API clients and higher-level wrappers in clients/js and clients/new-js.
3. Client-to-server data path
- Client libraries serialize collection operations, metadata filters, embeddings, and query parameters over HTTP/gRPC to a Chroma service.
- The JS packages are built from OpenAPI-generated code plus embedding-function helpers.
- The Python package includes fastapi and grpc client/server adapters, so the same logical API can run locally or remotely.
4. Core service decomposition
- In distributed mode, the backend is split into specialized services rather than one monolith.
- Major Rust services include:
- frontend / frontend-core for request handling and orchestration,
- worker for query execution, compaction, and function execution,
- log-service for write-ahead/log persistence,
- rust-sysdb for system metadata,
- garbage_collector for cleanup,
- s3heap-service / s3heap for heap-backed distributed coordination/storage utilities.
- go/cmd/coordinator and related Go packages provide an additional control-plane implementation centered on metadata, task coordination, and memberlist-based clustering.
5. System metadata and control plane
- Collection, tenant, database, segment, task, and topology metadata are managed separately from vector content.
- The metadata plane exists in multiple backends:
- SQLite for local/single-node deployments,
- Spanner for cloud/distributed deployments,
- Go and Rust sysdb implementations for service-side metadata APIs.
- Schema changes are explicit via migration directories for SQLite, Spanner, and Go metastore components.
6. Write path / ingest flow
- Document ingest enters through the frontend/API layer.
- The frontend validates requests, resolves collection/segment configuration, and routes work to the appropriate executor or worker path.
- Data is appended to log/queue structures, then materialized into segment files and index structures.
- Compaction and garbage collection later reduce log fragments and remove obsolete files.
7. Read path / query flow
- Queries are planned as execution expressions/operators, then dispatched to local or distributed executors.
- Workers fetch the relevant collection/segment state, execute vector search, sparse search, full-text filtering, metadata filtering, ranking, and projection, then merge results.
- The repo contains multiple query strategies: brute force, HNSW, SPANN, sparse/maxscore, and hybrid ranking/group-by operators.
8. Indexing and segment storage
- Segment logic is split between metadata, vector, and distributed concerns.
- Rust crates segment, index, blockstore, and log hold the core storage/indexing machinery.
- index implements HNSW, SPANN, sparse/full-text search, and quantization; blockstore manages blockfile-style persistence; log manages durable append-only logs.
- The Python side also has segment/index implementations for local mode, including SQLite-backed metadata and local HNSW persistence.
9. Embedding-function layer
- Embedding generation is intentionally pluggable and sits at the edge of the ingest/query pipeline.
- There are many built-in embedding function adapters in both Python and JS for providers such as OpenAI, Ollama, Cohere, Jina, VoyageAI, Google, Hugging Face, Bedrock, and Chroma-specific sparse models.
- A default ONNX model is preloaded in CI and deployment tooling, indicating embeddings can be local and offline as well as remote-service-backed.
10. Distributed coordination
- Distributed deployments rely on service discovery and membership/assignment logic.
- The repo includes memberlist-based components, rendezvous hashing, task orchestration, and role-specific Kubernetes manifests.
- Work queues, compaction schedulers, and function consumers coordinate asynchronous jobs across services rather than in-process threads only.
11. External infrastructure and persistence
- Persistent state is spread across SQLite, S3/object storage, and Spanner depending on deployment mode.
- Cloud/container delivery is supported with Docker, Docker Compose, Helm charts under k8s/distributed-chroma, and cloud-specific Terraform/CloudFormation examples under deployments and examples/deployments.
- Test infrastructure includes Kubernetes resources for Postgres, MinIO, Prometheus, Grafana, Jaeger, and OTel collector, showing observability and storage dependencies are first-class.
12. Observability, auth, and policy
- The repo has explicit telemetry and tracing packages for OpenTelemetry and product events.
- AuthN/AuthZ is implemented in the Python layer with basic auth, token auth, and simple RBAC modules, and mirrored in client tests.
- Quota and rate-limit modules indicate request governance is enforced as a separate concern from query execution.
13. Developer and release system
- CI/CD is substantial: GitHub Actions build/test/release pipelines exist for Rust, Python, Go, JS, containers, and Helm charts.
- The release surface spans PyPI, npm packages, Rust crates/binaries, Docker images, and service images.
- Documentation and examples are treated as part of the product, with generated API references and runnable sample apps that mirror real deployment paths.
14. What to draw in an architecture diagram
- Model this repo as a multi-language, multi-service vector database with a clear split between:
- client SDKs,
- API/frontend layer,
- execution workers,
- metadata/control plane,
- durable log and segment/index storage,
- external embedding providers and object stores.
- The most important directional flows are: client request → frontend → metadata resolution → log/segment/index update or query execution → storage/object store → response, with compaction/GC as asynchronous side channels.
Main entry points: worker Worker · frontend Frontend · rust-sysdb Rust SysDB · distributed-chroma Deployments
Components 20
Client SDKs
User-facing Python and JavaScript packages, plus generated API bindings and embedding helpers.
- Python API · chromadb package exposing collection CRUD, search, auth, quotas, and local or client-server modes.
chromadb - JS Client · TypeScript/JavaScript client packages and generated API wrappers for collection operations.
clients/js/packages/chromadb - JS Embeddings · Pluggable embedding-function packages for remote and local providers.
clients/new-js/packages/ai-embeddings
Rust Backend
Core service/runtime, execution, storage, and indexing machinery for local and distributed Chroma deployments.
- Frontend · Rust request handling, validation, auth, quota enforcement, and orchestration entrypoint.
rust/frontend - Frontend Core · Shared frontend logic for collection operations, middleware, retries, and attached functions.
rust/frontend-core - Worker · Distributed query, ingest, compaction, and function execution service.
rust/worker - Log Service · Log persistence, scrubbing, sealing, and regional coordination for append-only history.
rust/log-service - Segment · Local and distributed segment management, metadata, and version tracking.
rust/segment - Index · HNSW, SPANN, sparse, full-text, and quantization implementations.
rust/index - Blockstore · Blockfile-style persistence and in-memory storage primitives for segment data.
rust/blockstore - WAL3 · Low-level durable log, snapshot, copy, GC, and S3-backed replication primitives.
rust/wal3
Control Plane
Go and Rust metadata, coordination, and membership services for distributed deployments.
- Rust SysDB · System metadata service for tenants, databases, collections, segments, and tasks.
rust/rust-sysdb - Go Coordinator · Go metadata and task-coordination implementation with memberlist-based clustering.
go/cmd/coordinator - Memberlist · Cluster membership, leader election, and rendezvous hashing support.
rust/memberlist - S3 Heap · Heap-backed distributed coordination and storage utility layer.
rust/s3heap
Infra
Deployment, persistence, and observability surfaces that surround the core database.
- Storage · Local disk and S3/object-storage adapters used by logs, segments, and distributed services.
rust/storage - SQLite · Single-node metadata and queue persistence with explicit migrations.
rust/sqlite - Spanner Migrations · Schema management for distributed/cloud metadata and log backends.
rust/spanner-migrations - Telemetry · OpenTelemetry tracing and product-event emission across Python and Rust surfaces.
chromadb/telemetry - Deployments · Docker, Helm, Kubernetes, Terraform, and compose manifests for local and distributed runs.
k8s/distributed-chroma
Connections 23
- Python API → Frontend: HTTP/gRPC · Python clients and server adapters serialize collection operations to the Rust service layer.
- JS Client → Frontend: API calls · JS clients use generated OpenAPI wrappers to reach the Chroma service.
- JS Embeddings → Python API: provider parity · Embedding helpers mirror the same pluggable model ecosystem across SDKs.
- Frontend → Frontend Core: orchestrates · Frontend delegates shared request logic and policy enforcement to the core layer.
- Frontend Core → Rust SysDB: resolve metadata · Collection, segment, and task metadata are looked up through the system database.
- Frontend Core → Worker: dispatch work · Requests are routed to local or distributed executors for ingest and query paths.
- Frontend → Log Service: write path · Ingest appends to durable logs and queues before materialization.
- Worker → Segment: read/write segments · Workers load segment state, apply logs, and execute search/materialization.
- Worker → Index: search · Vector, sparse, full-text, and hybrid retrieval are executed through index implementations.
- Worker → Blockstore: persist segments · Segment files and index artifacts are written through blockfile-backed storage.
- Log Service → WAL3: durable log · Log service is built on WAL3 for snapshots, replication, and garbage collection.
- WAL3 → Storage: S3/local · Logs and snapshots can be stored on local disk or object storage.
- Segment → SQLite: local metadata · Single-node segment metadata and persistence use SQLite-backed state.
- Rust SysDB → SQLite: single-node · Local metadata backend is SQLite for non-distributed deployments.
- Rust SysDB → Spanner Migrations: cloud schema · Cloud/distributed metadata changes are managed via Spanner migrations.
- Go Coordinator → Memberlist: cluster membership · Go coordinator uses memberlist and rendezvous hashing for assignment and leadership.
- Go Coordinator → Rust SysDB: metadata API · The Go control plane manages the same tenant/database/collection/task concepts.
- S3 Heap → Storage: object-backed · Heap utilities coordinate distributed state over S3/object storage.
- Frontend → Telemetry: trace/events · API requests emit traces and product telemetry.
- Worker → Telemetry: trace/events · Execution services emit observability data for query and compaction flows.
- Deployments → Frontend: deploys · Kubernetes, Docker, and compose manifests define frontend service deployment.
- Deployments → Worker: deploys · Distributed runtime manifests include worker, queue, compaction, and function services.
- Deployments → Rust SysDB: deploys · System metadata service and migrations are part of deployment topology.