How acgodson/blueband-db works
Project Type & Purpose
• Blueband DB is an on-chain vector database canister built in Rust for the Internet Computer (ICP).
• It provides semantic document processing, hierarchical vector indexing (K-means), adaptive similarity search, and LRU caching—all exposed via Candid actor methods.
• A small TypeScript “actor” layer (test.ts) demonstrates client usage and embedding service integration.
File Structure
• Top‐Level:
• Cargo.toml, blueband_rust.did, dfx.json → canister config & build
• src/lib.rs → entry point for canister API
• src/compute/… → embedding, similarity, cache logic
• src/storage/… → document, collection, vector storage abstractions (memory / stable memory)
• src/export.rs → Wasm export glue
• test.ts, tsconfig.json → TypeScript client/actor test harness
• docs/ → architecture and API reference; .github/workflows for CI/CD and doc publishing.
Core Components & Responsibilities
A. Canister API Layer
• Exposed Candid methods: collection management, document ops, vector search
• Entrypoint in src/lib.rs calling into compute/storage modules
B. Compute Module
• embeddings.rs: OpenAI proxy calls for embeddings
• similarity.rs: exact vs approximate search (switching logic)
• cache.rs: in‐memory LRU cache for embeddings
C. Storage Module
• memory.rs: abstraction over in‐memory vs stable memory
• collections.rs, documents.rs, vectors.rs: CRUD for collections, docs, vectors
D. Export
• export.rs serializes Rust API to Wasm/Candid
E. Client / Frontend
• test.ts: TS actor stub illustrating actor calls
F. External Services
• OpenAI Embedding Proxy (HTTP endpoint for embedding generation)
Interactions & Data Flows
1. Client (TS/Actor) → calls canister methods over HTTP/Candid
2. API Layer → compute & storage modules
3. On calls requiring embeddings: compute.embeddings → HTTP → OpenAI proxy → returns vectors
4. compute.similarity → queries storage.vectors, may update cache
5. storage module → persists to ICP stable memory or in‐memory data structures
6. Results flow back through API → to client
Architectural Patterns & Principles
• Modular Rust code separating compute vs storage concerns
• Cache-aside LRU for embeddings to optimize performance
• Hierarchical K-means indexing for sublinear search
• Adaptive algorithm switching between exact/approximate search
• Candid actor boundary for clear client-server interface