# acgodson/blueband-db architecture

> Architecture diagram of the GitHub repository acgodson/blueband-db, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/acgodson/blueband-db
- Repository: https://github.com/acgodson/blueband-db

## Overview

##### Project Type & Purpose
- Blueband DB is an on-chain vector database canister built in Rust for the Internet Computer (ICP).
- It provides semantic document processing, hierarchical vector indexing (K-means), adaptive similarity search, and LRU caching—all exposed via Candid actor methods.
- A small TypeScript “actor” layer (test.ts) demonstrates client usage and embedding service integration.

##### File Structure
- Top‐Level:
- Cargo.toml, blueband_rust.did, dfx.json → canister config & build
- src/lib.rs → entry point for canister API
- src/compute/… → embedding, similarity, cache logic
- src/storage/… → document, collection, vector storage abstractions (memory / stable memory)
- src/export.rs → Wasm export glue
- test.ts, tsconfig.json → TypeScript client/actor test harness
- docs/ → architecture and API reference; .github/workflows for CI/CD and doc publishing.

##### Core Components & Responsibilities
A. Canister API Layer
- Exposed Candid methods: collection management, document ops, vector search
- Entrypoint in src/lib.rs calling into compute/storage modules
B. Compute Module
- embeddings.rs: OpenAI proxy calls for embeddings
- similarity.rs: exact vs approximate search (switching logic)
- cache.rs: in‐memory LRU cache for embeddings
C. Storage Module
- memory.rs: abstraction over in‐memory vs stable memory
- collections.rs, documents.rs, vectors.rs: CRUD for collections, docs, vectors
D. Export
- export.rs serializes Rust API to Wasm/Candid
E. Client / Frontend
- test.ts: TS actor stub illustrating actor calls
F. External Services
- OpenAI Embedding Proxy (HTTP endpoint for embedding generation)

##### Interactions & Data Flows
1. Client (TS/Actor) → calls canister methods over HTTP/Candid
2. API Layer → compute & storage modules
3. On calls requiring embeddings: compute.embeddings → HTTP → OpenAI proxy → returns vectors
4. compute.similarity → queries storage.vectors, may update cache
5. storage module → persists to ICP stable memory or in‐memory data structures
6. Results flow back through API → to client

##### Architectural Patterns & Principles
- Modular Rust code separating compute vs storage concerns
- Cache-aside LRU for embeddings to optimize performance
- Hierarchical K-means indexing for sublinear search
- Adaptive algorithm switching between exact/approximate search
- Candid actor boundary for clear client-server interface

## Diagram (Mermaid)

```mermaid
graph TB
    %% Client Layer
    subgraph "Client / Frontend" 
        direction TB
        TSActor["TS Actor (test.ts)"]:::frontend
        TSConfig["tsconfig.json"]:::frontend
    end

    %% Canister Configuration
    subgraph "Canister Configuration & Build"
        direction TB
        Cargo["Cargo.toml"]:::config
        DFX["dfx.json"]:::config
        PKG["package.json"]:::config
    end

    %% Canister Box
    subgraph "Blueband DB Canister (Wasm)"
        direction TB

        %% API Layer
        subgraph "API Layer"
            direction TB
            APILib["lib.rs"]:::onchain
            APITypes["types.rs"]:::onchain
            APIDID["blueband_rust.did"]:::onchain
        end

        %% Compute Module
        subgraph "Compute Module"
            direction TB
            ComputeMod["mod.rs"]:::onchain
            Embeddings["embeddings.rs"]:::onchain
            Similarity["similarity.rs"]:::onchain
            Cache["cache.rs"]:::onchain
        end

        %% Storage Module
        subgraph "Storage Module"
            direction TB
            StorageMod["mod.rs"]:::onchain
            Memory["memory.rs"]:::onchain
            Collections["collections.rs"]:::onchain
            Documents["documents.rs"]:::onchain
            Vectors["vectors.rs"]:::onchain
            StorageEngine["Stable Memory + In-Memory Structures"]:::storage
        end

        %% Export Layer
        Export["export.rs"]:::onchain

    end

    %% External Services
    EmbProxy["OpenAI Embedding Proxy\n(HTTP)"]:::external

    %% Data Flows
    TSActor -->|add_document / search| APILib
    TSConfig --> APILib
    APILib -->|calls| ComputeMod
    ComputeMod -->|use| Embeddings
    ComputeMod -->|use| Similarity
    ComputeMod -->|use| Cache
    Embeddings -->|POST /embed| EmbProxy
    EmbProxy -->|vectors| Embeddings
    Similarity -->|query| Vectors
    Similarity -->|cache lookup| Cache
    Similarity -->|store/read| StorageEngine
    Memory --> StorageEngine
    Collections --> StorageEngine
    Documents --> StorageEngine
    Vectors --> StorageEngine
    StorageMod --> Memory
    StorageMod --> Collections
    StorageMod --> Documents
    StorageMod --> Vectors
    ComputeMod --> StorageMod
    APILib -->|reply| TSActor
    Cargo --> APILib
    DFX --> APILib
    PKG --> APILib
    Export --> APILib

    %% Click Events
    click TSActor "https://github.com/acgodson/blueband-db/blob/main/test.ts"
    click TSConfig "https://github.com/acgodson/blueband-db/blob/main/tsconfig.json"
    click Cargo "https://github.com/acgodson/blueband-db/blob/main/Cargo.toml"
    click DFX "https://github.com/acgodson/blueband-db/blob/main/dfx.json"
    click PKG "https://github.com/acgodson/blueband-db/blob/main/package.json"
    click APILib "https://github.com/acgodson/blueband-db/blob/main/src/lib.rs"
    click APITypes "https://github.com/acgodson/blueband-db/blob/main/src/types.rs"
    click APIDID "https://github.com/acgodson/blueband-db/blob/main/blueband_rust.did"
    click ComputeMod "https://github.com/acgodson/blueband-db/blob/main/src/compute/mod.rs"
    click Embeddings "https://github.com/acgodson/blueband-db/blob/main/src/compute/embeddings.rs"
    click Similarity "https://github.com/acgodson/blueband-db/blob/main/src/compute/similarity.rs"
    click Cache "https://github.com/acgodson/blueband-db/blob/main/src/compute/cache.rs"
    click StorageMod "https://github.com/acgodson/blueband-db/blob/main/src/storage/mod.rs"
    click Memory "https://github.com/acgodson/blueband-db/blob/main/src/storage/memory.rs"
    click Collections "https://github.com/acgodson/blueband-db/blob/main/src/storage/collections.rs"
    click Documents "https://github.com/acgodson/blueband-db/blob/main/src/storage/documents.rs"
    click Vectors "https://github.com/acgodson/blueband-db/blob/main/src/storage/vectors.rs"
    click Export "https://github.com/acgodson/blueband-db/blob/main/src/export.rs"

    %% Styles
    classDef frontend fill:#dddddd,stroke:#666666,color:#000000
    classDef onchain fill:#cce5ff,stroke:#004085,color:#004085
    classDef storage fill:#d4edda,stroke:#155724,color:#155724
    classDef external fill:#fff3cd,stroke:#856404,color:#856404
    classDef config fill:#e2e3e5,stroke:#6c757d,color:#6c757d
```
