# lancedb/vectordb-recipes architecture

> Architecture diagram of the GitHub repository lancedb/vectordb-recipes, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lancedb/vectordb-recipes
- Repository: https://github.com/lancedb/vectordb-recipes

## Overview

##### Main Components
- Repository Structure Node – At the center, label it “VectorDB-recipes.”
- Divide the repository into its major sections. Based on the README organization, these are:
– Build from Scratch
Examples:
- “Build RAG from Scratch”
- “Local RAG from Scratch with Llama3”
- “Multi-Head RAG from Scratch”
– Multimodal
Examples:
- “Multimodal CLIP: DiffusionDB”
- “Multimodal CLIP: Youtube videos”
- “Cambrian-1: Vision centric exploration,” etc.
– RAG (Retrieval Augmented Generation)
Examples:
- “RAG using Deepseek R1 vs OpenAI o1”
- “RAG On PDF”
- “Cognee RAG”
- “RAG with Contextual Retrieval”
- “Improve RAG with HyDE,” etc.
– Vector Search
Examples:
- “Inbuilt Hybrid Search”
- “Hybrid search BM25 & LanceDB”
- “NER powered Semantic Search,” etc.
– Chatbot
Examples:
- “Databricks DBRX Website Bot”
- “CLI-based SDK Manual Chatbot”
- “Chatbot with any website (Crawl4AI)”
- “Context-Aware Chatbot using Llama 2 & LanceDB,” etc.
– Evaluation
Examples:
- “Monitoring and Tracing RAG using HoneyHive”
- “Evaluating RAG with RAGAs”
– AI Agents
Examples:
- “Trip Planner Swarm style Agent”
- “Multi Source Agent”
- “AI email assistant with Composio”
- “SuperAgent Autogen,” etc.
– Recommender Systems
Examples:
- “Movie Recommender”
- “Product Recommender”
- “Music Recommender”
– Concepts
Examples:
- “A Primer on Text Chunking and its Types”
- “Langchain LlamaIndex Chunking”
- “NER-dataset with Instructor”
- “Comparing Cohere Rerankers with LanceDB”
##### Relationships and Hierarchy
- Within the “Examples/Tutorials” section, further subdivide into the above categories (for instance, a branch for “Build from Scratch” then a branch for “RAG,” “Multimodal,” etc.).
- Under the “Concepts” section, clearly show these examples as supportive material that helps understand core techniques across all areas.
- The “Applications” (seen mostly in the “applications” directory in the file tree) can be diagrammed as separate nodes that integrate elements from those core examples—emphasizing that many are end-to-end ready apps with UI, backend integrations, and often Node-based as well.

##### Architectural Patterns and Technologies
- That many examples use common building blocks, such as:
– LanceDB as the vector search backend (serverless, integrated with Python and Typescript SDK)
– LLMs and language chain processing libraries (e.g., LangChain, LangGraph)
– Frontend components in Node-based projects (React, Next.js, Vite, Tailwind)
– Notebook-based prototypes (Jupyter/Colab notebooks) showcasing different experiment stages.

##### Additional Details
- Do not forget to include infrastructure details from files in directories such as .github (CI/CD workflows like “black.yml”, “examples-test.yml”) which imply automated testing and deployment pipelines. A brief node for “CI/CD” can be added as a supporting process.

- Use a clear, hierarchical layout that starts from the repository as a whole and then branches into the above sections.
– Another for production-ready applications,
– And another for conceptual material.
- Keep labeling detailed to capture the names of the examples (especially in RAG, AI Agents, Multimodal, Build from Scratch, and Concepts clusters).

## Diagram (Mermaid)

```mermaid
flowchart TD
    R["VectorDB-recipes"]:::repository
    R --> ET["Examples/Tutorials"]
    R --> AP["Applications/Projects"]
    R --> INF["CI/CD & Infra"]

    subgraph "Examples/Tutorials"
        subgraph "Build from Scratch"
            BS1["Local RAG from Scratch"]:::build
            BS2["Multi-Head RAG from Scratch"]:::build
            BS3["RAG from Scratch"]:::build
        end

        subgraph "Multimodal"
            MM1["Multimodal CLIP: DiffusionDB"]:::multimodal
            MM2["Multimodal Video Search (Example)"]:::multimodal
            MM3["Multimodal (Jina CLIPv2)"]:::multimodal
        end

        subgraph "RAG"
            RAG1["RAG using Deepseek R1 vs OpenAI o1"]:::rag
            RAG2["RAG On PDF"]:::rag
            RAG3["Cognee RAG"]:::rag
            RAG4["Improve RAG with HyDE"]:::rag
            RAG5["Contextual RAG"]:::rag
            RAG6["Advanced RAG Context Enrichment"]:::rag
        end

        subgraph "Vector Search"
            VS1["Inbuilt Hybrid Search"]:::vector
            VS2["Hybrid search BM25 & LanceDB"]:::vector
            VS3["NER powered Semantic Search"]:::vector
        end

        subgraph "Chatbot"
            CB1["CLI-based SDK Manual Chatbot"]:::chatbot
            CB2["Databricks DBRX Website Bot"]:::chatbot
            CB3["Context-Aware Chatbot using Llama 2 & LanceDB"]:::chatbot
        end

        subgraph "Evaluation"
            EV1["Monitoring and Tracing RAG using HoneyHive"]:::evaluation
        end

        subgraph "AI Agents"
            AA1["Trip Planner Swarm style Agent"]:::agents
            AA2["Multi Source Agent"]:::agents
            AA3["SuperAgent Autogen"]:::agents
            AA4["Agentic RAG"]:::agents
        end

        subgraph "Recommender Systems"
            RS1["Movie Recommender"]:::recommender
            RS2["Product Recommender"]:::recommender
        end

        subgraph "Concepts"
            CT1["A Primer on Text Chunking"]:::concepts
            CT2["Langchain LlamaIndex Chunking"]:::concepts
            CT3["NER-dataset with Instructor"]:::concepts
            CT4["Comparing Cohere Rerankers with LanceDB"]:::concepts
        end
    end

    subgraph "Applications/Projects"
        MV["Multimodal Video Search (App)"]:::applications
        CW["Chatbot w/ any website (App)"]:::applications
        ER["Evaluating RAG with RAGAs (App)"]:::applications
        MR["Music Recommender (App)"]:::applications
    end

    subgraph "CI/CD & Infra"
        CI["CI/CD Workflows"]:::cicd
        DE["Dependencies & Env"]:::cicd
    end

    %% Connections between nodes to illustrate flow/dependencies
    BS1 -->|"initiate_RAG"| RAG1
    BS3 --> RAG1
    VS3 --> CB3
    RAG2 --> EV1
    EV1 --> ER
    AA1 --> AA4
    AA2 --> AA3
    CT1 --> BS1
    CT2 --> BS1

    %% Click Events for Repository
    click R "https://github.com/lancedb/vectordb-recipes/blob/main/README.md"

    %% Build from Scratch Click Events
    click BS1 "https://github.com/lancedb/vectordb-recipes/tree/main/tutorials/Local-RAG-from-Scratch"
    click BS2 "https://github.com/lancedb/vectordb-recipes/tree/main/tutorials/Multi-Head-RAG-from-Scratch"
    click BS3 "https://github.com/lancedb/vectordb-recipes/tree/main/tutorials/RAG-from-Scratch"

    %% Multimodal Click Events
    click MM1 "https://github.com/lancedb/vectordb-recipes/tree/main/examples/multimodal_clip_diffusiondb"
    click MM2 "https://github.com/lancedb/vectordb-recipes/tree/main/examples/multimodal_video_search"
    click MM3 "https://github.com/lancedb/vectordb-recipes/tree/main/examples/multimodal_jina_clipv2"
    
    %% RAG Click Events
    click RAG1 "https://github.com/lancedb/vectordb-recipes/tree/main/examples/Deepseek_R1_VS_GPT_4o"
    click RAG2 "https://github.com/lancedb/vectordb-recipes/tree/main/examples/RAG-On-PDF"
    click RAG3 "https://github.com/lancedb/vectordb-recipes/tree/main/examples/cognee-RAG"
    click RAG4 "https://github.com/lancedb/vectordb-recipes/tree/main/examples/Advance-RAG-with-HyDE"
    click RAG5 "https://github.com/lancedb/vectordb-recipes/tree/main/examples/Contextual-RAG"
    click RAG6 "https://github.com/lancedb/vectordb-recipes/tree/main/examples/Advanced_RAG_Context_Enrichment_Window"
    
    %% Vector Search Click Events
    click VS1 "https://github.com/lancedb/vectordb-recipes/tree/main/examples/Inbuilt-Hybrid-Search"
    click VS2 "https://github.com/lancedb/vectordb-recipes/tree/main/examples/Hybrid_search_bm25_lancedb"
    click VS3 "https://github.com/lancedb/vectordb-recipes/tree/main/tutorials/NER-powered-Semantic-Search"
    
    %% Chatbot Click Events
    click CB1 "https://github.com/lancedb/vectordb-recipes/tree/main/examples/CLI-SDK-Manual-Chatbot-Locally"
    click CB2 "https://github.com/lancedb/vectordb-recipes/tree/main/examples/databricks_DBRX_website_bot"
    click CB3 "https://github.com/lancedb/vectordb-recipes/tree/main/tutorials/chatbot_using_Llama2_&_lanceDB"
    
    %% Evaluation Click Events
    click EV1 "https://github.com/lancedb/vectordb-recipes/tree/main/examples/HoneyHive_x_LanceDB"
    
    %% AI Agents Click Events
    click AA1 "https://github.com/lancedb/vectordb-recipes/tree/main/examples/Trip_planner_swarm_style_agent"
    click AA2 "https://github.com/lancedb/vectordb-recipes/tree/main/examples/Multi-source-Agent"
    click AA3 "https://github.com/lancedb/vectordb-recipes/tree/main/examples/SuperAgent_Autogen"
    click AA4 "https://github.com/lancedb/vectordb-recipes/tree/main/tutorials/Agentic_RAG"
    
    %% Recommender Systems Click Events
    click RS1 "https://github.com/lancedb/vectordb-recipes/tree/main/examples/movie-recommender"
    click RS2 "https://github.com/lancedb/vectordb-recipes/tree/main/examples/product-recommender"
    
    %% Concepts Click Events
    click CT1 "https://github.com/lancedb/vectordb-recipes/tree/main/tutorials/different-types-text-chunking-in-RAG"
    click CT2 "https://github.com/lancedb/vectordb-recipes/tree/main/tutorials/Langchain-LlamaIndex-Chunking"
    click CT3 "https://github.com/lancedb/vectordb-recipes/tree/main/tutorials/NER-dataset-with-Instructor"
    click CT4 "https://github.com/lancedb/vectordb-recipes/tree/main/tutorials/cohere-reranker"
    
    %% Applications/Projects Click Events
    click MV "https://github.com/lancedb/vectordb-recipes/tree/main/applications/multimodal-search"
    click CW "https://github.com/lancedb/vectordb-recipes/tree/main/applications/Chatbot_with_Parler_TTS"
    click ER "https://github.com/lancedb/vectordb-recipes/tree/main/applications/evaluate_RAG"
    click MR "https://github.com/lancedb/vectordb-recipes/tree/main/applications/Music_Recommendation"
    
    %% CI/CD & Infra Click Events
    click CI "https://github.com/lancedb/vectordb-recipes/tree/main/.github/workflows"
    click DE "https://github.com/lancedb/vectordb-recipes/blob/main/package.json"

    %% Styles
    class R repository
    class BS1,BS2,BS3 build
    class MM1,MM2,MM3 multimodal
    class RAG1,RAG2,RAG3,RAG4,RAG5,RAG6 rag
    class VS1,VS2,VS3 vector
    class CB1,CB2,CB3 chatbot
    class EV1 evaluation
    class AA1,AA2,AA3,AA4 agents
    class RS1,RS2 recommender
    class CT1,CT2,CT3,CT4 concepts
    class MV,CW,ER,MR applications
    class CI,DE cicd

classDef repository fill:#D0D6FF,stroke:#000,stroke-width:1px;
classDef build fill:#AEC6CF,stroke:#000,stroke-width:1px;
classDef multimodal fill:#B6E2A1,stroke:#000,stroke-width:1px;
classDef rag fill:#FFB3B3,stroke:#000,stroke-width:1px;
classDef vector fill:#ADD8E6,stroke:#000,stroke-width:1px;
classDef chatbot fill:#FFC107,stroke:#000,stroke-width:1px;
classDef evaluation fill:#F3E5AB,stroke:#000,stroke-width:1px;
classDef agents fill:#D0E1F9,stroke:#000,stroke-width:1px;
classDef recommender fill:#F7C6C7,stroke:#000,stroke-width:1px;
classDef concepts fill:#D1F2EB,stroke:#000,stroke-width:1px;
classDef applications fill:#E2D6FF,stroke:#000,stroke-width:1px;
classDef cicd fill:#F5CBA7,stroke:#000,stroke-width:1px;
```
