# mastra-ai/rag-workshop-code architecture

> Architecture diagram of the GitHub repository mastra-ai/rag-workshop-code, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mastra-ai/rag-workshop-code
- Repository: https://github.com/mastra-ai/rag-workshop-code

## Overview

##### Main Components
- Document Sources: The project contains two kinds of source documents – one in the regular documents folder (src/documents) and another in the bonus section (src/bonus/documents). These files include markdown, JSON, and even TypeScript documents to be upserted into a vector store.
- Chunking Modules: Several examples demonstrate different chunking methods:
- Character Chunking, Recursive Chunking, JSON Chunking, and Markdown Chunking (found under src/examples/01-04).
- Embedding & Vector Store Operations: Examples showing embedding (src/examples/05-embedding.ts), vector upserting (src/examples/06-vector-upserting.ts), similarity search (src/examples/07-vector-search.ts), and reranking (src/examples/08-vector-reranking.ts).
- Agents: Modules related to the search functionalities are implemented within the agents folder (src/mastra/agents) along with corresponding examples (src/examples/09-basic-search-usage.ts and src/examples/10-query-vector-usage.ts). They are responsible for both basic keyword and semantic (vector-based) searches.
- Bonus Examples: An additional section (src/bonus/) provides advanced, code-aware search functionalities including examples targeting code usage (like 01-find-code-usage.ts) and documents (authentication, logging, etc).

##### Relationships/Interactions
- Flow from Document Ingestion → Chunking → Embedding → Upserting into the Vector Store.
- Retrieval path: When a user query is submitted, the relevant agent (from the agents module) leverages vector search operations (by querying PgVector via PostgreSQL) and may also perform re-ranking to produce the end result.
- The bonus examples demonstrate similar flows but are tailored for code searches, indicating specialized handling and processing logic.

##### Important Architectural Patterns and Design Principles
- Separation of Concerns: The directory structure is segmented (documents, examples, mastra, bonus) where each module is clearly responsible for a specific function: ingesting documents, processing/chunking, vector operations, and search functionalities.
- Modular Design: Tools and agents (found in src/mastra) are separate modules that can be recombined to form different instructions.
- Data Flow Pipelines: The overall process from document ingestion to vector search represents a clear pipeline pattern—each step passes data to the next.

- From “Documents” to “Chunking Strategies”.
- From “Chunking” to “Embedding Service”.
- From “Embedding” to “Vector Store (PgVector/Database)”.
- From “Vector Store” to “Agents” (basic and semantic).
- Optionally, highlight a secondary flow for the bonus examples (code search path).

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Document Sources"
        DocStd["Standard Documents"]:::documents
        DocBonus["Bonus Code Documents"]:::documents
    end

    subgraph "Chunking Strategies"
        ChunkChar["Character Chunking"]:::processing
        ChunkRec["Recursive Chunking"]:::processing
        ChunkJSON["JSON Chunking"]:::processing
        ChunkMarkdown["Markdown Chunking"]:::processing
    end

    Embedding["Embedding Service"]:::processing

    subgraph "Vector Store (PgVector Database)"
        VecUpsert["Vector Upserting"]:::vector
        VecSearch["Similarity Search"]:::vector
        VecRerank["Reranking"]:::vector
    end

    subgraph "Agents"
        BasicAgent["Basic Search Agent"]:::agents
        QueryAgent["Query-Vector Agent"]:::agents
        BasicEx["Basic Search Example"]:::agents
        QueryEx["Query-Vector Example"]:::agents
    end

    subgraph "Bonus Code-Aware Search"
        FindUsage["Find Code Usage"]:::bonus
        BonusAgent["Bonus Agent"]:::bonus
        BonusTool["Bonus Tool"]:::bonus
    end

    OpenAI["OpenAI API"]:::external
    Cohere["Cohere API"]:::external
    PostgreSQL["PostgreSQL (PgVector)"]:::external

    %% Document Ingestion Flow
    DocStd -->|"ingest"| ChunkChar
    DocStd -->|"ingest"| ChunkRec
    DocStd -->|"ingest"| ChunkJSON
    DocStd -->|"ingest"| ChunkMarkdown

    ChunkChar --> Embedding
    ChunkRec --> Embedding
    ChunkJSON --> Embedding
    ChunkMarkdown --> Embedding

    Embedding -->|"generate"| VecUpsert

    %% External Integration from Embedding Service
    Embedding -->|"calls"| OpenAI
    Embedding -->|"calls"| Cohere

    %% Vector Store Operations
    VecUpsert --> PostgreSQL
    VecUpsert --> VecSearch
    VecSearch --> PostgreSQL
    VecSearch --> VecRerank
    VecRerank --> PostgreSQL
    VecRerank -->|"triggers"| BasicAgent

    %% Agents Connections
    BasicEx --> BasicAgent
    QueryEx --> QueryAgent

    %% Bonus Code-Aware Flow
    DocBonus -->|"ingestBonus"| FindUsage
    FindUsage --> BonusAgent
    BonusAgent --> BonusTool
    BonusTool -->|"integrates"| BasicAgent

    %% Classes Definitions for Colors
    classDef documents fill:#D6E9F8,stroke:#1F78B4,stroke-width:2px;
    classDef processing fill:#DFFFD6,stroke:#33A02C,stroke-width:2px;
    classDef vector fill:#FADBD8,stroke:#E74C3C,stroke-width:2px;
    classDef agents fill:#E8DAEF,stroke:#8E44AD,stroke-width:2px;
    classDef bonus fill:#F0E68C,stroke:#B8860B,stroke-width:2px;
    classDef external fill:#FFF2CC,stroke:#FF7F0E,stroke-width:2px;

    %% Click Events
    click DocStd "https://github.com/mastra-ai/rag-workshop-code/tree/main/src/documents"
    click DocBonus "https://github.com/mastra-ai/rag-workshop-code/tree/main/src/bonus/documents"
    click ChunkChar "https://github.com/mastra-ai/rag-workshop-code/blob/main/src/examples/01-character-chunking.ts"
    click ChunkRec "https://github.com/mastra-ai/rag-workshop-code/blob/main/src/examples/02-recursive-chunking.ts"
    click ChunkJSON "https://github.com/mastra-ai/rag-workshop-code/blob/main/src/examples/03-json-chunking.ts"
    click ChunkMarkdown "https://github.com/mastra-ai/rag-workshop-code/blob/main/src/examples/04-markdown-chunking.ts"
    click Embedding "https://github.com/mastra-ai/rag-workshop-code/blob/main/src/examples/05-embedding.ts"
    click VecUpsert "https://github.com/mastra-ai/rag-workshop-code/blob/main/src/examples/06-vector-upserting.ts"
    click VecSearch "https://github.com/mastra-ai/rag-workshop-code/blob/main/src/examples/07-vector-search.ts"
    click VecRerank "https://github.com/mastra-ai/rag-workshop-code/blob/main/src/examples/08-vector-reranking.ts"
    click BasicAgent "https://github.com/mastra-ai/rag-workshop-code/blob/main/src/mastra/agents/basic-agent.ts"
    click QueryAgent "https://github.com/mastra-ai/rag-workshop-code/blob/main/src/mastra/agents/query-vector-agent.ts"
    click BasicEx "https://github.com/mastra-ai/rag-workshop-code/blob/main/src/examples/09-basic-search-usage.ts"
    click QueryEx "https://github.com/mastra-ai/rag-workshop-code/blob/main/src/examples/10-query-vector-usage.ts"
    click FindUsage "https://github.com/mastra-ai/rag-workshop-code/blob/main/src/bonus/01-find-code-usage.ts"
    click BonusAgent "https://github.com/mastra-ai/rag-workshop-code/blob/main/src/bonus/agent.ts"
    click BonusTool "https://github.com/mastra-ai/rag-workshop-code/blob/main/src/bonus/tool.ts"
```
