# marcominerva/sqldatabasevectorsearch architecture

> Architecture diagram of the GitHub repository marcominerva/sqldatabasevectorsearch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/marcominerva/sqldatabasevectorsearch
- Repository: https://github.com/marcominerva/sqldatabasevectorsearch

## Overview

##### Main components
- User Interface / Web Frontend: Represented by the Blazor Web App components (under Components, Pages, Layout, etc.). This handles user interactions (document uploads, asking questions, etc.).
- API Endpoints: The Minimal API endpoints (AskEndpoints.cs, DocumentEndpoints.cs) that allow external systems to interact with the application.
- Data Access Layer: Uses Entity Framework Core to interact with the underlying Azure SQL Database. This includes the ApplicationDbContext, entity definitions (Document, DocumentChunk, etc.) and database migration files.
- Services: Core business logic is provided by services such as ChatService, DocumentService, TokenizerService, and VectorSearchService. They handle embedding generation, performing vector searches, tokenization, and chat completions.
- Content Handling Modules: ContentDecoders for processing different file types (PDF, DOCX, TXT) and TextChunkers for segmenting file content.
- Configuration & Settings: Managed through various appsettings.json files and Settings classes (e.g., AppSettings, AzureOpenAISettings) that hold connection strings, OpenAI configuration, and vector dimensions.
- External Dependencies: Azure SQL Database for storage, EFCore.SqlServer.VectorSearch library for vector search integration, and Semantic Kernel for interacting with Azure OpenAI services.

##### Relationships and interactions
- The Web App and API endpoints serve as the entry points for user actions and programmatic calls.
- Requests from the client trigger calls to the backend services (e.g., DocumentService for importing documents or ChatService for answering questions).
- These services interact with the Data Access Layer to store or retrieve data. There’s a bidirectional flow: read/write operations between the EF Core context and the database.
- The embedding and vector search capabilities (via VectorSearchService) further integrate with Azure OpenAI through Semantic Kernel and also rely on properly structured vector data stored in the database.
- Content decoders and text chunkers are used in preprocessing to extract and prepare data before further processing by the embedding service.

##### Important architectural patterns and design principles
- Separation of concerns: The project distinctly separates the UI layer, API layer, business logic (services), data access, and content processing modules.
- Modular design: Individual components like content decoders and text chunkers can be swapped or extended, highlighting flexibility.
- Use of Entity Framework Core for ORM and migrations ensuring data and model consistency.
- Integration of external libraries such as EFCore.SqlServer.VectorSearch and Semantic Kernel, emphasizing extensibility.
- Blazor-based frontend supports a more interactive web application experience, while the minimal API provides programmatic access.

##### Relevant technologies, frameworks, and libraries
- Blazor for the front end.
- ASP.NET Core Minimal API for backend HTTP-based interactions.
- Entity Framework Core with SQL Server and vector support libraries.
- Azure SQL Database as the data store.
- Semantic Kernel (for OpenAI integration).
- Use of standard .NET practices (dependency injection, configuration files).

##### Elements
- Components: Frontend (Blazor Web App), API Endpoints, Core Services, Data Access Layer, Database (Azure SQL Database).
- Flow paths for both document ingestion and the chat/QA features, including response streaming.
- Notes for components with special configuration or external dependency requirements, like the settings for embedding dimensions and SQL column sizes.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Frontend Layer
    subgraph "Frontend Layer"
        A["User Interface (Blazor Web App)"]:::frontend
    end

    %% API Layer
    subgraph "API Layer"
        B["API Endpoints"]:::backend
    end

    %% Business Services
    subgraph "Business Services"
        C1["DocumentService"]:::services
        C2["ChatService"]:::services
        C3["TokenizerService"]:::services
        C4["VectorSearchService"]:::services
    end

    %% Content Processing
    subgraph "Content Processing"
        subgraph "Content Decoders"
            D1a["DocxContentDecoder"]:::content
            D1b["PdfContentDecoder"]:::content
            D1c["TextContentDecoder"]:::content
        end
        subgraph "Text Chunkers"
            D2a["DefaultTextChunker"]:::content
            D2b["ITextChunker"]:::content
            D2c["MarkdownTextChunker"]:::content
        end
    end

    %% Data Access Layer
    subgraph "Data Access Layer"
        E["ApplicationDbContext & Entities"]:::data
    end

    %% Configuration & Settings
    subgraph "Configuration"
        F1["Settings Classes"]:::config
        subgraph "Configuration Files"
            F2a["appsettings.json"]:::config
            F2b["appsettings.Development.json"]:::config
        end
    end

    %% External Systems
    subgraph "External Systems"
        G1["Azure SQL Database"]:::external
        G2["Azure OpenAI (Semantic Kernel)"]:::external
    end

    %% Connections
    A -->|"triggers"| B
    B -->|"calls"| C1
    B -->|"calls"| C2
    B -->|"calls"| C3
    B -->|"calls"| C4

    %% Content Processing to DocumentService
    D1a -->|"preprocess"| C1
    D1b -->|"preprocess"| C1
    D1c -->|"preprocess"| C1
    D2a -->|"segment"| C1
    D2b -->|"segment"| C1
    D2c -->|"segment"| C1

    %% Business Services to Data Access Layer
    C1 -->|"read/write"| E
    C4 -->|"read/write"| E

    %% ChatService uses External OpenAI
    C2 -->|"invoke"| G2

    %% VectorSearchService interacts with Azure SQL Database
    C4 -->|"queries"| G1

    %% Configuration usage
    F1 ---|"provides"| C1
    F1 ---|"provides"| C2
    F1 ---|"provides"| C3
    F1 ---|"provides"| C4
    F2a ---|"configures"| B
    F2b ---|"configures"| B

    %% Click Events
    click A "https://github.com/marcominerva/sqldatabasevectorsearch/tree/master/SqlDatabaseVectorSearch/Components"
    click B "https://github.com/marcominerva/sqldatabasevectorsearch/tree/master/SqlDatabaseVectorSearch/Endpoints"
    click C1 "https://github.com/marcominerva/sqldatabasevectorsearch/tree/master/SqlDatabaseVectorSearch/Services"
    click C2 "https://github.com/marcominerva/sqldatabasevectorsearch/tree/master/SqlDatabaseVectorSearch/Services"
    click C3 "https://github.com/marcominerva/sqldatabasevectorsearch/tree/master/SqlDatabaseVectorSearch/Services"
    click C4 "https://github.com/marcominerva/sqldatabasevectorsearch/tree/master/SqlDatabaseVectorSearch/Services"
    click E "https://github.com/marcominerva/sqldatabasevectorsearch/tree/master/SqlDatabaseVectorSearch/DataAccessLayer"
    click D1a "https://github.com/marcominerva/sqldatabasevectorsearch/tree/master/SqlDatabaseVectorSearch/ContentDecoders"
    click D1b "https://github.com/marcominerva/sqldatabasevectorsearch/tree/master/SqlDatabaseVectorSearch/ContentDecoders"
    click D1c "https://github.com/marcominerva/sqldatabasevectorsearch/tree/master/SqlDatabaseVectorSearch/ContentDecoders"
    click D2a "https://github.com/marcominerva/sqldatabasevectorsearch/tree/master/SqlDatabaseVectorSearch/TextChunkers"
    click D2b "https://github.com/marcominerva/sqldatabasevectorsearch/tree/master/SqlDatabaseVectorSearch/TextChunkers"
    click D2c "https://github.com/marcominerva/sqldatabasevectorsearch/tree/master/SqlDatabaseVectorSearch/TextChunkers"
    click F1 "https://github.com/marcominerva/sqldatabasevectorsearch/tree/master/SqlDatabaseVectorSearch/Settings"
    click F2a "https://github.com/marcominerva/sqldatabasevectorsearch/blob/master/SqlDatabaseVectorSearch/appsettings.json"
    click F2b "https://github.com/marcominerva/sqldatabasevectorsearch/blob/master/SqlDatabaseVectorSearch/appsettings.Development.json"

    %% Styles
    classDef frontend fill:#FFDDCC,stroke:#333,stroke-width:2px;
    classDef backend fill:#CCFFCC,stroke:#333,stroke-width:2px;
    classDef services fill:#CCCCFF,stroke:#333,stroke-width:2px;
    classDef data fill:#FFCCCC,stroke:#333,stroke-width:2px;
    classDef content fill:#FFFFCC,stroke:#333,stroke-width:2px;
    classDef config fill:#CCFFFF,stroke:#333,stroke-width:2px;
    classDef external fill:#FFE6E6,stroke:#333,stroke-width:2px;
```
