# fahadahmedk/ragdocmanager3 architecture

> Architecture diagram of the GitHub repository fahadahmedk/ragdocmanager3, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/fahadahmedk/ragdocmanager3
- Repository: https://github.com/fahadahmedk/ragdocmanager3

## Overview

##### Main Components
- API Layer: Under src/rag_doc_manager/api, you have routes for documents, registrations, and search. This is the primary entry point for client requests.
- Document Management: Located in src/rag_doc_manager/document_manager, this module handles contracts and the logic for managing documents (including interaction with external services, such as Azure).
- Customer Management: Under src/rag_doc_manager/customer_manager, it handles customer-specific data models and remote schema management.
- Document Processing: Found in src/rag_doc_manager/document_processor, including submodules for chunking, document loading, and embedding generation. This indicates a pipeline that processes raw documents into indexed content.
- Index and Search Components: The index (src/rag_doc_manager/index) and index_manager (src/rag_doc_manager/index_manager) modules deal with creating or managing document indices. In addition, the search module along with its adaptors (e.g., azure_ai_query_engine) shows how actual search queries are handled.
- Storage Layer: Under src/rag_doc_manager/storage, multiple submodules exist:
– Database Manager: Likely managing a NoSQL database such as Cosmos DB (in cosmosdb_manager.py).
– Object Storage: Likely interfacing with Azure Blob Storage for document files.
– Secrets Management: Modules to handle credentials and connect securely to external resources through Azure Key Vault.
- Utility Components: General helper functions in src/rag_doc_manager/utils.
- Main Entrypoint: src/rag_doc_manager/main.py, which probably ties multiple components together and starts the service.

##### Relationships and Interactions
- The client interacts with the API endpoints. The API layer routes calls (e.g., document submission, registration, search) to the appropriate underlying modules.
- For document ingestion, the API forwards incoming documents to the Document Manager and subsequently to the Document Processing module, where files are chunked, loaded, and embedded.
- Processed document data is then handed off to the Index or Index Manager components to create or update searchable indices.
- The Search module handles queries by interfacing with appropriate indexing structures, potentially leveraging external services (such as Azure AI query services).
- The storage layer ensures persistence: the database component records metadata and histories, object storage holds raw documents, and secrets management secures credentials.
- Customer Manager likely works in parallel to ensure that customer-specific schemas or data configurations are respected during indexing or processing.

##### Architectural Patterns and Design Principles
- Extensibility: Factories (seen in several “factory.py” files) suggest that the project uses design patterns to support plug-in like behavior (for chunkers, document loaders, and embedders). This is an important detail to include.

##### Technologies, Frameworks, and Libraries
- Python is the primary language.
- Azure services (blob storage, key vault, Cosmos DB, and possibly AI indexing and querying engines) are critical.
- Docker is likely used for containerization (given Dockerfile and .dockerignore).
- CI/CD integration is hinted at within the README (GitLab pipelines).

##### Essential Elements
- Clear, labeled blocks (with names like “API Routes”, “Document Manager”, “Document Processor”, “Index Manager”, “Storage – Database”, “Storage – Blob”, “Secrets Management”).
- Optionally, include notations or legends that indicate the type of service (internal modules vs. external services) and technologies used.
- That external services (Azure components) are clearly depicted with their branding or labels.

## Diagram (Mermaid)

```mermaid
flowchart TD
    CLIENT["Client"]:::external
    CLIENT -->|"APIcall"| API["API Routes"]:::api

    subgraph "API Layer"
        API["API Routes"]:::api
    end

    subgraph "Business Logic"
        DM["Document Manager"]:::business
        CM["Customer Manager"]:::business
    end

    subgraph "Document Processing"
        PROC["Processor Logic"]:::process
        DL["Document Loaders"]:::process
        CH["Chunkers"]:::process
        EMB["Embedders"]:::process
    end

    subgraph "Indexing and Search"
        IDX["Index Components"]:::index
        IM["Index Manager"]:::index
        SRCH["Search Module"]:::index
    end

    subgraph "Storage Layer"
        DB["Database Manager"]:::storage
        OB["Object Storage"]:::storage
        SEC["Secrets Management"]:::storage
        HIS["History Storage"]:::storage
    end

    subgraph "Utilities & Entrypoint"
        UT["Utilities"]:::util
        MAIN["Main Entrypoint"]:::util
    end

    %% Relationships
    API -->|"documentRequest"| DM
    API -->|"searchRequest"| SRCH

    DM -->|"processDocument"| PROC
    CM -->|"customerData"| DM

    PROC -->|"loadDocument"| DL
    DL -->|"chunkDocument"| CH
    CH -->|"generateEmbeddings"| EMB
    EMB -->|"updateIndex"| IDX

    IDX -->|"manageIndex"| IM
    IM -->|"storeIndexUpdates"| DB
    SRCH -->|"queryIndex"| IDX

    DM -->|"storeMetadata"| DB
    DM -->|"uploadDocs"| OB
    DM -->|"accessSecrets"| SEC

    %% External Storage interactions
    OB -->|"storeFiles"| DB

    %% Service startup
    MAIN -->|"initService"| API
    UT -->|"supportFunctions"| DM

    %% Styles
    classDef external fill:#f9e79f,stroke:#333,stroke-width:2px;
    classDef api fill:#ffcccc,stroke:#900,stroke-width:2px;
    classDef business fill:#ccffcc,stroke:#090,stroke-width:2px;
    classDef process fill:#cce5ff,stroke:#009,stroke-width:2px;
    classDef index fill:#ffe5cc,stroke:#996600,stroke-width:2px;
    classDef storage fill:#e6ccff,stroke:#660099,stroke-width:2px;
    classDef util fill:#ffffcc,stroke:#999900,stroke-width:2px;

    %% Click Events
    click API "https://github.com/fahadahmedk/ragdocmanager3/tree/main/src/rag_doc_manager/api/routes"
    click DM "https://github.com/fahadahmedk/ragdocmanager3/tree/main/src/rag_doc_manager/document_manager"
    click CM "https://github.com/fahadahmedk/ragdocmanager3/tree/main/src/rag_doc_manager/customer_manager"
    click DL "https://github.com/fahadahmedk/ragdocmanager3/tree/main/src/rag_doc_manager/document_processor/document_loaders"
    click CH "https://github.com/fahadahmedk/ragdocmanager3/tree/main/src/rag_doc_manager/document_processor/chunkers"
    click EMB "https://github.com/fahadahmedk/ragdocmanager3/tree/main/src/rag_doc_manager/document_processor/embedders"
    click PROC "https://github.com/fahadahmedk/ragdocmanager3/blob/main/src/rag_doc_manager/document_processor/processor.py"
    click IDX "https://github.com/fahadahmedk/ragdocmanager3/tree/main/src/rag_doc_manager/index"
    click IM "https://github.com/fahadahmedk/ragdocmanager3/tree/main/src/rag_doc_manager/index_manager"
    click SRCH "https://github.com/fahadahmedk/ragdocmanager3/tree/main/src/rag_doc_manager/search"
    click DB "https://github.com/fahadahmedk/ragdocmanager3/tree/main/src/rag_doc_manager/storage/database_manager"
    click OB "https://github.com/fahadahmedk/ragdocmanager3/tree/main/src/rag_doc_manager/storage/object"
    click SEC "https://github.com/fahadahmedk/ragdocmanager3/tree/main/src/rag_doc_manager/storage/secrets"
    click HIS "https://github.com/fahadahmedk/ragdocmanager3/tree/main/src/rag_doc_manager/storage/history"
    click UT "https://github.com/fahadahmedk/ragdocmanager3/tree/main/src/rag_doc_manager/utils"
    click MAIN "https://github.com/fahadahmedk/ragdocmanager3/blob/main/src/rag_doc_manager/main.py"
```
