How fahadahmedk/ragdocmanager2 works
Main Components
• API Layer: Represented by the “api” directory, this layer exposes REST endpoints. The subdirectories indicate routes for “documents”, “registration”, and “search”.
• Document Processing Module: Under “document_processor” you have submodules for chunkers, document loaders, and embedders, plus processing utilities. This module is responsible for transforming raw documents into indexed content.
• Document Management Module: In “document_manager”, which contains logic for document management including Azure-specific implementations.
• Indexing and Search Subsystems:
• “index” and “index_manager” hold indexing logic, including adapters for Azure AI indexing.
• “search” contains search logic with an adapter for Azure AI query.
• Customer Management Module: The “customer_manager” handles customer-specific data and configurations.
• Storage Services: The “storage” directory is divided into:
• Database manager (with a CosmosDB manager)
• History storage
• Object storage (azure_blob_storage)
• Secrets storage, including Azure Key Vault integrations
• Utils: The “utils” module provides helper functions such as for I/O operations.
• Main Entry Point: “main.py” acts as the application’s entry point to wire all these components.
Relationships and Interactions
• API routes receive incoming requests (such as document uploads, registration, and search queries) and then forward them to the appropriate service components.
• Document Management and Processing modules interact; documents might first be loaded and chunked, then embedded, and subsequently sent to the indexing module.
• The indexing module (comprising “index” and “index_manager”) builds indexes used by the search module to efficiently retrieve documents.
• Customer management ensures that customer-specific configurations or policies are applied across the document and indexing workflows.
• The storage layer interacts with several modules for data persistence, including saving raw documents, the indexing data, and maintaining history, as well as managing authentication and secrets via the secrets module.
Architectural Patterns and Key Technologies
• Modular Design: The project is clearly partitioned into specialized modules (API, document processing, indexing, search, storage) that interact with one another.
• Adapter/Factory Patterns: Many modules (document_processor’s chunkers, document_loaders, embedders; search and index adaptors) use factory and adapter patterns to allow integration with different external providers (e.g., Azure AI services).
• Integration with External Services: The project leverages Azure’s services for document management (azure_document_manager), indexing (azure_ai_indexing_engine), query (azure_ai_query_engine), storage (azure_blob_storage), databases (cosmosdb_manager), and secrets (azure_key_vault).