How seekdavidlee/all-in-one-chatbot works
Main Components
• Ingestion Module:
• Data source ingestion and chunking (reflected in src/Ingestions, e.g., IIngestionDataSource implementations, IngestionProcessor, and specialized classes like LocalDirectoryIngestion, RestApiIngestion).
• Embedding integration (src/Embeddings) with both local model embedding and Azure OpenAI support.
• Indexing/publishing into a vector database (src/VectorDbs, including AzureAISearch and ChromaDbClient).
• Inference Module:
• Core chatbot logic: handles query input, intent determination, document retrieval, and LLM responses (located in src/Inferences, with sub-steps such as DetermineIntentStep, DetermineReplyStep, and RetrievedDocumentsStep).
• HTTP server support (via the HttpChatbotCommand) for handling inference requests over HTTP, probably driven by environment variables and routing to the proper workflow.
• Evaluation Module:
• Ground truth handling and evaluation workflows (in src/Evals), including the evaluation runner, metric workflow, and excel-based ground truth readers.
• Operations for both local and remote evaluations (commands such as local-evals and remote-evals indicate this modularity).
• Command/Orchestration Layer:
• Various commands (as seen in src/Commands, each mapping to a different mode such as ingest, chatbot, delete-search, import-ground-truths, etc.) to trigger the respective workflows.
• Language Model (LLM) Integration:
• Classes under src/Llms represent both Azure OpenAI and local LLM clients, along with supporting models and options.
• Supporting Infrastructure:
• Configuration management (src/Configurations), logging (src/Logging), and the overall Program.cs that initiates the console application.
• Docker configuration (Dockerfile, .dockerignore) and build/deployment scripts (Build.ps1, RunContainerApps.ps1) that illustrate containerization and deployment practices.
Relationships and Interactions
• Ingestion Workflow:
• Data flows from various ingestion data sources through the IngestionProcessor. It undergoes text chunking, then the content is embedded (via the IEmbedding implementations), and finally stored/indexed in a vector database via the IVectorDb interface.
• This flow is potentially executed either as a direct process or via a queue (as evidenced by commands like ingest-queue-processing) to enable distributed processing.
• Inference Workflow:
• User queries (from console or HTTP) are processed by the inference engine working through a series of workflow steps (like intent determination, document retrieval, and final response generation).
• The system harnesses either a local inference workflow (with local LLM support) or delegates to an Azure-based LLM.
• The HTTP server component adds an interface layer over this entire inference pipeline.
• Evaluation Workflow:
• Ground truth data (extracted by IGroundTruthReader, primarily from Excel documents) is fed into the EvaluationRunner.
• The workflow compares inference outputs against ground truths, performs metric evaluations, and produces summarized reports (with commands like summarize and show-metric-eval).
• Cross-module Interactions:
• Commands serve as entry points to specific orchestration flows (each command in src/Commands triggers a set of operations from ingestion, inference, or evaluation).
• Common utilities (configurations, logging, error handling) are shared across modules.
Architectural Patterns and Design Principles
• Modular separation: Although it is a monolithic codebase, the solution cleanly isolates ingestion, inference, and evaluation concerns into separate namespaces/directories.
• Command Pattern: The different commands encapsulate the behavior required to start various workflows, allowing for decoupling of the main program from specific operations.
• Dependency Injection: Interfaces such as IEmbedding, IVectorDb, ILanguageModel, and IGroundTruthReader suggest that dependency injection is used to keep components loosely coupled.
• Containerization: The provided Dockerfile and container-specific scripts show that the design is optimized for deployment in portable container environments.
Technologies and Frameworks
• .NET (with its project file AIOChatbot.csproj) as the core framework.
• Azure OpenAI and Azure services (e.g., Blob storage, Azure AI Search) as external services.
• Local implementations of embeddings and LLM (for scenarios where hosted services are not preferred).
• ChromaDB as an alternative vector database.
• PowerShell scripts for build and deployment automation.