How predixus/dynarag works
Project Type & Overall Purpose
• Project Type: A backend service implementing a RAG pipeline.
• Purpose: To provide fast and robust operations for adding text chunks, performing vector similarity (using PGVector in PostgreSQL), and leveraging LLM-based summarisation/response generation. Essentially, it is engineered for optimal performance and minimal overhead, ideal for inclusion in larger applications that require efficient text chunk handling and retrieval.
Key Components
a. Core Backend Service:
• Depicted by the main executable (e.g., “dynarag.go”) which wires together the service’s business logic.
• Initialization process where migrations are executed (see “db-init-scripts” and “migrations” directories).
b. Database Component:
• PostgreSQL with the PGVector extension configured via migrations.
• That the “internal/store” package manages database interactions—including schema definitions, SQL queries (via sqlc-generated code), and migrations.
c. Embedding/Feature Extraction Module:
• Use a module block labeled “Embedding Module” (internal/embed) which interacts with the Onnx runtime and Hugot library.
• That, on first inference, it downloads the necessary .onnx models from Huggingface and stores them locally under a models directory.
d. LLM Integration Module:
• “LLM Module” (internal/llm) as the component managing the interface to various external LLM providers (e.g., Groq or Ollama).
• That this module handles the generation/summarization aspect of the pipeline.
e. RAG Orchestration Layer:
• “RAG Module” (internal/rag) that serves as the final summarisation layer and application of system prompts, combining the outputs from the embedding/LLM modules.
• That it manipulates the text chunks before feeding the LLM.
f. Utility and Shared Modules:
g. Client Operations/Examples:
• Example clients (from the examples/ folder) demonstrating how operations like “Chunk”, “Similar”, “Query”, and “PurgeChunks” are invoked.
Relationships and Data Flow
• How user/client requests enter the main service which then calls the RAG orchestration layer.
• That migrations initialize/configure the database, setting up tables and extensions before any client operations commence.
Architectural Patterns & Design Principles
• That the project leverages a modular design:
• Each “internal” package (llm, rag, embed, store, utils) represents a clear separation of concerns.
• The project takes advantage of Go’s strengths, including concurrency and a single binary deployment.
• How the separation also helps in optimising latency (e.g., by pushing inference and database query latencies into the parts of the pipeline with the lowest round trip time).
• That the project uses configuration files like docker-compose.yml for container setups and a Makefile for build tasks.