How henriquelca/ragflow works
Main Components
• Frontend: The “web” directory contains a React/TypeScript-based web client that handles user interaction, integration of charts/graphs, file management, and real-time flows.
• Backend API: The “api” folder includes multiple application modules (such as canvas_app, conversation_app, document_app, etc.), utilities, database models and services, and the entry point for the server (ragflow_server.py).
• Agent Engine: The “agent” directory consists of modular components or “agents” (e.g., generate, retrieval, rewrite, etc.) that help in orchestrating the workflows (e.g., text chunking, LLM invocation, question answering).
• Document Parsing (Deepdoc): The “deepdoc” folder contains parsers for various document formats (docx, pdf, excel, etc.), OCR, vision modules, and resume parsing capabilities.
• SDK Modules: The “sdk” folder gives developers integration points and a Python SDK to work with the system, exposing modules for agent, chat, document, etc.
• Docker & Deployment: Dockerfiles, docker-compose files, and helm charts (in “docker” and “helm” folders) illustrate containerization and orchestration strategies.
Relationships and Interactions
• The API acts as a gateway that routes requests to the appropriate internal services. This includes calls to database services (e.g., MySQL, Redis, Elasticsearch/Infinity) and agent functions.
• The Agent Engine interacts with both the RAG component (to invoke LLM or retrieval operations) and with external LLM/embedding services, as indicated in the configuration files.
• The Deepdoc module parses incoming documents and feeds structured data to the RAG components or agents for further processing.
• The SDK serves as an abstraction layer for third-party integrations and developer tooling.
• Docker, helm, and other orchestration components indicate that each major system module is containerized and deployed within a multi-service environment.
Architectural Patterns and Design Principles
• Modular Design: Each major component (agent, API, deepdoc, rag, web, sdk) is self-contained, which facilitates plug-and-play for new features, agents, or parsers.
• Microservices/Service-Oriented Architecture: Although the system can be run as a monolith, the presence of separate Docker containers and service configuration files suggests a move toward service isolation.
• Layered Architecture: The API layer abstracts interactions from the frontend, while backend services encapsulate business logic such as LLM operations, document parsing, and database services.
• Extensibility: Through well-defined agents, templates, and SDK modules, developers can extend system functionalities or integrate new processing components.
Relevant Technologies, Frameworks, and Libraries
• Frontend: Built with React + TypeScript, along with supporting libraries (e.g., ESLint, Prettier, Tailwind CSS) evident in configuration files.
• Backend: Python-based with frameworks likely surrounding Flask or a similar asynchronous framework (e.g., usage of “uv” for launching services) and database ORM setups.
• Document Understanding: Deepdoc uses a suite of Python parsers for various formats and OSS OCR solutions.
• Containerization and Orchestration: Docker, Docker Compose, and Helm are used for deploying services.
• External Services: Integration with LLMs, embedding models, Elasticsearch/Infinity for document indexing, and other tools like Redis, MySQL, and MinIO.
Final Detailed Steps
a. Overall system boundary and add the major layers (Frontend, Backend, Processing Components, Deployment Infrastructure).
b. Subdivide the backend layer into microservice components such as the API endpoints, agents, document processing (deepdoc), and RAG core.
c. Interactions from user request (via the web UI) passing into the API, triggering document processing or agent jobs, and finally returning processed responses.
d. Side components like the SDK for developers and external service integration for LLM and indexing engines.