How kevin-khoamai/ragflow works
Project Type and Purpose
• This is RAGFlow, an open-source, full-stack Retrieval-Augmented Generation (RAG) engine.
• Purpose: provide a complete RAG workflow that ingests heterogeneous documents, extracts and indexes chunks (“DeepDoc”), performs retrieval + re-ranking, runs LLM-based generation, and exposes APIs + UI for business integration.
High-Level Architectural Domains
• Frontend (web/)
• React + TypeScript UI for dataset management, chat/flow editors, agents, document viewers.
• Communicates over HTTP/REST with the backend API.
• Backend API Layer (api/)
• FastAPI/Uvicorn apps: each “app” module under api/apps handles a domain (chat, dataset, file, document, kb, llm, auth, plugin, etc.).
• API gateway/ragflow_server.py routes requests to these applications.
• Exposes SDK (sdk/python/ragflow_sdk) and HTTP OpenAPI.
• Core RAG Engine (rag/, deepdoc/, agent/, agentic_reasoning/)
• rag/: orchestrates retrieval, chunking, embedding search, LLM calls, re-ranking.
• deepdoc/: document parsers (pdf, docx, ppt, excel, html, markdown, etc.) + vision OCR/layout.
• agent/: modular “Agent” components (search, code, email, iterate, template) with DSL builder (canvas.py + component/).
• agentic_reasoning/: deep research workflows, multi-step prompts.
• Sandbox Executor (sandbox/)
• gVisor-backed container manager for code execution.
• HTTP service for running untrusted code in isolated environments.
• Plugins and Integrations (plugin/, intergrations/)
• Plugin manager embeds LLM-tool plugins, Chrome extension, ChatGPT-WeChat bridge.
• Data Stores and External Services (docker/, helm/)
• Relational DB: MySQL (api/db/db_models.py).
• Object storage: MinIO for raw docs.
• Search engine: Elasticsearch or Infinity (vector + full-text).
• Cache: Redis.
• LLM & embedding endpoints: OpenAI/Azure/HuggingFace/Tavily, etc.
• Additional: Langfuse, MCP server for multi-tenant.
Essential Elements to Capture
• Labels: component names, technologies, ports (optional).