# fufankeji/fufan-chat-api architecture

> Architecture diagram of the GitHub repository fufankeji/fufan-chat-api, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/fufankeji/fufan-chat-api
- Repository: https://github.com/fufankeji/fufan-chat-api

## Overview

1. **System boundary and purpose**
   - This repo is a Python backend for an LLM-based Q&A platform, with a separately built Vue frontend served from `static/dist`.
   - The production backend lives primarily under `server/`; the rest of the tree contains reusable data-processing code, RAG evaluation code, deployment assets, and playground demos.

2. **Request entry and API surface**
   - Runtime starts from `startup.py`, which boots the backend app in `server/main.py`.
   - HTTP routes are assembled in `server/api_router.py` and fan out into specialized chat endpoints for general chat, knowledge-base chat, search chat, agent chat, and recommendation chat.
   - The backend is designed as a service layer around FastAPI-style request handling, with streaming callbacks for model responses.

3. **Conversation and chat orchestration**
   - The core orchestration is in `server/chat/`.
   - `chat.py` acts as the generic conversation entry point, while `agent_chat.py`, `knowledge_base_chat.py`, `search_engine_chat.py`, and `recommendation_chat.py` implement the different application scenarios.
   - Conversation state and history are part of the request flow, so each chat mode can combine current user input with stored session context.

4. **LLM integration layer**
   - `server/model_workers/` abstracts model backends and keeps the app decoupled from a specific provider.
   - The repo targets both local/open-source models and hosted APIs, with examples and fixes for GLM/ChatGLM-style models called out in the README.
   - `server/minx_chat_openai.py` and `server/embeddings_api.py` indicate a provider-agnostic interface for chat generation and embeddings.
   - Streaming generation is important in the architecture: model output is not treated as a single blocking response.

5. **Persistence and application metadata**
   - `server/db/` is the relational persistence layer, with SQLAlchemy-style models and repositories.
   - It stores users, conversations, messages, knowledge bases, and knowledge files, which are the main business entities in the system.
   - This database layer supports authentication/authorization, session history, and knowledge-base ownership boundaries.

6. **Knowledge base subsystem**
   - `server/knowledge_base/` manages local KB lifecycle: initialization, caching, document indexing, and retrieval service selection.
   - There are two vector-store backends: Faiss and Milvus, surfaced through `kb_service/faiss_kb_service.py` and `kb_service/milvus_kb_service.py`.
   - `kb_cache/` indicates a cache layer so KB metadata and vector-store handles can be reused efficiently.
   - This subsystem is the main boundary between uploaded/content files and RAG query-time retrieval.

7. **Document ingestion and text processing**
   - Ingestion starts outside the main server in `document_loaders/`, `text_splitter/`, and `data/parse/`.
   - PDF and other source documents are normalized, parsed, and split into chunks before indexing.
   - The `data/parse` area also includes rule-based and unstructured parsing logic, showing that ingestion is not a simple file upload path but a preprocessing pipeline.

8. **Retrieval-Augmented Generation flow**
   - For KB Q&A, the typical flow is: user query → retrieve relevant chunks from Faiss/Milvus → assemble prompt with history/context → generate answer through the selected model backend.
   - Reranking is a first-class part of retrieval: `server/reranker/` and the `fufanrag/retriever/` code show that initial retrieval results are refined before generation.
   - This RAG path is the main architecture for private knowledge-base question answering.

9. **Web search / internet retrieval path**
   - The search scenario adds an external search provider and then re-indexes retrieved web content for downstream answering.
   - `server/agent/tools/duckgosearch.py` and the README’s Serper/Google-search flow suggest that external search is treated as a tool-backed retrieval source rather than a separate app.
   - This path differs from private KB Q&A by introducing an internet-fetch, extraction, rerank, and temporary indexing stage.

10. **Agent subsystem and tool use**
    - `server/agent/` contains the agent framework, tool selection, callback handling, and custom agent definitions.
    - The agent path is boundary-driven: the model can call tools, receive tool outputs, and continue reasoning before final response generation.
    - This subsystem is distinct from plain chat because it adds execution of external tools and tool-routing logic.

11. **Recommendation scenario**
    - `server/chat/recommendation_chat.py` implements the recommendation-oriented conversational flow.
    - Supporting data lives under `scripts/course_data/`, which suggests the recommendation system is built around educational content and user behavior/history.
    - This is not a generic recommender microservice; it is integrated into the same conversational backend and uses chat history plus domain data.

12. **RAG evaluation and offline experiments**
    - The `fufanrag/` package is a self-contained evaluation/research pipeline, separate from the runtime API.
    - It includes dataset handling, retrieval, generation, metrics, and evaluator logic, plus sample test/index/result artifacts under `fufanrag/data/`.
    - This package is important architecturally because it provides the measurement loop for improving retrieval and generation quality, but it is not the live serving path.

13. **Deployment and infrastructure**
    - `docker/` provides containerization, and `docs/` includes operational guidance for MySQL and Milvus.
    - The system depends on MySQL for relational state and on a vector database layer for RAG; Faiss is used for local vector stores, while Milvus is used where a service-backed vector DB is needed.
    - The architecture is therefore a multi-store design: relational DB for app state, vector store for semantic retrieval, and model endpoints for inference.

14. **Non-production and example code**
    - `playground/` contains many isolated demos for FastAPI, FastChat, LangChain, MySQL, and model deployment experiments.
    - These are useful for understanding integration patterns, but they are not the core runtime graph of the product.
    - For an architecture diagram, they should be shown as supporting examples, not as part of the serving path.

15. **Key high-level data flow**
    - User request enters the API gateway/router, is routed to a chat mode, fetches session/user metadata from MySQL, optionally retrieves from a vector store or external search, optionally invokes tools through the agent layer, and finally calls the selected LLM backend to stream an answer back.
    - Offline flows run in parallel: document parsing and preprocessing populate knowledge bases and indexes, while evaluation pipelines measure retrieval/generation quality.

Main entry points: [`server/api_router.py`](https://github.com/fufankeji/fufan-chat-api/blob/main/server/api_router.py) (API router), [`server/minx_chat_openai.py`](https://github.com/fufankeji/fufan-chat-api/blob/main/server/minx_chat_openai.py) (Model API), [`server/chat/knowledge_base_chat.py`](https://github.com/fufankeji/fufan-chat-api/blob/main/server/chat/knowledge_base_chat.py) (KB chat), [`server/chat/chat.py`](https://github.com/fufankeji/fufan-chat-api/blob/main/server/chat/chat.py) (Chat core)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Serving runtime"]
  node_startup["Startup<br/>entrypoint<br/>[startup.py]"]
  node_main["Server app<br/>fastapi app<br/>[main.py]"]
  node_router["API router<br/>http routing<br/>[api_router.py]"]
  node_chat_core["Chat core<br/>conversation flow<br/>[chat.py]"]
  node_kb_chat["KB chat<br/>rag chat"]
  node_search_chat["Search chat<br/>web retrieval chat"]
  node_agent_chat["Agent chat<br/>tool-using chat<br/>[agent_chat.py]"]
  node_reco_chat["Reco chat<br/>recommendation chat"]
  node_db[("Database layer<br/>sql persistence<br/>[base.py]")]
  node_memory["Conversation memory<br/>session memory"]
  node_model_api["Model API<br/>llm adapter"]
  node_workers["Model workers<br/>backend adapters<br/>[base.py]"]
  node_kb_service["KB services<br/>vector retrieval<br/>[base.py]"]
  node_kb_cache["KB cache<br/>metadata cache<br/>[base.py]"]
  node_reranker["Reranker<br/>ranking stage<br/>[reranker.py]"]
  node_agent_tools["Agent tools<br/>tooling<br/>[tools_select.py]"]
end

subgraph group_offline["Offline pipelines"]
  node_document_loaders["Loaders<br/>document intake<br/>[interface.py]"]
  node_parsing["Parsing<br/>text preprocessing"]
  node_splitter["Text split<br/>chunking"]
  node_eval_pipeline["RAG eval<br/>evaluation pipeline<br/>[pipeline.py]"]
end

subgraph group_support["Supporting assets"]
  node_deploy["Docker<br/>deployment"]
  node_frontend["Frontend dist<br/>built vue app<br/>[index.html]"]
  node_playground["Playground<br/>examples<br/>[startupV1.py]"]
end

node_startup -->|"boots"| node_main
node_main -->|"registers routes"| node_router
node_router -->|"generic chat"| node_chat_core
node_router -->|"kb qa"| node_kb_chat
node_router -->|"web search"| node_search_chat
node_router -->|"agent mode"| node_agent_chat
node_router -->|"recommendation"| node_reco_chat
node_chat_core -->|"loads history"| node_memory
node_chat_core -->|"stores state"| node_db
node_chat_core -->|"streams response"| node_model_api
node_model_api -->|"uses backends"| node_workers
node_kb_chat -->|"retrieves chunks"| node_kb_service
node_kb_chat -->|"reuses handles"| node_kb_cache
node_kb_chat -->|"refines results"| node_reranker
node_kb_chat -->|"generates answer"| node_model_api
node_search_chat -->|"search tool"| node_agent_tools
node_search_chat -->|"reranks web docs"| node_reranker
node_agent_chat -->|"selects tools"| node_agent_tools
node_agent_chat -->|"reasoning loop"| node_model_api
node_reco_chat -->|"reads course data"| node_db
node_reco_chat -->|"responds"| node_model_api
node_document_loaders -->|"normalizes docs"| node_parsing
node_parsing -->|"chunks text"| node_splitter
node_splitter -->|"indexes KB"| node_kb_service
node_eval_pipeline -->|"tests retrieval"| node_kb_service
node_eval_pipeline -->|"tests generation"| node_model_api
node_deploy -->|"runs"| node_startup
node_frontend -->|"calls api"| node_router
node_playground -->|"experiments"| node_model_api

click node_startup "https://github.com/fufankeji/fufan-chat-api/blob/main/startup.py"
click node_main "https://github.com/fufankeji/fufan-chat-api/blob/main/server/main.py"
click node_router "https://github.com/fufankeji/fufan-chat-api/blob/main/server/api_router.py"
click node_chat_core "https://github.com/fufankeji/fufan-chat-api/blob/main/server/chat/chat.py"
click node_kb_chat "https://github.com/fufankeji/fufan-chat-api/blob/main/server/chat/knowledge_base_chat.py"
click node_search_chat "https://github.com/fufankeji/fufan-chat-api/blob/main/server/chat/search_engine_chat.py"
click node_agent_chat "https://github.com/fufankeji/fufan-chat-api/blob/main/server/chat/agent_chat.py"
click node_reco_chat "https://github.com/fufankeji/fufan-chat-api/blob/main/server/chat/recommendation_chat.py"
click node_db "https://github.com/fufankeji/fufan-chat-api/blob/main/server/db/base.py"
click node_memory "https://github.com/fufankeji/fufan-chat-api/blob/main/server/memory/conversation_db_buffer_memory.py"
click node_model_api "https://github.com/fufankeji/fufan-chat-api/blob/main/server/minx_chat_openai.py"
click node_workers "https://github.com/fufankeji/fufan-chat-api/blob/main/server/model_workers/base.py"
click node_kb_service "https://github.com/fufankeji/fufan-chat-api/blob/main/server/knowledge_base/kb_service/base.py"
click node_kb_cache "https://github.com/fufankeji/fufan-chat-api/blob/main/server/knowledge_base/kb_cache/base.py"
click node_reranker "https://github.com/fufankeji/fufan-chat-api/blob/main/server/reranker/reranker.py"
click node_agent_tools "https://github.com/fufankeji/fufan-chat-api/blob/main/server/agent/tools_select.py"
click node_document_loaders "https://github.com/fufankeji/fufan-chat-api/blob/main/document_loaders/interface.py"
click node_parsing "https://github.com/fufankeji/fufan-chat-api/blob/main/data/parse/unstructured_processor.py"
click node_splitter "https://github.com/fufankeji/fufan-chat-api/blob/main/text_splitter/chinese_recursive_text_splitter.py"
click node_eval_pipeline "https://github.com/fufankeji/fufan-chat-api/blob/main/fufanrag/pipeline/pipeline.py"
click node_deploy "https://github.com/fufankeji/fufan-chat-api/tree/main/docker/Dockerfile"
click node_frontend "https://github.com/fufankeji/fufan-chat-api/blob/main/static/dist/index.html"
click node_playground "https://github.com/fufankeji/fufan-chat-api/blob/main/playground/test_startup/startupV1.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_startup,node_main,node_router,node_chat_core,node_kb_chat,node_search_chat,node_agent_chat,node_reco_chat,node_db,node_memory,node_model_api,node_workers,node_kb_service,node_kb_cache,node_reranker,node_agent_tools toneBlue
class node_document_loaders,node_parsing,node_splitter,node_eval_pipeline toneAmber
class node_deploy,node_frontend,node_playground toneMint
```

## Components

### Serving runtime

Live API, chat orchestration, persistence, model access, and retrieval.

- [Startup](https://github.com/fufankeji/fufan-chat-api/blob/main/startup.py): entrypoint · Bootstraps the backend process. · `startup.py`
- [Server app](https://github.com/fufankeji/fufan-chat-api/blob/main/server/main.py): fastapi app · Creates and runs the backend application. · `server/main.py`
- [API router](https://github.com/fufankeji/fufan-chat-api/blob/main/server/api_router.py): http routing · Assembles request routes and dispatches chat scenarios. · `server/api_router.py`
- [Chat core](https://github.com/fufankeji/fufan-chat-api/blob/main/server/chat/chat.py): conversation flow · Generic conversation entry point and shared chat utilities. · `server/chat/chat.py`
- [KB chat](https://github.com/fufankeji/fufan-chat-api/blob/main/server/chat/knowledge_base_chat.py): rag chat · Knowledge-base question answering orchestration. · `server/chat/knowledge_base_chat.py`
- [Search chat](https://github.com/fufankeji/fufan-chat-api/blob/main/server/chat/search_engine_chat.py): web retrieval chat · Internet-search-backed answering flow. · `server/chat/search_engine_chat.py`
- [Agent chat](https://github.com/fufankeji/fufan-chat-api/blob/main/server/chat/agent_chat.py): tool-using chat · Tool-augmented reasoning and action loop. · `server/chat/agent_chat.py`
- [Reco chat](https://github.com/fufankeji/fufan-chat-api/blob/main/server/chat/recommendation_chat.py): recommendation chat · Conversational recommendation flow. · `server/chat/recommendation_chat.py`
- [Database layer](https://github.com/fufankeji/fufan-chat-api/blob/main/server/db/base.py): sql persistence · Models and repositories for users, conversations, messages, knowledge bases, and files. · `server/db/base.py`
- [Conversation memory](https://github.com/fufankeji/fufan-chat-api/blob/main/server/memory/conversation_db_buffer_memory.py): session memory · Buffers and loads conversation history for chat flows. · `server/memory/conversation_db_buffer_memory.py`
- [Model API](https://github.com/fufankeji/fufan-chat-api/blob/main/server/minx_chat_openai.py): llm adapter · Provider-agnostic chat and embedding interfaces. · `server/minx_chat_openai.py`
- [Model workers](https://github.com/fufankeji/fufan-chat-api/blob/main/server/model_workers/base.py): backend adapters · Pluggable model backends for local and hosted inference. · `server/model_workers/base.py`
- [KB services](https://github.com/fufankeji/fufan-chat-api/blob/main/server/knowledge_base/kb_service/base.py): vector retrieval · Knowledge-base lifecycle, caching, and vector-store service selection. · `server/knowledge_base/kb_service/base.py`
- [KB cache](https://github.com/fufankeji/fufan-chat-api/blob/main/server/knowledge_base/kb_cache/base.py): metadata cache · Caches KB metadata and vector-store handles. · `server/knowledge_base/kb_cache/base.py`
- [Reranker](https://github.com/fufankeji/fufan-chat-api/blob/main/server/reranker/reranker.py): ranking stage · Refines retrieval results before generation. · `server/reranker/reranker.py`
- [Agent tools](https://github.com/fufankeji/fufan-chat-api/blob/main/server/agent/tools_select.py): tooling · Search tool integration and tool selection for agents. · `server/agent/tools_select.py`

### Offline pipelines

Ingestion, parsing, preprocessing, and evaluation tooling that support the runtime.

- [Loaders](https://github.com/fufankeji/fufan-chat-api/blob/main/document_loaders/interface.py): document intake · Parses source files into normalized documents. · `document_loaders/interface.py`
- [Parsing](https://github.com/fufankeji/fufan-chat-api/blob/main/data/parse/unstructured_processor.py): text preprocessing · Rule-based and unstructured document parsing pipeline. · `data/parse/unstructured_processor.py`
- [Text split](https://github.com/fufankeji/fufan-chat-api/blob/main/text_splitter/chinese_recursive_text_splitter.py): chunking · Splits parsed text into retrieval-friendly chunks. · `text_splitter/chinese_recursive_text_splitter.py`
- [RAG eval](https://github.com/fufankeji/fufan-chat-api/blob/main/fufanrag/pipeline/pipeline.py): evaluation pipeline · Offline retrieval, generation, and metric evaluation workflow. · `fufanrag/pipeline/pipeline.py`

### Supporting assets

Deployment, built frontend, docs, and playground examples.

- [Docker](https://github.com/fufankeji/fufan-chat-api/tree/main/docker/Dockerfile): deployment · Containerization and compose deployment assets. · `docker/Dockerfile`
- [Frontend dist](https://github.com/fufankeji/fufan-chat-api/blob/main/static/dist/index.html): built vue app · Prebuilt frontend bundle served from static/dist. · `static/dist/index.html`
- [Playground](https://github.com/fufankeji/fufan-chat-api/blob/main/playground/test_startup/startupV1.py): examples · Integration demos and experimental entry points. · `playground/test_startup/startupV1.py`

## Connections

- Startup → Server app: boots · Process startup enters the server application.
- Server app → API router: registers routes · The app exposes HTTP endpoints through the router.
- API router → Chat core: generic chat · Routes fan into the shared conversation entry point.
- API router → KB chat: kb qa · Routes KB question answering requests.
- API router → Search chat: web search · Routes internet-retrieval chat requests.
- API router → Agent chat: agent mode · Routes tool-using agent requests.
- API router → Reco chat: recommendation · Routes recommendation-oriented conversations.
- Chat core → Conversation memory: loads history · Shared chat flow combines current input with session context.
- Chat core → Database layer: stores state · Conversation state and business entities are persisted in MySQL.
- Chat core → Model API: streams response · Chat flow invokes model generation through provider-agnostic APIs.
- Model API → Model workers: uses backends · Chat and embedding APIs delegate to concrete model workers.
- KB chat → KB services: retrieves chunks · KB chat queries the vector-store service layer.
- KB chat → KB cache: reuses handles · KB metadata and store handles are cached for efficiency.
- KB chat → Reranker: refines results · Retrieved passages are reranked before prompt assembly.
- KB chat → Model API: generates answer · KB RAG answers are streamed from the model layer.
- Search chat → Agent tools: search tool · Web search is exposed as a tool-backed retrieval path.
- Search chat → Reranker: reranks web docs · Fetched content is refined before answering.
- Agent chat → Agent tools: selects tools · Agent flow chooses and executes tools.
- Agent chat → Model API: reasoning loop · The agent alternates tool outputs and model reasoning.
- Reco chat → Database layer: reads course data · Recommendation flow uses stored application data.
- Reco chat → Model API: responds · Recommendations are delivered through the same LLM layer.
- Loaders → Parsing: normalizes docs · Document loaders feed the parsing pipeline.
- Parsing → Text split: chunks text · Parsed documents are split for indexing.
- Text split → KB services: indexes KB · Chunked content becomes knowledge-base index material.
- RAG eval → KB services: tests retrieval · Offline evaluation measures retrieval quality.
- RAG eval → Model API: tests generation · Evaluation also exercises generation behavior.
- Docker → Startup: runs · Container deployment starts the backend entrypoint.
- Frontend dist → API router: calls api · The built frontend consumes the backend routes.
- Playground → Model API: experiments · Playground code demonstrates integration patterns.
