How whyhow-ai/knowledge-table works
Main Components
• Frontend: The UI built with TypeScript/React (using Vite and Bun) that users interact with. It includes components for displaying tables, handling document uploads, and interacting with various interface elements (like file drop areas, table controls, and menu buttons).
• Backend: A Python-based API (using FastAPI/uvicorn) that exposes endpoints for document extraction, query processing, and graph operations. Its structure is organized into clearly defined submodules (API endpoints, models, services, schemas, and core configuration).
• Services: A collection of services within the backend responsible for:
• Document Extraction (document endpoints and services)
• Graph and Query processing (graph endpoints and query endpoints)
• Integrations with external providers such as OpenAI for language and embedding services, as well as potential integration with the Unstructured API.
• Vector Database integrations: The backend supports multiple vector databases (Milvus, Qdrant, etc.) via specific service factories.
• External Integrations: External AI providers (OpenAI initially) for extracting and embedding, as well as optional integrations like the Unstructured API. Additionally, integration with vector databases (Milvus, Qdrant) plays a key role.
• Data Stores: Although not a classical database layer from relational perspectives, the project integrates with vector databases and is planning support for traditional databases like PostgreSQL or MongoDB in the roadmap.
Architectural Patterns & Key Principles
• Separation of Concerns: Clearly separate frontend presentation, API endpoint handling, business logic/services, and external integrations.
• Modularity and Extensibility: Use factories for embedding, LLM, loaders, and vector DB services, indicating design patterns that support plug-and-play components.
• API-Centric Design: The backend is a REST API server (using FastAPI/uvicorn) organized with versioning (v1) and clearly defined endpoint modules.
• Containerization & Environment Configuration: Multiple Dockerfiles and docker-compose configurations suggest an architecture optimized for containerized deployment.
Final Recommendations to the Principal Software Engineer
• How the system facilitates RAG pipelines by converting unstructured documents into structured data tables and graphs, and highlight the integration flexibility (e.g., support for multiple vector DBs, future integrations with additional LLM providers).