How vuhai2002/chatbot-ai works
Project Type and Purpose
• This is a backend API service (no UI/front end) that allows users to upload documents, converts them into vector embeddings, stores metadata and embeddings, and answers natural-language queries.
• It’s an open-source tool / microservice built on FastAPI, leveraging OpenAI, a vector DB (FAISS/ChromaDB), PostgreSQL, and Firebase Storage.
File Structure
• Top-level:
– main.py (entry point)
– app/ (application package)
– app/api/routes/ (FastAPI routers for upload, ask, list, delete)
– app/core/ (business logic: document_processor, embedding, qa_chain, storage)
– app/db/ (vector_store, models for metadata)
– app/utils/ (logger)
– config.py (.env integration)
• Key patterns:
– Layered architecture (API layer, core services layer, data layer)
– Clear separation of concerns
Extract Insights from README
• Components: FastAPI, OpenAI, Firebase Storage, FAISS/ChromaDB, PostgreSQL
• Data flow: Upload → vectorize → store → semantic search → QA
• Endpoints: /documents/upload, /ask, /documents, DELETE /documents/{id}
a. Main components:
1) Client (e.g., Postman or frontend)
2) FastAPI Service (app/main.py)
3) API Routers (app/api/routes)
4) Core Services (document_processor, embedding, qa_chain, storage)
5) Metadata DB (PostgreSQL)
6) Vector DB (FAISS or ChromaDB)
7) File Storage (Firebase Storage)
8) External API (OpenAI Embeddings & Completion)
b. Data flows / interactions:
• Client → HTTP POST /documents/upload → FastAPI Router
• Router calls document_processor → chunks & cleans document
• document_processor → OpenAI Embeddings → returns vectors
• Core/storage → save original file to Firebase Storage
• Core/storage → save metadata to PostgreSQL
• Core/storage → save embeddings to vector DB
• Client → HTTP POST /ask → Router → qa_chain
• qa_chain → vector DB (semantic similarity search) → top-k chunks
• qa_chain → OpenAI Completion API → answer synthesis → client
• Client → GET/DELETE /documents → Router → storage → PostgreSQL & (optionally) vector DB & Storage
c. Architectural patterns:
• Layered architecture (Presentation/API → Business/Core → Data/Storage)
• Adapter pattern for interchangeable vector DB (faiss vs chroma)
• Dependency injection via FastAPI for services and config
d. Key technologies: FastAPI, SQLAlchemy (PostgreSQL), FAISS/Chroma, Firebase SDK, OpenAI Python SDK
1) Sketch the user/client on the left.
5) Connect:
• Client → API Routers → Core Services
• Core → Firebase (store files)
• Core → OpenAI (embeddings & QA)
• Core → PostgreSQL (metadata)
• Core → Vector DB (vectors)
• Client ← Core (answers, document list, delete confirmations)