How mehedihasan-75/ai-document-companion works
1. Overall shape
This repo is a FastAPI backend for an “AI Document Companion” system: users authenticate, upload documents, trigger ingestion, and then ask questions in persisted conversations. The architecture is cleanly layered into routes → dependencies → services → models/schemas → DB/vector stores.
2. HTTP boundary
src/main.py builds the FastAPI app and wires routers from src/routes/. The route layer is split by domain: auth_routes.py, file_routes.py, process_routes.py, and conversation_routes.py. The API includes both standard JSON endpoints and a streaming SSE endpoint for chat responses.
3. Authentication and request scoping
Authentication is JWT-based. src/dependencies/auth.py resolves the current user from the bearer token, and protected routes depend on that user object. This is the main multi-tenant boundary: nearly every file/document/conversation/query path is scoped by user_id.
4. Document upload and file handling
file_routes.py and file_service.py handle upload validation, MIME/type allowlisting, size checks, and file persistence under an upload directory. Uploaded files are represented in the database as Document records, with later processing separated from the initial upload.
5. Background ingestion pipeline
Processing is intentionally decoupled from upload. process_routes.py triggers process_service.py, which dispatches ingestion as a background task and writes status artifacts that clients can poll. The ingestion path runs through ingestion_service.py, unstructured_service.py, chunk_service.py, and document_service.py.
6. Multimodal parsing and chunking
The ingestion pipeline uses Unstructured in hi-res mode to parse PDFs, DOCX, PPTX, spreadsheets, CSV, HTML, JSON, markdown, and text into typed elements. Those elements are then chunked by title/heading structure rather than raw character splitting, preserving tables, lists, and section boundaries.
7. Dual-store retrieval architecture
The core RAG design uses two stores:
- ChromaDB for vector retrieval of summaries of chunks
- SQLite docstore for the original chunk content
vector_service.py manages both stores, while retrieval_service.py and rag_chain.py resolve retrieved summary hits back to originals before generation. This is a key architectural boundary: retrieval is summary-based, generation is original-text-based.
8. LLM and summarization services
llm_service.py centralizes model instances, and the repo assumes local Ollama as the LLM runtime. Different models are used for different jobs: text summarization/QA and vision summarization for images. That makes ingestion and answer generation dependent on local model availability rather than external SaaS APIs.
9. Conversation memory and answer generation
Conversations are first-class database entities in conversation.py and message.py. conversation_service.py loads a bounded history window from the DB, persists each user/assistant turn, and stores source metadata per assistant message. The RAG chain is composed in rag_chain.py, and streaming_service.py turns chain events into SSE status/delta/completion events.
10. Data model and persistence
SQLAlchemy models under src/models/ define the durable state: User, Document, Chunk, Conversation, and Message. src/db/ contains the engine/session setup and shared base/mixin patterns. This repo is built to work with SQLite for local development and PostgreSQL-compatible configuration for production.
11. Validation, schemas, and error boundaries
src/schemas/ defines request/response contracts for auth, file, and conversation flows. src/core/exceptions.py provides the service-level error boundary so internal failures are translated into controlled HTTP responses, rather than leaking implementation details.
12. Supporting layers and configuration
src/config/ centralizes environment settings, constants, file-type allowlists, and prompt templates. src/core/logger.py handles logging. The repo also includes docs for auth, DB, FastAPI, frontend SSE integration, and the full RAG pipeline, which are useful references when drawing the architecture diagram.
13. External/runtime dependencies that matter architecturally
The main external pieces are Ollama for local LLM inference, Unstructured for layout-aware document parsing, ChromaDB for vector search, and SQLAlchemy for relational persistence. These are not just libraries; they define the system’s runtime topology: file upload and DB-backed metadata are local, while LLM inference and vector retrieval sit in separate service boundaries.
14. Primary end-to-end data flow
The main flow is: upload file → store metadata → background ingestion → parse/chunk/summarize → write summary vectors to Chroma + originals to SQLite docstore → user asks question in a conversation → retrieve by vector search → resolve originals → build prompt with history and citations → stream answer via SSE → persist message and sources.
Main entry points: base.py Base · index.py Router Index · ingestion_service.py Ingestion · conversation_service.py Conversation Service
Components 29
HTTP API
FastAPI app and request handlers
- App · Builds the FastAPI application and wires routers
src/main.py - Router Index · Aggregates route modules into the app
src/routes/index.py - Auth Routes · Login and token-related endpoints
src/routes/auth_routes.py - File Routes · Document upload and file lifecycle endpoints
src/routes/file_routes.py - Process Routes · Endpoints for triggering ingestion and processing
src/routes/process_routes.py - Chat Routes · Conversation and streaming question-answer endpoints
src/routes/conversation_routes.py
Application Services
Business logic, ingestion, retrieval, and response orchestration
- File Service · Validates, stores, and registers uploaded files
src/services/file_service.py - Process Service · Starts background ingestion and tracks processing status
src/services/process_service.py - Ingestion · Coordinates parsing, chunking, and persistence of document content
src/services/ingestion_service.py - Parser · Uses Unstructured to extract typed elements from files
src/services/unstructured_service.py - Chunker · Builds structure-aware chunks from parsed elements
src/services/chunk_service.py - Document Service · Stores document and chunk metadata in the relational store
src/services/document_service.py - Vector Store · Manages Chroma vector search and SQLite docstore access
src/services/vector_service.py - Retriever · Resolves summary hits back to original chunks for answering
src/services/retrieval_service.py - RAG Chain · Builds prompts, history, retrieval context, and citations
src/services/rag_chain.py - Streaming · Converts chain events into streaming response events
src/services/streaming_service.py - Conversation Service · Persists turns and loads bounded conversation history
src/services/conversation_service.py - LLM Service · Centralizes local Ollama model access for summarization and QA
src/services/llm_service.py
Data Layer
ORM models and database/session infrastructure
- User · Authenticated account identity
src/models/user.py - Document · Uploaded document metadata and processing state
src/models/document.py - Chunk · Persisted chunk metadata and content references
src/models/chunk.py - Conversation · Persistent chat thread for a user
src/models/conversation.py - Message · Stored user and assistant turns with source metadata
src/models/message.py - Session · Engine and session configuration for relational persistence
src/db/session.py - Base · Shared declarative base and model foundation
src/db/base.py
Support
Config, dependencies, and shared core utilities
- Auth Dep · Resolves the current user from bearer token
src/dependencies/auth.py - DB Dep · Provides database sessions to application layers
src/dependencies/db.py - Config · Environment settings, constants, file types, and prompts
src/config/environment.py - Core · Central exception handling and logging
src/core/exceptions.py
Connections 41
- App → Router Index: includes · Registers the API routers
- Router Index → Auth Routes: mounts · Exposes authentication endpoints
- Router Index → File Routes: mounts · Exposes upload endpoints
- Router Index → Process Routes: mounts · Exposes ingestion trigger endpoints
- Router Index → Chat Routes: mounts · Exposes chat endpoints
- Auth Routes → DB Dep: uses · Requires database access
- File Routes → Auth Dep: scopes · Requires authenticated user context
- File Routes → File Service: calls · Delegates file validation and persistence
- Process Routes → Auth Dep: scopes · Requires authenticated user context
- Process Routes → Process Service: calls · Starts ingestion jobs
- Chat Routes → Auth Dep: scopes · Requires authenticated user context
- Chat Routes → Conversation Service: calls · Loads and saves conversation turns
- Chat Routes → Streaming: streams via · Serves SSE chat responses
- File Service → Document: writes · Creates document records
- Process Service → Ingestion: dispatches · Hands off work to the ingestion pipeline
- Ingestion → Parser: parses with · Extracts typed elements from files
- Ingestion → Chunker: chunks with · Builds structure-aware chunks
- Ingestion → Document Service: persists via · Stores document and chunk metadata
- Ingestion → Vector Store: indexes via · Writes summaries to vector store and originals to docstore
- Chunker → Chunk: produces · Creates chunk entities
- Document Service → Document: updates · Maintains document state
- Document Service → Chunk: stores · Persists chunk records
- Vector Store → LLM Service: summarizes with · Uses local models for summaries and QA
- Retriever → Vector Store: queries · Performs vector search and docstore lookup
- RAG Chain → Retriever: retrieves · Fetches relevant context
- RAG Chain → Conversation Service: loads history · Uses prior turns as chat context
- RAG Chain → LLM Service: generates with · Invokes local LLM for final answers
- Streaming → RAG Chain: consumes · Turns chain output into SSE events
- Conversation Service → Conversation: writes · Persists threads
- Conversation Service → Message: writes · Persists turns and source metadata
- Auth Dep → User: resolves · Converts token claims into a user identity
- Session → Base: builds on · Session setup uses the shared ORM base
- User → Base: extends · ORM entity
- Document → Base: extends · ORM entity
- Chunk → Base: extends · ORM entity
- Conversation → Base: extends · ORM entity
- Message → Base: extends · ORM entity
- File Service → DB Dep: uses · Requires a database session
- Process Service → DB Dep: uses · Persists processing status
- Conversation Service → DB Dep: uses · Loads and stores chat history
- Core → Config: relies on · Shared errors and logging follow app settings