How jagan-16/mini_knowledge_base works
This is a document-grounded knowledge-base API with a Streamlit client: users upload documents, then ask questions with optional document/metadata scope and receive cited answers; conversation history can be revisited. The graph shows the API-owned upload and question workflows, extraction/classification and vector retrieval, persistence, and LLM integration. The tree and source index document a richer extraction stack and frontend, but their internal wiring is only included where references or excerpts support it; unsampled relationships are omitted.
Main entry points: document_service.py Document processing · query_service.py Query orchestration · main.py FastAPI endpoints · database_model.py Relational and vector DB
Components 26
API and client
- Streamlit UI
frontend/streamlit_app.py - API client
frontend/api_client.py - FastAPI endpoints
main.py
Document ingestion
- Document processing
services/extraction/document_service.py - File validation
services/validation/document_validation.py - File storage
services/extraction/file_storage_service.py - Document extraction
services/extraction - Metadata classification
services/metadata_services/metadata_classification.py - Metadata context
services/metadata_services/metadata_content_service.py - Chunking
services/extraction/dockling_chuncking_service.py - Embedding generation
services/extraction/embedding_service.py
Question answering
- Query orchestration
services/query_service.py - Filter mapping
metadata_filter_mapper.py - Vector retrieval
services/retrieval_service.py - Retrieval queries
repositories/retrieval_repository.py - Metadata query builder
repositories/metadata_filter_query_builder.py - Reranking
services/reranking_service.py - Prompt construction
services/prompt_service.py - LLM completion
services/llm_service.py - Groq client
services/groq_service.py
Conversation history
- Conversation service
services/conversation_service.py
Persistence
- Document repository
repositories/document_repository.py - Chunk repository
repositories/chunk_repository.py - Relational and vector DB
database_model.py
Other components
- User
- Groq service
Connections 31
- User → Streamlit UI: uses
- Streamlit UI → API client: requests
- API client → FastAPI endpoints: calls API
- User → FastAPI endpoints: uploads and asks
- FastAPI endpoints → Document processing: processes upload
main.py - FastAPI endpoints → Query orchestration: handles question
main.py - FastAPI endpoints → Document repository: lists documents
main.py - Document processing → File validation: validates file
services/extraction/document_service.py - Document processing → File storage: stores file
services/extraction/document_service.py - Document processing → Document extraction: extracts content
services/extraction/document_service.py - Document processing → Metadata classification: classifies metadata
services/extraction/document_service.py - Metadata classification → Metadata context: builds context
services/metadata_services/metadata_classification.py - Document processing → Chunking: creates chunks
services/extraction/document_service.py - Document processing → Embedding generation: embeds chunks
services/extraction/document_service.py - Document processing → Document repository: saves document
services/extraction/document_service.py - Document processing → Chunk repository: saves chunks
services/extraction/document_service.py - Query orchestration → Embedding generation: embeds question
services/query_service.py - Query orchestration → Filter mapping: maps filters
services/query_service.py - Query orchestration → Vector retrieval: retrieves matches
services/query_service.py - Vector retrieval → Retrieval queries: queries vectors
services/retrieval_service.py - Retrieval queries → Metadata query builder: builds metadata query
repositories/retrieval_repository.py - Retrieval queries → Relational and vector DB: reads chunks
repositories/retrieval_repository.py - Query orchestration → Reranking: reranks matches
services/query_service.py - Query orchestration → Prompt construction: builds prompt
services/query_service.py - Query orchestration → LLM completion: generates answer
services/query_service.py - LLM completion → Groq client: requests completion
services/llm_service.py - Groq client → Groq service: calls model
services/groq_service.py - Query orchestration → Conversation service: loads and saves exchange
services/query_service.py - Conversation service → Relational and vector DB: persists messages
- Document repository → Relational and vector DB: persists documents
repositories/document_repository.py - Chunk repository → Relational and vector DB: persists chunks
repositories/chunk_repository.py