# jagan-16/mini_knowledge_base architecture

> Architecture diagram of the GitHub repository jagan-16/mini_knowledge_base, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jagan-16/mini_knowledge_base
- Repository: https://github.com/jagan-16/mini_knowledge_base

## Overview

This is a document-grounded knowledge-base API with a Streamlit client: users upload documents, then ask questions with optional document/metadata scope and receive cited answers; conversation history can be revisited. The graph shows the API-owned upload and question workflows, extraction/classification and vector retrieval, persistence, and LLM integration. The tree and source index document a richer extraction stack and frontend, but their internal wiring is only included where references or excerpts support it; unsampled relationships are omitted.

Main entry points: [`services/extraction/document_service.py`](https://github.com/jagan-16/mini_knowledge_base/blob/main/services/extraction/document_service.py) (Document processing), [`services/query_service.py`](https://github.com/jagan-16/mini_knowledge_base/blob/main/services/query_service.py) (Query orchestration), [`main.py`](https://github.com/jagan-16/mini_knowledge_base/blob/main/main.py) (FastAPI endpoints), [`database_model.py`](https://github.com/jagan-16/mini_knowledge_base/blob/main/database_model.py) (Relational and vector DB)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["API and client"]
  node_streamlit["Streamlit UI<br/>[streamlit_app.py]"]
  node_api_client["API client<br/>[api_client.py]"]
  node_http_api["FastAPI endpoints<br/>[main.py]"]
end

subgraph group_ingest["Document ingestion"]
  node_document_service["Document processing"]
  node_file_validation["File validation"]
  node_file_storage["File storage"]
  node_extraction["Document extraction"]
  node_metadata["Metadata classification"]
  node_metadata_content["Metadata context"]
  node_chunking["Chunking"]
  node_embedding["Embedding generation"]
end

subgraph group_query["Question answering"]
  node_query_service["Query orchestration<br/>[query_service.py]"]
  node_filter_mapping["Filter mapping"]
  node_retrieval["Vector retrieval"]
  node_retrieval_repository["Retrieval queries"]
  node_filter_builder["Metadata query builder"]
  node_reranking["Reranking"]
  node_prompt["Prompt construction<br/>[prompt_service.py]"]
  node_llm["LLM completion<br/>[llm_service.py]"]
  node_groq["Groq client<br/>[groq_service.py]"]
end

subgraph group_conversations["Conversation history"]
  node_conversation_service["Conversation service"]
end

subgraph group_storage["Persistence"]
  node_document_repository["Document repository"]
  node_chunk_repository["Chunk repository"]
  node_database[("Relational and vector DB<br/>[database_model.py]")]
end

node_user(("User"))
node_external_model(("Groq service"))

node_user -->|"uses"| node_streamlit
node_streamlit -.->|"requests"| node_api_client
node_api_client -.->|"calls API"| node_http_api
node_user -.->|"uploads and asks"| node_http_api
node_http_api -->|"processes upload"| node_document_service
node_http_api -->|"handles question"| node_query_service
node_http_api -->|"lists documents"| node_document_repository
node_document_service -->|"validates file"| node_file_validation
node_document_service -->|"stores file"| node_file_storage
node_document_service -->|"extracts content"| node_extraction
node_document_service -->|"classifies metadata"| node_metadata
node_metadata -.->|"builds context"| node_metadata_content
node_document_service -->|"creates chunks"| node_chunking
node_document_service -->|"embeds chunks"| node_embedding
node_document_service -->|"saves document"| node_document_repository
node_document_service -->|"saves chunks"| node_chunk_repository
node_query_service -->|"embeds question"| node_embedding
node_query_service -->|"maps filters"| node_filter_mapping
node_query_service -->|"retrieves matches"| node_retrieval
node_retrieval -->|"queries vectors"| node_retrieval_repository
node_retrieval_repository -->|"builds metadata query"| node_filter_builder
node_retrieval_repository -->|"reads chunks"| node_database
node_query_service -->|"reranks matches"| node_reranking
node_query_service -->|"builds prompt"| node_prompt
node_query_service -->|"generates answer"| node_llm
node_llm -->|"requests completion"| node_groq
node_groq -->|"calls model"| node_external_model
node_query_service -->|"loads and saves exchange"| node_conversation_service
node_conversation_service -.->|"persists messages"| node_database
node_document_repository -.->|"persists documents"| node_database
node_chunk_repository -.->|"persists chunks"| node_database

click node_streamlit "https://github.com/jagan-16/mini_knowledge_base/blob/main/frontend/streamlit_app.py"
click node_api_client "https://github.com/jagan-16/mini_knowledge_base/blob/main/frontend/api_client.py"
click node_http_api "https://github.com/jagan-16/mini_knowledge_base/blob/main/main.py"
click node_document_service "https://github.com/jagan-16/mini_knowledge_base/blob/main/services/extraction/document_service.py"
click node_file_validation "https://github.com/jagan-16/mini_knowledge_base/blob/main/services/validation/document_validation.py"
click node_file_storage "https://github.com/jagan-16/mini_knowledge_base/blob/main/services/extraction/file_storage_service.py"
click node_extraction "https://github.com/jagan-16/mini_knowledge_base/tree/main/services/extraction"
click node_metadata "https://github.com/jagan-16/mini_knowledge_base/blob/main/services/metadata_services/metadata_classification.py"
click node_metadata_content "https://github.com/jagan-16/mini_knowledge_base/blob/main/services/metadata_services/metadata_content_service.py"
click node_chunking "https://github.com/jagan-16/mini_knowledge_base/blob/main/services/extraction/dockling_chuncking_service.py"
click node_embedding "https://github.com/jagan-16/mini_knowledge_base/blob/main/services/extraction/embedding_service.py"
click node_query_service "https://github.com/jagan-16/mini_knowledge_base/blob/main/services/query_service.py"
click node_filter_mapping "https://github.com/jagan-16/mini_knowledge_base/blob/main/metadata_filter_mapper.py"
click node_retrieval "https://github.com/jagan-16/mini_knowledge_base/blob/main/services/retrieval_service.py"
click node_retrieval_repository "https://github.com/jagan-16/mini_knowledge_base/blob/main/repositories/retrieval_repository.py"
click node_filter_builder "https://github.com/jagan-16/mini_knowledge_base/blob/main/repositories/metadata_filter_query_builder.py"
click node_reranking "https://github.com/jagan-16/mini_knowledge_base/blob/main/services/reranking_service.py"
click node_prompt "https://github.com/jagan-16/mini_knowledge_base/blob/main/services/prompt_service.py"
click node_llm "https://github.com/jagan-16/mini_knowledge_base/blob/main/services/llm_service.py"
click node_groq "https://github.com/jagan-16/mini_knowledge_base/blob/main/services/groq_service.py"
click node_conversation_service "https://github.com/jagan-16/mini_knowledge_base/blob/main/services/conversation_service.py"
click node_document_repository "https://github.com/jagan-16/mini_knowledge_base/blob/main/repositories/document_repository.py"
click node_chunk_repository "https://github.com/jagan-16/mini_knowledge_base/blob/main/repositories/chunk_repository.py"
click node_database "https://github.com/jagan-16/mini_knowledge_base/blob/main/database_model.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_streamlit,node_api_client,node_http_api,node_user toneBlue
class node_document_service,node_file_validation,node_file_storage,node_extraction,node_metadata,node_metadata_content,node_chunking,node_embedding toneAmber
class node_query_service,node_filter_mapping,node_retrieval,node_retrieval_repository,node_filter_builder,node_reranking,node_prompt,node_llm,node_groq toneMint
class node_conversation_service toneRose
class node_document_repository,node_chunk_repository,node_database,node_external_model toneIndigo
```

## Components

### API and client

- [Streamlit UI](https://github.com/jagan-16/mini_knowledge_base/blob/main/frontend/streamlit_app.py): `frontend/streamlit_app.py`
- [API client](https://github.com/jagan-16/mini_knowledge_base/blob/main/frontend/api_client.py): `frontend/api_client.py`
- [FastAPI endpoints](https://github.com/jagan-16/mini_knowledge_base/blob/main/main.py): `main.py`

### Document ingestion

- [Document processing](https://github.com/jagan-16/mini_knowledge_base/blob/main/services/extraction/document_service.py): `services/extraction/document_service.py`
- [File validation](https://github.com/jagan-16/mini_knowledge_base/blob/main/services/validation/document_validation.py): `services/validation/document_validation.py`
- [File storage](https://github.com/jagan-16/mini_knowledge_base/blob/main/services/extraction/file_storage_service.py): `services/extraction/file_storage_service.py`
- [Document extraction](https://github.com/jagan-16/mini_knowledge_base/tree/main/services/extraction): `services/extraction`
- [Metadata classification](https://github.com/jagan-16/mini_knowledge_base/blob/main/services/metadata_services/metadata_classification.py): `services/metadata_services/metadata_classification.py`
- [Metadata context](https://github.com/jagan-16/mini_knowledge_base/blob/main/services/metadata_services/metadata_content_service.py): `services/metadata_services/metadata_content_service.py`
- [Chunking](https://github.com/jagan-16/mini_knowledge_base/blob/main/services/extraction/dockling_chuncking_service.py): `services/extraction/dockling_chuncking_service.py`
- [Embedding generation](https://github.com/jagan-16/mini_knowledge_base/blob/main/services/extraction/embedding_service.py): `services/extraction/embedding_service.py`

### Question answering

- [Query orchestration](https://github.com/jagan-16/mini_knowledge_base/blob/main/services/query_service.py): `services/query_service.py`
- [Filter mapping](https://github.com/jagan-16/mini_knowledge_base/blob/main/metadata_filter_mapper.py): `metadata_filter_mapper.py`
- [Vector retrieval](https://github.com/jagan-16/mini_knowledge_base/blob/main/services/retrieval_service.py): `services/retrieval_service.py`
- [Retrieval queries](https://github.com/jagan-16/mini_knowledge_base/blob/main/repositories/retrieval_repository.py): `repositories/retrieval_repository.py`
- [Metadata query builder](https://github.com/jagan-16/mini_knowledge_base/blob/main/repositories/metadata_filter_query_builder.py): `repositories/metadata_filter_query_builder.py`
- [Reranking](https://github.com/jagan-16/mini_knowledge_base/blob/main/services/reranking_service.py): `services/reranking_service.py`
- [Prompt construction](https://github.com/jagan-16/mini_knowledge_base/blob/main/services/prompt_service.py): `services/prompt_service.py`
- [LLM completion](https://github.com/jagan-16/mini_knowledge_base/blob/main/services/llm_service.py): `services/llm_service.py`
- [Groq client](https://github.com/jagan-16/mini_knowledge_base/blob/main/services/groq_service.py): `services/groq_service.py`

### Conversation history

- [Conversation service](https://github.com/jagan-16/mini_knowledge_base/blob/main/services/conversation_service.py): `services/conversation_service.py`

### Persistence

- [Document repository](https://github.com/jagan-16/mini_knowledge_base/blob/main/repositories/document_repository.py): `repositories/document_repository.py`
- [Chunk repository](https://github.com/jagan-16/mini_knowledge_base/blob/main/repositories/chunk_repository.py): `repositories/chunk_repository.py`
- [Relational and vector DB](https://github.com/jagan-16/mini_knowledge_base/blob/main/database_model.py): `database_model.py`

### Other components

- **User**
- **Groq service**

## Connections

- User → Streamlit UI: uses
- Streamlit UI → API client: requests
- API client → FastAPI endpoints: calls API
- User → FastAPI endpoints: uploads and asks
- FastAPI endpoints → Document processing: processes upload (evidence: `main.py`)
- FastAPI endpoints → Query orchestration: handles question (evidence: `main.py`)
- FastAPI endpoints → Document repository: lists documents (evidence: `main.py`)
- Document processing → File validation: validates file (evidence: `services/extraction/document_service.py`)
- Document processing → File storage: stores file (evidence: `services/extraction/document_service.py`)
- Document processing → Document extraction: extracts content (evidence: `services/extraction/document_service.py`)
- Document processing → Metadata classification: classifies metadata (evidence: `services/extraction/document_service.py`)
- Metadata classification → Metadata context: builds context (evidence: `services/metadata_services/metadata_classification.py`)
- Document processing → Chunking: creates chunks (evidence: `services/extraction/document_service.py`)
- Document processing → Embedding generation: embeds chunks (evidence: `services/extraction/document_service.py`)
- Document processing → Document repository: saves document (evidence: `services/extraction/document_service.py`)
- Document processing → Chunk repository: saves chunks (evidence: `services/extraction/document_service.py`)
- Query orchestration → Embedding generation: embeds question (evidence: `services/query_service.py`)
- Query orchestration → Filter mapping: maps filters (evidence: `services/query_service.py`)
- Query orchestration → Vector retrieval: retrieves matches (evidence: `services/query_service.py`)
- Vector retrieval → Retrieval queries: queries vectors (evidence: `services/retrieval_service.py`)
- Retrieval queries → Metadata query builder: builds metadata query (evidence: `repositories/retrieval_repository.py`)
- Retrieval queries → Relational and vector DB: reads chunks (evidence: `repositories/retrieval_repository.py`)
- Query orchestration → Reranking: reranks matches (evidence: `services/query_service.py`)
- Query orchestration → Prompt construction: builds prompt (evidence: `services/query_service.py`)
- Query orchestration → LLM completion: generates answer (evidence: `services/query_service.py`)
- LLM completion → Groq client: requests completion (evidence: `services/llm_service.py`)
- Groq client → Groq service: calls model (evidence: `services/groq_service.py`)
- Query orchestration → Conversation service: loads and saves exchange (evidence: `services/query_service.py`)
- Conversation service → Relational and vector DB: persists messages
- Document repository → Relational and vector DB: persists documents (evidence: `repositories/document_repository.py`)
- Chunk repository → Relational and vector DB: persists chunks (evidence: `repositories/chunk_repository.py`)
