# muntasir2179/knowly architecture

> Architecture diagram of the GitHub repository muntasir2179/knowly, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/muntasir2179/knowly
- Repository: https://github.com/muntasir2179/knowly

## Overview

##### Project Type & Purpose
- Knowly is a self-contained, offline conversational AI application (full-stack, but all components run locally).
- It offers three user workflows:
- Normal chat (text or voice → LLM)
- RAG (document upload → embed → vector DB → LLM)
- Multimodal RAG (image upload → image processing → LLM)

##### File-Structure Analysis
Top-level files & directories:
- app.py – Streamlit entry point (UI + controller)
- audio_transcribe.py – audio → text module (librosa)
- image_handler.py – image OCR/caption module
- text_processor.py – document parsing & chunking (pypdfium2, unstructured)
- chromadb_operations.py – vector DB interface (chromadb)
- llm_response.py – orchestrates prompts → LLM (ctransformers/llama-cpp-python via Ollama)
- prompt_templates.py – prompt engineering
- get_models.py + modelfiles/ – model download & storage
- utils.py – shared utilities
- config.yaml – customizable settings
- style.css – UI styling
- run.sh, requirements.txt – setup scripts

##### Key README Insights
- Uses Streamlit for UI, Ollama for local model serving, chromadb for embeddings, InstructorEmbedding/sentence-transformers.
- Emphasizes offline operation and model-switching.

##### Identifying Main Components
A. Frontend/UI Layer
- Streamlit app (app.py + style.css) presenting:
- File upload widget (PDF, image)
- Model-selection dropdown
B. Controller/Router
- app.py logic routes user actions to appropriate service (audio_transcribe, text_processor, image_handler)
C. Processing Services
1. Audio Service (audio_transcribe.py → librosa → speech-to-text)
2. Document Service (text_processor.py → pypdfium2/unstructured → chunking)
3. Image Service (image_handler.py → OCR/caption)
D. Embedding & Retrieval
- Embedding Models (InstructorEmbedding or sentence-transformers)
- chromadb_operations.py interacting with local Chromadb vector store
E. LLM Service
- llm_response.py orchestrating:
- Prompt_templates
- Calls out to local Ollama/ctransformers via llama-cpp-python
F. Data Stores & Model Files
- modelfiles/ (multiple Modelfile formats)
- Vector DB (Chromadb)
- Local cache (embeddings, conversation history)
G. Configuration & Utilities
- config.yaml for thresholds, model paths
- utils.py for file I/O, logging, common routines

##### Defining Relationships & Data Flows
- User → Streamlit UI → Controller
- Controller → (branch)
- Normal Chat → llm_response → LLM → UI
- RAG → Document Service → Embedding Model → Vector DB → Retrieval → llm_response → UI
- Multimodal RAG → Image Service → llm_response → UI
- All calls to LLM go through the same llm_response layer, which loads models from modelfiles and posts to Ollama.
- config.yaml feeds into every service for toggles (e.g., chunk size, model choice).

##### Architectural Patterns & Principles
- Layered architecture: UI → Controllers → Services → Data
- Separation of concerns: each modality is a self-contained module
- Dependency inversion: high-level llm_response module relies on abstracted model interfaces
- Single-process monolith with modular Python packages

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Actors
    User["User"]:::actor

    %% UI Layer
    subgraph "UI Layer"
        UI["Streamlit UI"]:::ui
    end

    %% Controller
    subgraph "Controller"
        Router["Controller/Router"]:::service
    end

    %% Processing Services
    subgraph "Processing Services"
        Audio["AudioTranscribe Service"]:::service
        Document["Document Service"]:::service
        Image["Image Service"]:::service
    end

    %% Embedding & Retrieval
    subgraph "Embedding & Retrieval"
        ModelLoader["Embedding Model Loader"]:::service
        EmbInterface["Chromadb Operations"]:::service
        VectorDB["Chromadb Vector Store"]:::db
    end

    %% LLM Service
    subgraph "LLM Service"
        Prompt["Prompt Templates"]:::service
        LLM["LLM Inference via Ollama"]:::service
    end

    %% Data Stores & Utils
    subgraph "Data & Config"
        ModelFiles["Model Files Storage"]:::storage
        Config["Configuration"]:::config
        Utils["Shared Utilities"]:::util
        Setup["Setup Scripts & Dependencies"]:::util
    end

    %% Core Flows
    User --> UI
    UI --> Router

    Router -->|"Normal Chat"| LLM
    Router -->|"RAG Document"| Document
    Router -->|"Multimodal RAG"| Image

    Document --> ModelLoader
    ModelLoader --> EmbInterface
    EmbInterface --> VectorDB
    EmbInterface --> LLM

    Image --> LLM

    LLM --> UI

    %% Config & Utils Dependencies
    Config -.-> UI
    Config -.-> Router
    Config -.-> Audio
    Config -.-> Document
    Config -.-> Image
    Config -.-> ModelLoader
    Config -.-> EmbInterface
    Config -.-> LLM

    Utils -.-> Audio
    Utils -.-> Document
    Utils -.-> Image
    Utils -.-> EmbInterface
    Utils -.-> LLM

    %% Model Files & Setup
    ModelFiles -.-> ModelLoader
    ModelFiles -.-> LLM
    Setup -.-> ModelLoader
    Setup -.-> EmbInterface

    %% Click Events
    click UI "https://github.com/muntasir2179/knowly/blob/ollama_main_v2/app.py"
    click Router "https://github.com/muntasir2179/knowly/blob/ollama_main_v2/app.py"
    click Audio "https://github.com/muntasir2179/knowly/blob/ollama_main_v2/audio_transcribe.py"
    click Document "https://github.com/muntasir2179/knowly/blob/ollama_main_v2/text_processor.py"
    click Image "https://github.com/muntasir2179/knowly/blob/ollama_main_v2/image_handler.py"
    click EmbInterface "https://github.com/muntasir2179/knowly/blob/ollama_main_v2/chromadb_operations.py"
    click ModelLoader "https://github.com/muntasir2179/knowly/blob/ollama_main_v2/get_models.py"
    click LLM "https://github.com/muntasir2179/knowly/blob/ollama_main_v2/llm_response.py"
    click Prompt "https://github.com/muntasir2179/knowly/blob/ollama_main_v2/prompt_templates.py"
    click ModelFiles "https://github.com/muntasir2179/knowly/tree/ollama_main_v2/modelfiles/"
    click Config "https://github.com/muntasir2179/knowly/blob/ollama_main_v2/config.yaml"
    click Utils "https://github.com/muntasir2179/knowly/blob/ollama_main_v2/utils.py"
    click Setup "https://github.com/muntasir2179/knowly/blob/ollama_main_v2/run.sh"

    %% Styles
    classDef ui fill:#CCE5FF,stroke:#000,stroke-width:1px;
    classDef service fill:#B3D7FF,stroke:#000,stroke-width:1px;
    classDef db fill:#C2F0C2,stroke:#000,stroke-width:1px;
    classDef storage fill:#FFF2CC,stroke:#000,stroke-width:1px;
    classDef config fill:#E5CCFF,stroke:#000,stroke-width:1px;
    classDef util fill:#F0F0F0,stroke:#000,stroke-width:1px;
    classDef actor fill:#FFD1DC,stroke:#000,stroke-width:1px;
```
