How muntasir2179/knowly works
Project Type & Purpose
• Knowly is a self-contained, offline conversational AI application (full-stack, but all components run locally).
• It offers three user workflows:
• Normal chat (text or voice → LLM)
• RAG (document upload → embed → vector DB → LLM)
• Multimodal RAG (image upload → image processing → LLM)
File-Structure Analysis
Top-level files & directories:
• app.py – Streamlit entry point (UI + controller)
• audio_transcribe.py – audio → text module (librosa)
• image_handler.py – image OCR/caption module
• text_processor.py – document parsing & chunking (pypdfium2, unstructured)
• chromadb_operations.py – vector DB interface (chromadb)
• llm_response.py – orchestrates prompts → LLM (ctransformers/llama-cpp-python via Ollama)
• prompt_templates.py – prompt engineering
• get_models.py + modelfiles/ – model download & storage
• utils.py – shared utilities
• config.yaml – customizable settings
• style.css – UI styling
• run.sh, requirements.txt – setup scripts
Key README Insights
• Uses Streamlit for UI, Ollama for local model serving, chromadb for embeddings, InstructorEmbedding/sentence-transformers.
• Emphasizes offline operation and model-switching.
Identifying Main Components
A. Frontend/UI Layer
• Streamlit app (app.py + style.css) presenting:
• File upload widget (PDF, image)
• Model-selection dropdown
B. Controller/Router
• app.py logic routes user actions to appropriate service (audio_transcribe, text_processor, image_handler)
C. Processing Services
1. Audio Service (audio_transcribe.py → librosa → speech-to-text)
2. Document Service (text_processor.py → pypdfium2/unstructured → chunking)
3. Image Service (image_handler.py → OCR/caption)
D. Embedding & Retrieval
• Embedding Models (InstructorEmbedding or sentence-transformers)
• chromadb_operations.py interacting with local Chromadb vector store
E. LLM Service
• llm_response.py orchestrating:
• Prompt_templates
• Calls out to local Ollama/ctransformers via llama-cpp-python
F. Data Stores & Model Files
• modelfiles/ (multiple Modelfile formats)
• Vector DB (Chromadb)
• Local cache (embeddings, conversation history)
G. Configuration & Utilities
• config.yaml for thresholds, model paths
• utils.py for file I/O, logging, common routines
Defining Relationships & Data Flows
• User → Streamlit UI → Controller
• Controller → (branch)
• Normal Chat → llm_response → LLM → UI
• RAG → Document Service → Embedding Model → Vector DB → Retrieval → llm_response → UI
• Multimodal RAG → Image Service → llm_response → UI
• All calls to LLM go through the same llm_response layer, which loads models from modelfiles and posts to Ollama.
• config.yaml feeds into every service for toggles (e.g., chunk size, model choice).
Architectural Patterns & Principles
• Layered architecture: UI → Controllers → Services → Data
• Separation of concerns: each modality is a self-contained module
• Dependency inversion: high-level llm_response module relies on abstracted model interfaces
• Single-process monolith with modular Python packages