How binayakkoirala/kanun_gpt works
Project Type and Purpose
• Type: Full-stack AI-powered Q&A web application
• Purpose: Provide an interface for querying the Constitution of Nepal using a local Hugging Face–based chat model and a Chroma vector database of embeddings.
File Structure
• app.py – Flask web application that handles user requests, performs embedding lookups, invokes the language model, and returns responses.
• ingest.py – Preprocessing script: downloads/splits Constitution text, generates embeddings, and stores them in Chroma DB (stored/nepdata/chroma.sqlite3).
• stored/nepdata/chroma.sqlite3 – Local vector database storing document embeddings.
• templates/index.html – Single-page front-end for user input and display of answers.
• README.md – Describes the use of the “neural-chat-7b-v3-1.Q4_K_M.gguf” GGUF model and Chroma DB.
README for Additional Insights
• Model: TheBloke’s “neural-chat-7B-v3-1-GGUF” loaded locally.
• Database: Chroma vector store used to index Constitution segments.
• Framework: Flask is used to serve the UI and API endpoints.
Main Components and Their Interactions
A. Front-end (Browser)
– Index.html
– Sends user questions via HTTP POST to Flask.
B. Flask Back-end (app.py)
– HTTP API endpoint (/ask)
– Query embedding engine
– Context retrieval from Chroma
– Language model inference
– Response formatting
C. Vector Database (Chroma DB)
– SQLite file storing embeddings and metadata.
D. Embedding & Ingestion Pipeline (ingest.py)
– Constitution PDF downloader/splitter
– Embedding generation
– Data loading into Chroma DB
E. Language Model
– Locally hosted GGUF file loaded via Hugging Face-compatible loader (e.g., llama.cpp, transformers).
1. ingest.py → Chroma DB (populate embeddings)
2. Browser → Flask (/ask)
3. Flask → Embedding function (encode user query)
4. Embedding function → Chroma DB (retrieve top-k similar segments)
5. Flask → Language Model (send retrieved context + user query)
6. Language Model → Flask (generated answer)
7. Flask → Browser (HTTP response with answer)
Architectural Patterns and Principles
• Separation of concerns: ingestion vs. online serving
• Stateless web server: each request independently retrieves context, infers, and responds
• Local resource hosting: model and DB are file-based, no external cloud services
• Layered approach: Presentation → API → Retrieval → Inference → Persistence