How prakhar-pandey28/turium_ai_project works
Project Type and Purpose
• This is a full-stack, production-style Retrieval-Augmented Generation (RAG) web application.
• Purpose: let users ingest short text notes or URLs, store them in a vectorized SQLite store, and ask questions against that content via a FastAPI backend and a React+Vite frontend.
File Structure
• Top level:
• README.md, .gitignore
• backend/ – Python FastAPI service
• frontend/vite-project/ – React application powered by Vite
• backend/ contains:
• main.py – FastAPI app entrypoint
• chunker.py, embeddings.py, html_parser.py, llm.py, vector.py, db.py – core pipeline modules
• Dockerfile, render.yaml, requirements.txt, runtime.txt – deployment/config
• frontend/vite-project/ contains the standard Vite + React scaffold:
• src/ with App.jsx, main.jsx, style sheets, assets
• index.html, package.json, vite.config.js
• No other microservices—backend monolith handles both ingestion and query.
README Insights
• Ingestion pipeline using Jina AI Reader, chunking strategy, and embeddings.
• RAG pipeline using cosine similarity over JSON-serialized vectors in SQLite and Groq LLM API.
• Deployment on Render (backend) and Vercel (frontend).
Main Components
a. User / Frontend (React + Vite)
b. Backend (FastAPI monolith)
· Ingestion sub-components: URL fetcher (Jina AI Reader), chunker, embedding client, vector store interface
· Query sub-components: embedding client, vector search, LLM client
c. Data Store: SQLite (vectors stored as JSON blobs plus metadata)
d. External Services:
· Jina AI Reader (url → markdown)
· Jina AI Embeddings API (text chunks → embeddings)
· Groq LLM API (question + context → answer)
Relationships and Data Flow
• Frontend ↔ Backend via HTTP/HTTPS (POST /ingest, GET /items, POST /query)
• Ingestion Flow:
1. Frontend posts text or URL → Backend API
2. For URL: HTTP fetch → Jina AI Reader → clean markdown → chunker
3. For text: chunker directly
4. chunker → embeddings API → vectors → store in SQLite
• Query Flow:
1. Frontend posts question → Backend API
2. Backend sends question to embeddings API → query vector
3. In-memory cosine similarity search over SQLite vectors → top-k chunks
4. top-k chunks + question → Groq API → answer → Backend → Frontend
Architectural Patterns & Principles
• Layered separation:
• Presentation (React frontend)
• API Layer (FastAPI)
• Pipeline Layer (ingestion vs. query logic)
• Persistence Layer (SQLite)
• Monolith backend with clear module boundaries (chunker, embeddings, vector search, LLM)
• External dependencies abstracted behind thin client modules (embeddings, HTML parsing, LLM)
• Configuration as code: Dockerfile, render.yaml, environment variable examples