How lukablaskovic/edu_bot works
Project Type and Purpose
• EduBot is a full-stack, cloud-connected RAG (Retrieval-Augmented Generation) chatbot built in Python.
• Front-end: Streamlit multi-page UI for chat, user profile, and learning resources.
• Back-end: Modular RAG pipeline (RAPTOR, SQL-RAG, Web-scraper, Direct LLM) using LlamaIndex.
• Data stores: Chroma DB for embeddings, SQLite for structured data, file store for uploaded docs.
• External integrations: OpenAI/Anthropic/Ollama APIs, Google OAuth, University news website.
File Structure Insights
• Top-level scripts: 🤖EduBot.py (Streamlit launcher), chatbot.py, intent_agent.py, modules/*.py
• Front-end pages: pages/👤Korisnički_profil.py, pages/📚Datoteke.py
• Modules directory:
• raptor_module.py – builds & queries hierarchical RAPTOR index
• sqlrag_module.py – translates text→SQL & queries SQLite
• web_scraper_module.py – scrapes news via requests + BeautifulSoup
• db/database.db – local SQLite for academic data
• Data directories:
• chroma_db/ – persistent embedding store (Chroma)
• uploaded_files/ – user-uploaded learning resources
• Config & credentials: .env_template, openai_key.py, google_credentials.json (external), settings.py
• Prompts: prompts/ → tool descriptions for each RAG module
• Tests & notebooks: test_chatbot.py, evaluation/*.ipynb
Key Architectural Patterns
• Modular “Query Engine” plugin pattern: Router Query Engine dispatches to specialized engines.
• RAG pipeline: Retrieval (RAPTOR tree / SQL / Web-scrape) → generation (LLM).
• Multi-modal ingestion: PDF/text/slides → embeddings → hierarchical index.
• External service orchestration: OAuth for user info; REST calls to news site; LLM API calls.
Main Components & Their Relationships
A. User & UI Layer
• User’s browser runs Streamlit front-end (🤖EduBot.py) → pages & sidebar.
• Google OAuth module provides user identity.
B. Router & Controller
• User query → Router Query Engine (LLMSingleSelector)
• Chooses one of: RAPTOR engine, SQL-RAG engine, Web-scraper engine, or default LLM.
C. RAG Modules (each a separate component)
1. RAPTOR Module
– Ingests uploaded_files → llamaindex RAPTOR Pack → builds tree in chroma_db
– On query: llamaindex RAPTOR Retrieval → returns relevant text chunks
2. SQL-RAG Module
– Query text → text-to-SQL transformer → runs against modules/db/database.db → returns rows
3. Web-scraper Module
– Fetches & parses University news site → returns article text
4. Direct LLM Module
– For chit-chat or fallback queries → direct call to selected LLM
D. LLM Generation
• Once retrieval result available → call chosen LLM (OpenAI, Anthropic, Ollama) → generate answer.
E. Response & Logging
• Generated answer → Streamlit UI
• Actions & errors logged to app.log
• Module responsibilities (e.g., “raptor_module.py: hierarchical embedding + retrieval”).
• Dynamic choice logic in Router (LLMSingleSelector).
• Multi-document ingestion path vs. single-query retrieval.
• Configuration/credentials boundary (e.g., .env, google_credentials.json) as a secure vault.