How urjcdslab/vilt works
Project Type and Purpose
• ViLT is a full-stack tutoring platform designed for virtual tutoring at ETSII (URJC). It helps teachers upload material (PDFs) for processing and indexing (using ChromaDB) while students query the system. The platform uses advanced NLP techniques, language models (Llama 3.1, OpenAI), LangChain, and retrieval augmented generation (RAG) techniques.
• This project not only handles content management (uploading, indexing, document processing) but also orchestrates dynamic conversation flows and feedback loops to fine-tune responses.
Main Architectural Components
a. Frontend / User Interface:
• Templates: HTML files in the “templates” folder for various user roles (admin, teacher, student, chatbot interfaces).
• Static Assets: CSS, JS, and image resources in “static” which style and manage user interactions.
b. Backend Application Layer:
• The “vilt” directory contains core modules. These include:
• app.py: Entry-point for the application.
• Module managers such as chatbotmanager, careermanager, dashmanager, databasemanager, docmanager, embeddingmanager, infomanager, keywordsmanager, promptmanager, timemanager, and usermanager. Each one encapsulates a specific domain (e.g., user management, chatbot logic, document processing).
• The “bbdd-management” folder contains scripts dedicated to database tasks (creation, deletion, CSV management) which indicate structured external processes for handling data.
c. Data and External Content:
• “data-external” contains external CSVs (for careers, student lists, and syllabi) that appear to be raw data sources ingested into the system.
d. Document and Content Processing:
• PDF uploads from professors are processed in the workflow (document manager likely manages parsing and indexing documents). The “docs” folder itself and the external test folders for processed documents indicate that documentation and content processing is a key part.
e. Integration with External Services:
• The system requires connections to MongoDB (database layer).
• It integrates with external language model APIs (OpenAI, OLLAMA) and uses LangChain for query processing.
• ChromaDB is used for fast document indexing and retrieval.
f. Testing:
• The “testes” folder provides test scripts for key functionalities such as database operations, chatbot training, dash management, and data processing—indicating robust testing practices.
Relationships and Data Flow
• Users (teachers and students) interact through the web interface (rendered via templates and static assets).
• For content ingestion:
• Teachers upload PDFs → routed to the docmanager (and embeddings creation/deletion modules) → content is indexed into ChromaDB.
• Query and tutoring workflow:
• Students submit queries → routed through chatbotmanager where incoming queries are processed; LangChain and language models (Llama, OpenAI) are used to fetch the best response.
• Feedback is collected (through templates for feedback) and stored via the databasemanager.
• Database Handling:
• The database management scripts and the databasemanager module interact with MongoDB to store and retrieve necessary data (user information, content metadata, feedback).
• Management dashboards and administrative modules (admin-add, admin-modify templates, etc.) indicate separation of functionality based on roles.
Architectural Patterns and Design Principles
• Modular decomposition: Each core functionality (chatbot operations, document management, user administration) is encapsulated in its own module.
• Separation of concerns: Clear separation between presentation (frontend), business logic (managers in the “vilt” folder), and data handling (database management and external CSV ingestion).
• Data flow directions: Clarify how data flows from user requests → backend processing modules → database or external APIs, and then responses back to the user.