How su77ungr/casalioy works
Main Components
• Ingestion Module:
• Represented by the “ingest.py” script (within the casalioy folder).
• Responsible for consuming documents from the “source_documents” directory (various formats: CSV, PDF, DOCX, etc.), parsing and chunking them, and creating embeddings.
• Uses LangChain tools and llama-cpp-based embeddings (e.g., LlamaCppEmbeddings).
• Vector Database Component:
• A local vector store implemented using Qdrant – stored in the “db” folder.
• Acts as the repository for the generated document embeddings.
• Query / LLM Processing Module:
• The “startLLM.py” script initiates the process of handling user queries.
• Performs a similarity search on the vector database to retrieve relevant document segments.
• Forwards those segments as context into the chosen LLM (which may be GPT4All-J or a similar model compatible with LlamaCpp).
• GUI Interface:
• The “gui.py” (in the casalioy folder) provides a Streamlit-based web interface.
• Allows users to chat interactively with the local LLM, abstracting the CLI details.
• Utility and Environment Setup:
• “load_env.py” manages environment variables loaded from the .env file, ensuring configuration (like MODEL_PATH, chunk size, etc.) is centralized.
• Other utility scripts such as “CustomChains.py”, “ask_libgen.py”, and modules under “misc” provide additional functionality (e.g., conversion utilities, debugging, and even model conversion from GGML to GGUF).
Data Flow and Interactions
• Document Ingestion Flow:
• Documents in “source_documents” are processed by ingest.py.
• The ingestion process outputs embeddings which are then saved into the Qdrant vector store (local “db” folder).
• Query Processing Flow:
• When a user submits a query (via CLI “startLLM.py” or the Streamlit GUI):
→ The query is used to perform a similarity search on the local vector store.
→ The most relevant document segments are retrieved.
→ The selected context is forwarded along with the query to the LLM for answer generation.
• LLM and External Dependencies:
• The system relies on external libraries such as LangChain and llama-cpp for core functionalities.
• Dockerfiles and GitHub workflows encapsulate the environment for deployment or GPU support.
Architectural Patterns and Design Principles
• Modular and Layered Architecture:
• The project separates concerns into ingestion, storage, processing, and interaction layers.
• Configurability:
• With environment files (.env) and configuration scripts (load_env.py), the toolkit supports customization and extensibility.
• Containerization:
• Dockerfiles and CI/CD GitHub workflows ensure that the application can be deployed in a consistent environment.