# quivrhq/quivr architecture

> Architecture diagram of the GitHub repository quivrhq/quivr, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/quivrhq/quivr
- Repository: https://github.com/quivrhq/quivr

## Overview

##### Project Type & Purpose
- Quivr is a Python-based library aimed at enabling “a second brain” by providing an opinionated Retrieval Augmented Generation (RAG) system. It facilitates ingestion of various file types (PDF, TXT, Markdown, etc.), processing (using multiple parsers/implementations), interfacing with different Large Language Models (LLMs), and supporting customizable workflows.
- The README emphasizes quick setup, integration with diverse LLM providers (OpenAI, Anthropic, Mistral, etc.), and a workflow-based pipeline for retrieving and generating answers—making it clear that the project is designed as a core backend engine that can be plugged into applications (with optional UI/examples like chatbot integrations).

##### Analyzing the File Structure
- The top-level directories include configuration files, documentation (docs), examples, and a core directory.
- Within the core directory, the “quivr_core” package is structured into modules that each focus on a specific component:
- The “brain” module represents the core orchestrator for managing the “second brain” concept.
- The “files” module handles file ingestion and parsing.
- The “language” module deals with language models and related utilities.
- The “llm” and “llm_tools” directories address interfacing with LLM endpoints and additional tools (like web search).
- The “processor” module features various “implementations” (default, megaparse, simple_txt, tika, etc.) for processing and extracting content.
- The “rag” module sets up the retrieval augmented generation pipeline including entities, prompts, and workflow orchestration.
- The “storage” module manages file or local storage options.
- There are also extensive tests and examples demonstrating how to integrate with external systems such as chatbot UIs (chainlit), indicating an extensible, modular design.

a. Main Components to Represent:
- Ingestion & File Management – Represented by the “files” module, which accepts any file input. Also note potential additional parsers from the “processor” module.
- Brain/Orchestration Core – The “brain” component that acts as the central manager coordinating ingestion, processing, and query answering.
- Language Model Interface – Show the integration point for LLMs (in “llm” and “llm_tools”), which interface with external providers (OpenAI, Anthropic, etc.). Also note that reranking and other additional AI tools might be part of this layer.
- Storage & Persistence – Represent the “storage” module, which persists processed documents (supporting both file-based and local storage approaches).
- Configuration & Extensibility – Indicate that workflow configurations (YAML files) are used to customize behavior and routing between components.
- External Integrations – Optionally, include integrations with external search (internet search, etc.) and connectors such as chainlit for chatbot user interfaces, as shown in the examples.

b. Relationships & Interactions:
- Files are ingested → handed off to processors.
- Processed content is stored in the storage module.
- The Brain orchestrates building the “knowledge” base and triggers the RAG workflow.
- During query time, user questions pass through workflow nodes (filter_history, rewrite, retrieve, generate_rag).
- The query then interfaces with the LLM layer to get a final response.
- How the workflow (the DAG) coordinates these interactions.

c. Architectural Patterns/Design Principles:
- Modular Separation: Each module (files, processor, brain, rag, llm, storage) has a clear responsibility.
- Configurability: The use of YAML-based workflow definitions illustrates a pluggable and configurable pipeline architecture.
- Extensibility: Multiple implementations for file processing indicate that the system is designed for easy extensibility and customization.
- Integration Focus: Clear interfaces for external APIs (LLM providers, internet search tools) and additional plugin points.

d. Relevant Technologies, Frameworks, or Libraries:
- Python (3.10+)
- Docker (Dockerfile.test present)
- GitHub workflows for CI/CD
- Examples show integration with chainlit (for chat interfaces)
- External LLM API integrations (OpenAI, Anthropic, Mistral, etc.)

## Diagram (Mermaid)

```mermaid
flowchart TD
    UQ("User Query"):::query
    BO("Brain Orchestration"):::core
    IFM("Ingestion & File Management"):::processing
    PP("Processing Pipeline"):::processing
    SP("Storage & Persistence"):::processing
    WG("Workflow Engine (RAG Pipeline)"):::core
    LLM("Language Model Interface"):::core
    CE("Configuration & Extensibility"):::config
    EI("External Integrations (Chatbot UI)"):::integration

    UQ -->|"sends_query"| BO
    BO -->|"initiates_ingestion"| IFM
    IFM -->|"parses_files"| PP
    PP -->|"stores_content"| SP
    BO -->|"triggers_workflow"| WG
    WG -->|"calls_LLM"| LLM
    LLM -->|"returns_response"| BO
    CE -->|"configures"| WG
    EI -->|"integrates"| BO

    click IFM "https://github.com/quivrhq/quivr/blob/main/core/quivr_core/files/file.py"
    click BO "https://github.com/quivrhq/quivr/blob/main/core/quivr_core/brain/brain.py"
    click PP "https://github.com/quivrhq/quivr/tree/main/core/quivr_core/processor"
    click WG "https://github.com/quivrhq/quivr/blob/main/core/quivr_core/rag/quivr_rag.py"
    click LLM "https://github.com/quivrhq/quivr/blob/main/core/quivr_core/llm/llm_endpoint.py"
    click LLM "https://github.com/quivrhq/quivr/tree/main/core/quivr_core/llm_tools"
    click SP "https://github.com/quivrhq/quivr/blob/main/core/quivr_core/storage/file.py"
    click CE "https://github.com/quivrhq/quivr/blob/main/core/example_workflows/talk_to_file_rag_config_workflow.yaml"
    click EI "https://github.com/quivrhq/quivr/tree/main/examples/chatbot"

    classDef query fill:#f9d,stroke:#333,stroke-width:2px;
    classDef core fill:#c9f,stroke:#333,stroke-width:2px;
    classDef processing fill:#cfc,stroke:#333,stroke-width:2px;
    classDef config fill:#fcf,stroke:#333,stroke-width:2px;
    classDef integration fill:#ffc,stroke:#333,stroke-width:2px;
```
