# chrys/yasrl architecture

> Architecture diagram of the GitHub repository chrys/yasrl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/chrys/yasrl
- Repository: https://github.com/chrys/yasrl

## Overview

##### Project type and purpose
- yasrl is an open-source Python library (not a standalone web app) for building Retrieval-Augmented Generation (RAG) pipelines.
- It provides opinionated defaults, a configuration system, pluggable providers for embeddings/LLM/vector stores, pipelines for indexing and querying, and evaluation hooks.

##### File structure
- Top level:
- docs/ – design docs, configuration guides, project specs
- src/yasrl/ – core package
- tests/ – unit tests covering every module
- src/yasrl subpackages:
- config/ – manager.py (loads .env/config), models.py (pydantic models)
- loaders.py – ingestion of raw data
- text_processor.py – cleanup, tokenization, chunking
- vector_store.py – interface to in-memory or external (Postgres) stores
- providers/ – embeddings.py, llm.py (pluggable provider abstractions)
- query_processor.py – builds retrieval + generation workflow
- pipeline.py – orchestrates index vs. ask vs. complete pipelines
- evaluation/ – base.py (metrics), cli.py (evaluate via CLI)
- exceptions.py, models.py – domain models & errors
- config files: .env.example, pyproject.toml, requirements.txt

##### README for additional insights
- Confirms design: core RAG pipeline, optional extras for OpenAI/Gemini/Ollama, PostgreSQL, dev/test/docs.

a. Main components
1. Configuration Layer
– Config Manager (reads .env, YAML/JSON configs via pydantic models)
2. Data Ingestion & Processing
– Loader(s)
– Text Processor (preprocessing, chunking)
3. Vector Store
– In-memory (default) or external (Postgres)
4. Embeddings Provider
– Abstract interface; concrete clients for OpenAI, Gemini, Ollama
5. LLM Provider
– Abstract interface; concrete clients for OpenAI, Gemini, Ollama
6. Query Processor & Pipeline
– Retrieval + Generation orchestration
– Pipeline module selects index vs. ask vs. complete
7. Evaluation
– Base metrics & CLI runner
8. CLI / API (if used) for end-user interactions
9. External Services
– LLM APIs
– PostgreSQL (vector store)
– Environment (AWS SSM, Vault, etc. if used)

b. Relationships & interactions
1. Config Manager → provides settings (API keys, endpoints, model names)
2. Loader → Text Processor → Vector Store (indexing pipeline)
3. Query Processor ↔ Vector Store (retrieval)
4. Query Processor → Embeddings Provider (to embed queries/doc chunks)
5. Query Processor → LLM Provider (to complete or answer)
6. Pipeline module drives Data Ingestion or Query workflows end-to-end
7. Evaluation CLI invokes Pipeline & Query Processor, collects metrics

c. Architectural patterns & principles
- Pipeline pattern for ETL and inference
- Provider (Strategy) pattern for swapping embeddings/LLM/vector store backends
- Dependency injection via Config Manager and pydantic models
- Modular layered design: config → ingestion → processing → storage → inference → evaluation

d. Key technologies & libraries
- Python 3.8+
- pydantic (config models)
- AnyIO / click (CLI in evaluation/cli.py)
- HTTP clients (requests or aiohttp) for LLM providers
- SQLAlchemy or psycopg2 for Postgres vector store (optional)
- pytest for tests

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Configuration Layer
    subgraph "Configuration Layer"
        direction TB
        ConfigManager["Config Manager"]:::core
        ConfigModels["Config Models"]:::core
    end

    %% Core Modules
    subgraph "Core Modules"
        direction TB
        Loaders["Loader(s)"]:::core
        TextProcessor["Text Processor"]:::core
        VectorStore["Vector Store"]:::core
        QueryProcessor["Query Processor"]:::core
        Pipeline["Pipeline Orchestrator"]:::core
        ExceptionsModels["Exceptions & Domain Models"]:::core
    end

    %% Providers
    subgraph "Providers (Extension Points)"
        direction TB
        EmbeddingsProvider["Embeddings Provider Interface"]:::extension
        LLMProvider["LLM Provider Interface"]:::extension
    end

    %% Evaluation Layer
    subgraph "Evaluation Framework"
        direction TB
        EvalBase["Evaluation Base"]:::core
        EvalCLI["Evaluation CLI"]:::core
    end

    %% External Services
    subgraph "External Services"
        direction TB
        LLMAPIs["LLM APIs (OpenAI/Gemini/Ollama)"]:::external
        Postgres["PostgreSQL (Vector Store)"]:::external
        Env["Environment (SSM/Vault)"]:::external
    end

    %% Data Flow & Dependencies
    ConfigManager -->|"loads settings"| ConfigModels
    ConfigManager -->|"provides config"| Loaders
    ConfigManager -->|"provides config"| TextProcessor
    ConfigManager -->|"provides config"| EmbeddingsProvider
    ConfigManager -->|"provides config"| LLMProvider
    ConfigManager -->|"provides config"| VectorStore

    Loaders -->|"raw documents"| TextProcessor
    TextProcessor -->|"cleaned chunks"| VectorStore
    VectorStore -->|"vectors"| QueryProcessor
    QueryProcessor -->|"embeds chunks"| EmbeddingsProvider
    EmbeddingsProvider -->|"embeddings"| QueryProcessor
    QueryProcessor -->|"calls LLM"| LLMProvider
    LLMProvider -->|"generated text"| QueryProcessor

    Pipeline -->|"runs ingestion"| Loaders
    Pipeline -->|"runs inference"| QueryProcessor

    EvalCLI -->|"invokes pipeline"| Pipeline
    EvalCLI -->|"collects metrics"| EvalBase

    EvalBase -->|"uses exceptions/models"| ExceptionsModels
    Loaders -->|"uses exceptions/models"| ExceptionsModels
    TextProcessor -->|"uses exceptions/models"| ExceptionsModels
    QueryProcessor -->|"uses exceptions/models"| ExceptionsModels

    %% External integrations
    EmbeddingsProvider -->|"HTTP calls"| LLMAPIs
    LLMProvider -->|"HTTP calls"| LLMAPIs
    VectorStore -->|"stores/retrieves"| Postgres
    ConfigManager -->|"secrets"| Env

    %% Click Events
    click ConfigManager "https://github.com/chrys/yasrl/blob/main/src/yasrl/config/manager.py"
    click ConfigModels "https://github.com/chrys/yasrl/blob/main/src/yasrl/config/models.py"
    click Loaders "https://github.com/chrys/yasrl/blob/main/src/yasrl/loaders.py"
    click TextProcessor "https://github.com/chrys/yasrl/blob/main/src/yasrl/text_processor.py"
    click VectorStore "https://github.com/chrys/yasrl/blob/main/src/yasrl/vector_store.py"
    click EmbeddingsProvider "https://github.com/chrys/yasrl/blob/main/src/yasrl/providers/embeddings.py"
    click LLMProvider "https://github.com/chrys/yasrl/blob/main/src/yasrl/providers/llm.py"
    click QueryProcessor "https://github.com/chrys/yasrl/blob/main/src/yasrl/query_processor.py"
    click Pipeline "https://github.com/chrys/yasrl/blob/main/src/yasrl/pipeline.py"
    click EvalBase "https://github.com/chrys/yasrl/blob/main/src/yasrl/evaluation/base.py"
    click EvalCLI "https://github.com/chrys/yasrl/blob/main/src/yasrl/evaluation/cli.py"
    click ExceptionsModels "https://github.com/chrys/yasrl/blob/main/src/yasrl/exceptions.py"

    %% Styles
    classDef core fill:#AED6F1,stroke:#1B4F72,stroke-width:1px
    classDef extension fill:#F5B041,stroke:#A04000,stroke-width:1px
    classDef external fill:#ABEBC6,stroke:#186A3B,stroke-width:1px
```
