# krlabsorg/lettucedetect architecture

> Architecture diagram of the GitHub repository krlabsorg/lettucedetect, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/krlabsorg/lettucedetect
- Repository: https://github.com/krlabsorg/lettucedetect

## Overview

##### Project type and purpose
- LettuceDetect is a Python‑based library and toolkit for token‑level hallucination detection in Retrieval‑Augmented Generation (RAG) systems.
- It comprises three main “modes” of operation:
a) A model training and evaluation pipeline (trainer, evaluator, preprocess)
b) An inference library exposing a `HallucinationDetector` API
c) A web service (FastAPI) + client library + Streamlit demo

##### File structure
- Top‑level packages:
- `lettucedetect/` – core modules: datasets, preprocessors, model trainer, inference, evaluator
- `lettucedetect_api/` – FastAPI server (`server.py`), Python client (`client.py`), Pydantic models, tests
- Support directories:
- `scripts/` – CLI entrypoints for downloading data, preprocessing, training, evaluating, starting server, running demos
- `demo/` – Jupyter notebooks + Streamlit demo harness
- `assets/`, `docs/` – static assets, user documentation
- Configuration & CI: `pyproject.toml`, GitHub workflows, tests folder

##### README for architectural insights
- Model stages: preprocess RAGTruth → train token classifier (ModernBERT encoder) → evaluate (example/span level)
- Inference modes: “transformer” method, span or token‑level output
- Optional web API: start via `scripts/start_api.py`, client library usage

##### Main components & relationships
a. Data Layer
- External dataset repo (RAGTruth) → `scripts/download_dataset.py` → raw files →
- `lettucedetect/preprocess/preprocess_ragtruth.py` → cleaned JSON
b. Training Pipeline
- Input: preprocessed JSON + model checkpoint (e.g., ModernBERT)
- Modules: `trainer.py` (train loop), `evaluator.py` (metrics), `datasets/hallucination_dataset.py`
- Output: saved model artifacts
c. Inference Library
- `inference.py` exposes `HallucinationDetector.predict(...)`
- Takes context, question, answer → returns token or span predictions
d. Web API
- FastAPI app in `lettucedetect_api/server.py`
- Endpoints: `/detect_tokens`, `/detect_spans`
- Use the inference library under the hood
- Client in `lettucedetect_api/client.py` wraps HTTP calls
e. Demo Frontends
- Jupyter notebooks (`demo/*.ipynb`)
- Streamlit app (`demo/streamlit_demo.py`)
f. Orchestration Scripts
- Scripts under `scripts/` tie together the data, training, evaluation, and serving steps

##### Architectural patterns & design principles
- Clear separation of concerns: data → preprocess → train/evaluate → inference → service → client
- Modular Python package structure (core vs. API vs. CLI)
- Use of Pydantic models for API request/response typing
- CI/CD for testing (`.github/workflows/python-tests.yml`)
- Lightweight microservice (FastAPI) for model serving

##### Detail level
- Essential modules (preprocess, datasets, trainer, inference, server, client)
- External dependencies (HuggingFace, RAGTruth dataset)
- Optional pathways (span vs. token output; dev vs. prod API modes)
- Avoid over‑splitting trivial modules; group related files under logical components

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Layer
    subgraph "Data Layer"
        direction TB
        External["External Data Source\n(RAGTruth, HuggingFace Models)"]:::data
        Download["External Data Downloader Script\n(scripts/download_dataset.py)"]:::data
        Pre1["Preprocessor: RAGTruth\n(lettucedetect/preprocess/preprocess_ragtruth.py)"]:::data
        Pre2["Preprocessor: RAGBench\n(lettucedetect/preprocess/preprocess_ragbench.py)"]:::data
    end

    %% Core ML Layer
    subgraph "Core ML Layer"
        direction TB
        Dataset["Hallucination Dataset\n(lettucedetect/datasets/hallucination_dataset.py)"]:::core
        Trainer["Training Pipeline\n(lettucedetect/models/trainer.py)"]:::core
        Evaluator["Evaluation Module\n(lettucedetect/models/evaluator.py)"]:::core
        ModelStorage["Model Artifacts Storage\n(HF Hub / Local)"]:::core
        InferenceLib["Inference Library\n(lettucedetect/models/inference.py)"]:::core
    end

    %% API Layer
    subgraph "API Layer"
        direction TB
        Models["Pydantic Models\n(lettucedetect_api/models.py)"]:::api
        Server["FastAPI Service\n(lettucedetect_api/server.py)"]:::api
        Client["Python Client Library\n(lettucedetect_api/client.py)"]:::api
    end

    %% Presentation Layer
    subgraph "Demo Applications"
        direction TB
        Notebook1["Jupyter: Local Detection\n(demo/detection.ipynb)"]:::demo
        Notebook2["Jupyter: API Detection\n(demo/detection_api.ipynb)"]:::demo
        Streamlit["Streamlit Demo App\n(demo/streamlit_demo.py)"]:::demo
    end

    %% Orchestration Scripts
    subgraph "Orchestration Scripts"
        direction TB
        TrainScript["Training Orchestration Script\n(scripts/train.py)"]:::cli
        EvalScript["Evaluation Orchestration Script\n(scripts/evaluate.py)"]:::cli
        StartAPIScript["API Startup Script\n(scripts/start_api.py)"]:::cli
        UploadDatasetScript["Dataset Upload Script\n(scripts/upload_dataset.py)"]:::cli
        AnalyzeTokens["Token Analysis Script\n(scripts/analyze_tokens.py)"]:::cli
        Translator["Translator Script\n(scripts/translate/translate.py)"]:::cli
        TranslatorTests["Translator Tests\n(scripts/translate/test_hal_tags.py)"]:::cli
    end

    %% Relationships
    External --> Download --> Pre1 --> Dataset
    Download --> Pre2 --> Dataset
    Dataset --> Trainer --> Evaluator --> ModelStorage --> InferenceLib
    InferenceLib --> Notebook1
    Server --> Notebook2
    Server --> Streamlit
    InferenceLib --> Server
    Models --> Server
    Server <-->|"HTTP"| Client

    TrainScript --> Trainer
    EvalScript --> Evaluator
    StartAPIScript --> Server
    UploadDatasetScript --> Download
    AnalyzeTokens --> InferenceLib

    %% Click Events
    click Download "https://github.com/krlabsorg/lettucedetect/blob/main/scripts/download_dataset.py"
    click Pre1 "https://github.com/krlabsorg/lettucedetect/blob/main/lettucedetect/preprocess/preprocess_ragtruth.py"
    click Pre2 "https://github.com/krlabsorg/lettucedetect/blob/main/lettucedetect/preprocess/preprocess_ragbench.py"
    click Dataset "https://github.com/krlabsorg/lettucedetect/blob/main/lettucedetect/datasets/hallucination_dataset.py"
    click Trainer "https://github.com/krlabsorg/lettucedetect/blob/main/lettucedetect/models/trainer.py"
    click Evaluator "https://github.com/krlabsorg/lettucedetect/blob/main/lettucedetect/models/evaluator.py"
    click InferenceLib "https://github.com/krlabsorg/lettucedetect/blob/main/lettucedetect/models/inference.py"
    click Server "https://github.com/krlabsorg/lettucedetect/blob/main/lettucedetect_api/server.py"
    click Models "https://github.com/krlabsorg/lettucedetect/blob/main/lettucedetect_api/models.py"
    click Client "https://github.com/krlabsorg/lettucedetect/blob/main/lettucedetect_api/client.py"
    click Notebook1 "https://github.com/krlabsorg/lettucedetect/blob/main/demo/detection.ipynb"
    click Notebook2 "https://github.com/krlabsorg/lettucedetect/blob/main/demo/detection_api.ipynb"
    click Streamlit "https://github.com/krlabsorg/lettucedetect/blob/main/demo/streamlit_demo.py"
    click TrainScript "https://github.com/krlabsorg/lettucedetect/blob/main/scripts/train.py"
    click EvalScript "https://github.com/krlabsorg/lettucedetect/blob/main/scripts/evaluate.py"
    click StartAPIScript "https://github.com/krlabsorg/lettucedetect/blob/main/scripts/start_api.py"
    click UploadDatasetScript "https://github.com/krlabsorg/lettucedetect/blob/main/scripts/upload_dataset.py"
    click AnalyzeTokens "https://github.com/krlabsorg/lettucedetect/blob/main/scripts/analyze_tokens.py"
    click Translator "https://github.com/krlabsorg/lettucedetect/blob/main/scripts/translate/translate.py"
    click TranslatorTests "https://github.com/krlabsorg/lettucedetect/blob/main/scripts/translate/test_hal_tags.py"

    %% Styles
    classDef data fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef core fill:#90EE90,stroke:#333,stroke-width:1px
    classDef api fill:#FFA500,stroke:#333,stroke-width:1px
    classDef demo fill:#D8BFD8,stroke:#333,stroke-width:1px
    classDef cli fill:#FFD700,stroke:#333,stroke-width:1px
```
