How krlabsorg/lettucedetect works
Project type and purpose
• LettuceDetect is a Python‑based library and toolkit for token‑level hallucination detection in Retrieval‑Augmented Generation (RAG) systems.
• It comprises three main “modes” of operation:
a) A model training and evaluation pipeline (trainer, evaluator, preprocess)
b) An inference library exposing a HallucinationDetector API
c) A web service (FastAPI) + client library + Streamlit demo
File structure
• Top‑level packages:
• lettucedetect/ – core modules: datasets, preprocessors, model trainer, inference, evaluator
• lettucedetect_api/ – FastAPI server (server.py), Python client (client.py), Pydantic models, tests
• Support directories:
• scripts/ – CLI entrypoints for downloading data, preprocessing, training, evaluating, starting server, running demos
• demo/ – Jupyter notebooks + Streamlit demo harness
• assets/, docs/ – static assets, user documentation
• Configuration & CI: pyproject.toml, GitHub workflows, tests folder
README for architectural insights
• Model stages: preprocess RAGTruth → train token classifier (ModernBERT encoder) → evaluate (example/span level)
• Inference modes: “transformer” method, span or token‑level output
• Optional web API: start via scripts/start_api.py, client library usage
Main components & relationships
a. Data Layer
• External dataset repo (RAGTruth) → scripts/download_dataset.py → raw files →
• lettucedetect/preprocess/preprocess_ragtruth.py → cleaned JSON
b. Training Pipeline
• Input: preprocessed JSON + model checkpoint (e.g., ModernBERT)
• Modules: trainer.py (train loop), evaluator.py (metrics), datasets/hallucination_dataset.py
• Output: saved model artifacts
c. Inference Library
• inference.py exposes HallucinationDetector.predict(...)
• Takes context, question, answer → returns token or span predictions
d. Web API
• FastAPI app in lettucedetect_api/server.py
• Endpoints: /detect_tokens, /detect_spans
• Use the inference library under the hood
• Client in lettucedetect_api/client.py wraps HTTP calls
e. Demo Frontends
• Jupyter notebooks (demo/*.ipynb)
• Streamlit app (demo/streamlit_demo.py)
f. Orchestration Scripts
• Scripts under scripts/ tie together the data, training, evaluation, and serving steps
Architectural patterns & design principles
• Clear separation of concerns: data → preprocess → train/evaluate → inference → service → client
• Modular Python package structure (core vs. API vs. CLI)
• Use of Pydantic models for API request/response typing
• CI/CD for testing (.github/workflows/python-tests.yml)
• Lightweight microservice (FastAPI) for model serving
Detail level
• Essential modules (preprocess, datasets, trainer, inference, server, client)
• External dependencies (HuggingFace, RAGTruth dataset)
• Optional pathways (span vs. token output; dev vs. prod API modes)
• Avoid over‑splitting trivial modules; group related files under logical components