How ist-daslab/panzamail works
Project type and purpose
• Panza is a full‐pipeline, on‐device machine‐learning system for automatically fine-tuning an LLM on your personal email history and then serving it via a Retrieval-Augmented-Generation (RAG) inference engine, exposed through multiple interfaces (CLI, GUI, web, JSON).
File structure
• Top‐level folders:
• configs/ – YAML configs for all stages (base, data_preparation, finetuning, interfaces, writer)
• data/ – raw and preprocessed email data and vector stores
• notebooks/ – Jupyter notebooks for demo and RAG experiments
• prompt_preambles/ – text templates used in prompting
• scripts/ – orchestration scripts for data prep, training, and inference
• src/panza/ – core Python modules for data_preparation, finetuning, interface, llm, prompting, retriever, utils, and the writer entry point
• tests/ – unit tests for key components
README for additional insights
• Three main stages:
1. Data Playback (data preparation + prompt‐summarization)
2. Local Fine-Tuning via RoSA or full-parameter methods
3. Serving with RAG (vector DB + retriever) and Interfaces (CLI, GUI, web, JSON)
• Dependencies/externals: Hugging Face (model download), W&B (optional logging), FAISS (vector store), GPU/CPU environment, Chrome extension for Gmail.
Main components and their relationships
a. Data Source:
• Input MBOX (.mbox in data/) → extracted by scripts/prepare_data → JSONL records in data/
b. Data Preparation Module (src/panza/data_preparation):
• Extract raw emails → summarize into (instruction, email) pairs → split into train/test → build FAISS index → store vector DB (data/*.faiss, .pkl)
c. Fine-Tuning Module (src/panza/finetuning):
• Preprocessing → pass training JSONL + base checkpoint → apply RoSA (LoRA + sparse) or full-parameter → output model checkpoints in checkpoints/
d. Vector Store (RAG):
• FAISS index built during prep; used at inference to retrieve context emails
e. Inference Engine (“writer”) (src/panza/writer.py + src/panza/llm + src/panza/retriever + src/panza/prompting):
• Loads user-specific checkpoint + configs →
• For each query: invoke retriever → fetch k relevant emails → generate prompt via prompting/*.py → call LLM backend (transformers / peft / local / ollama) → produce email text
f. Interfaces (src/panza/interface):
• CLI (cli.py), GUI (gui.py + gui_b.py), JSON (json.py), Web (web.py) → each wraps the writer module and exposes it over stdout, a local web server (port 5001 by default), or JSON file
g. External Services:
• Hugging Face API for base model download
• Weights & Biases for experiment tracking (optional)
• Chrome/Gmail plugin calling the local web interface
Architectural patterns and design principles
• Modular pipeline: clear stage separation (prep → train → serve)
• Config-driven: all behavior controlled by YAMLs in configs/ and overrides via CLI flags
• Dependency inversion: LLM backends abstracted under src/panza/llm/base.py
• Extensibility: new interfaces, retrievers, prompting strategies, or LLM backends plug in by dropping YAML + implementation modules