# ist-daslab/panzamail architecture

> Architecture diagram of the GitHub repository ist-daslab/panzamail, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ist-daslab/panzamail
- Repository: https://github.com/ist-daslab/panzamail

## Overview

##### Project type and purpose
- Panza is a full‐pipeline, on‐device machine‐learning system for automatically fine-tuning an LLM on your personal email history and then serving it via a Retrieval-Augmented-Generation (RAG) inference engine, exposed through multiple interfaces (CLI, GUI, web, JSON).

##### File structure
- Top‐level folders:
- configs/  – YAML configs for all stages (base, data_preparation, finetuning, interfaces, writer)
- data/    – raw and preprocessed email data and vector stores
- notebooks/ – Jupyter notebooks for demo and RAG experiments
- prompt_preambles/ – text templates used in prompting
- scripts/  – orchestration scripts for data prep, training, and inference
- src/panza/ – core Python modules for data_preparation, finetuning, interface, llm, prompting, retriever, utils, and the writer entry point
- tests/   – unit tests for key components

##### README for additional insights
- Three main stages:
1. **Data Playback** (data preparation + prompt‐summarization)
2. **Local Fine-Tuning** via RoSA or full-parameter methods
3. **Serving** with RAG (vector DB + retriever) and Interfaces (CLI, GUI, web, JSON)
- Dependencies/externals: Hugging Face (model download), W&B (optional logging), FAISS (vector store), GPU/CPU environment, Chrome extension for Gmail.

##### Main components and their relationships
a. **Data Source**:
- Input MBOX (.mbox in data/) → extracted by scripts/prepare_data → JSONL records in data/
b. **Data Preparation Module** (src/panza/data_preparation):
- Extract raw emails → summarize into (instruction, email) pairs → split into train/test → build FAISS index → store vector DB (data/*.faiss, .pkl)
c. **Fine-Tuning Module** (src/panza/finetuning):
- Preprocessing → pass training JSONL + base checkpoint → apply RoSA (LoRA + sparse) or full-parameter → output model checkpoints in checkpoints/
d. **Vector Store (RAG)**:
- FAISS index built during prep; used at inference to retrieve context emails
e. **Inference Engine (“writer”)** (src/panza/writer.py + src/panza/llm + src/panza/retriever + src/panza/prompting):
- Loads user-specific checkpoint + configs →
- For each query: invoke retriever → fetch k relevant emails → generate prompt via prompting/*.py → call LLM backend (transformers / peft / local / ollama) → produce email text
f. **Interfaces** (src/panza/interface):
- CLI (cli.py), GUI (gui.py + gui_b.py), JSON (json.py), Web (web.py) → each wraps the writer module and exposes it over stdout, a local web server (port 5001 by default), or JSON file
g. **External Services**:
- Hugging Face API for base model download
- Weights & Biases for experiment tracking (optional)
- Chrome/Gmail plugin calling the local web interface

##### Architectural patterns and design principles
- **Modular pipeline**: clear stage separation (prep → train → serve)
- **Config-driven**: all behavior controlled by YAMLs in configs/ and overrides via CLI flags
- **Dependency inversion**: LLM backends abstracted under src/panza/llm/base.py
- **Extensibility**: new interfaces, retrievers, prompting strategies, or LLM backends plug in by dropping YAML + implementation modules

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Preparation Stage
    subgraph "Data Ingestion & Preparation"
        direction TB
        PD["scripts/prepare_data.py"]:::prep
        EE["extract_emails.py"]:::prep
        PR["prepare_raft_emails.py"]:::prep
        SD["split_data.py"]:::prep
        SU["summarize_emails.py"]:::prep
        RA["rag.py"]:::prep
        JSONL[(Processed JSONL)]:::cyl
        FAISS[(FAISS Index)]:::cyl
        PD --> EE --> SU --> SD --> RA --> JSONL
        RA --> FAISS
    end

    %% Fine-Tuning Stage
    subgraph "Fine-Tuning Engine"
        direction TB
        PREP["src/panza/finetuning/preprocessing.py"]:::train
        TRAIN["src/panza/finetuning/train.py"]:::train
        S1["scripts/train_rosa.sh"]:::train
        S2["scripts/train_fft.sh"]:::train
        CKPT[(Model Checkpoints)]:::cyl
        JSONL --> PREP --> TRAIN --> CKPT
        TRAIN --> S1
        TRAIN --> S2
    end

    %% Serving Stage
    subgraph "Serving"
        direction TB
        subgraph "RAG Inference Pipeline"
            direction TB
            WR["src/panza/writer.py"]:::serv
            LLMB["src/panza/llm/base.py"]:::serv
            LLML["src/panza/llm/local.py"]:::serv
            LLLO["src/panza/llm/ollama.py"]:::serv
            RETB["src/panza/retriever/base.py"]:::serv
            RETN["src/panza/retriever/none.py"]:::serv
            FA["src/panza/retriever/faiss.py"]:::serv
            PB["src/panza/prompting/base.py"]:::serv
            PE["src/panza/prompting/email_prompting.py"]:::serv
            PS["src/panza/prompting/summarization_prompting.py"]:::serv
            PU["src/panza/prompting/utils.py"]:::serv
            CKPT --> WR
            FAISS --> RETB --> FA
            WR --> LLMB & LLML & LLLO
            WR --> RETB & RETN
            WR --> PB & PE & PS & PU
        end
        subgraph "Interface Layer"
            direction TB
            CLI["src/panza/interface/cli.py"]:::serv
            GUI1["src/panza/interface/gui.py"]:::serv
            GUI2["src/panza/interface/gui_b.py"]:::serv
            WEB["src/panza/interface/web.py"]:::serv
            JSONI["src/panza/interface/json.py"]:::serv
            WR --> CLI & GUI1 & GUI2 & WEB & JSONI
            CLI --> User["End User"]
            GUI1 --> User
            GUI2 --> User
            WEB --> User
            JSONI --> User
        end
    end

    %% Configuration Management
    subgraph "Configuration Management"
        direction TB
        C1["configs/base.yaml"]:::cfg
        C2["configs/panza_preparation.yaml"]:::cfg
        C3["configs/panza_finetuning.yaml"]:::cfg
        C4["configs/panza_writer.yaml"]:::cfg
        C5["configs/user/default.yaml"]:::cfg
        C6["configs/interfaces/cli.yaml"]:::cfg
        C7["configs/interfaces/gui.yaml"]:::cfg
        C8["configs/interfaces/web.yaml"]:::cfg
        C9["configs/interfaces/json.yaml"]:::cfg
        C1 --> PD & PREP & WR & CLI
        C2 --> PD
        C3 --> PREP & TRAIN
        C4 --> WR
        C5 --> WR
        C6 --> CLI
        C7 --> GUI1
        C8 --> WEB
        C9 --> JSONI
    end

    %% External Services
    subgraph "External Services"
        direction TB
        HF["Hugging Face"]:::ext
        WB["Weights & Biases"]:::ext
        CE["Chrome Extension"]:::ext
        HF -.-> PREP
        WB -.-> TRAIN
        CE --> WEB
    end

    %% Styles
    classDef prep fill:#D0E7FF,stroke:#428BCA,color:#000;
    classDef train fill:#DFF0D8,stroke:#3C763D,color:#000;
    classDef serv fill:#FCF8E3,stroke:#8A6D3B,color:#000;
    classDef ext fill:#EEEEEE,stroke:#777,color:#000;
    classDef cyl fill:#FFFFFF,stroke:#555,stroke-dasharray: 5 5,color:#000;
    classDef cfg fill:#F2F2F2,stroke:#AAA,color:#000;

    %% Click Events
    click PD "https://github.com/ist-daslab/panzamail/blob/main/scripts/prepare_data.py"
    click EE "https://github.com/ist-daslab/panzamail/blob/main/src/panza/data_preparation/extract_emails.py"
    click PR "https://github.com/ist-daslab/panzamail/blob/main/src/panza/data_preparation/prepare_raft_emails.py"
    click SD "https://github.com/ist-daslab/panzamail/blob/main/src/panza/data_preparation/split_data.py"
    click SU "https://github.com/ist-daslab/panzamail/blob/main/src/panza/data_preparation/summarize_emails.py"
    click RA "https://github.com/ist-daslab/panzamail/blob/main/src/panza/data_preparation/rag.py"
    click PREP "https://github.com/ist-daslab/panzamail/blob/main/src/panza/finetuning/preprocessing.py"
    click TRAIN "https://github.com/ist-daslab/panzamail/blob/main/src/panza/finetuning/train.py"
    click S1 "https://github.com/ist-daslab/panzamail/blob/main/scripts/train_rosa.sh"
    click S2 "https://github.com/ist-daslab/panzamail/blob/main/scripts/train_fft.sh"
    click FA "https://github.com/ist-daslab/panzamail/blob/main/src/panza/retriever/faiss.py"
    click WR "https://github.com/ist-daslab/panzamail/blob/main/src/panza/writer.py"
    click LLMB "https://github.com/ist-daslab/panzamail/blob/main/src/panza/llm/base.py"
    click LLML "https://github.com/ist-daslab/panzamail/blob/main/src/panza/llm/local.py"
    click LLLO "https://github.com/ist-daslab/panzamail/blob/main/src/panza/llm/ollama.py"
    click RETB "https://github.com/ist-daslab/panzamail/blob/main/src/panza/retriever/base.py"
    click RETN "https://github.com/ist-daslab/panzamail/blob/main/src/panza/retriever/none.py"
    click PB "https://github.com/ist-daslab/panzamail/blob/main/src/panza/prompting/base.py"
    click PE "https://github.com/ist-daslab/panzamail/blob/main/src/panza/prompting/email_prompting.py"
    click PS "https://github.com/ist-daslab/panzamail/blob/main/src/panza/prompting/summarization_prompting.py"
    click PU "https://github.com/ist-daslab/panzamail/blob/main/src/panza/prompting/utils.py"
    click CLI "https://github.com/ist-daslab/panzamail/blob/main/src/panza/interface/cli.py"
    click GUI1 "https://github.com/ist-daslab/panzamail/blob/main/src/panza/interface/gui.py"
    click GUI2 "https://github.com/ist-daslab/panzamail/blob/main/src/panza/interface/gui_b.py"
    click WEB "https://github.com/ist-daslab/panzamail/blob/main/src/panza/interface/web.py"
    click JSONI "https://github.com/ist-daslab/panzamail/blob/main/src/panza/interface/json.py"
```
