# yale-sys/confidential-prompting architecture

> Architecture diagram of the GitHub repository yale-sys/confidential-prompting, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yale-sys/confidential-prompting
- Repository: https://github.com/yale-sys/confidential-prompting

## Overview

##### Project type and purpose
- This is an open-source proof-of-concept CLI tool for Confidential Prompting.
- It implements Secure Partitioned Decoding (SPD) and Prompt Obfuscation (PO) around Llama models.
- Key use cases:
- Obfuscate sensitive tokens in prompts
- Spawn a multi-process “LLM Serving Server” and “Secure Prompt Vault” workers to simulate CVMs
- Test privacy budgets and decoding protocols in a non-CVM environment

##### File structure
- Top-level scripts:
- po.py       → Prompt Obfuscation
- smd.py      → LLM Serving Daemon (master & worker modes)
- prompt.py   → Prompt parser / loader
- llama.py    → Llama model wrapper (loads HF model, runs inference)
- attention.py & gqs.py → Modules for attention‐state handling and gradient queries
- logger.py   → Logging utility
- Single folder: no frontend/backend separation—this is a set of cooperating microservices/processes.
- Configuration & data:
- input.txt  → Sample prompt file
- states_dir/→ Directory where public/private attention states are saved
- requirements.txt, .gitignore, LICENSE, README.md — standard metadata

##### README for additional insights
- Three phases:
1) Prompt Obfuscation (po.py)
2) Start LLM Server (smd.py --standalone_master)
3) Spawn Secure Vaults (smd.py --standalone_worker or multi‐user)
- Parameters control γ (gamma), ε (epsilon), temperature, model, device, etc.
- Server & workers communicate over TCP (MASTER_ADDR, MASTER_PORT)

##### Main components & interactions
A. Prompt Obfuscator (po.py)
- Reads input.txt via prompt.py
- Generates γ virtual prompts, obfuscates <redacted> tokens
- Calls llama.py/attention.py to sample attention states
- Writes public/private states to states_dir/
B. LLM Serving Server (smd.py master)
- Loads HF model via llama.py
- Reads states_dir/
- Listens on TCP port for worker connections
- Distributes token‐generation jobs (SPD protocol)
C. Secure Prompt Vault Worker (smd.py worker)
- Connects to master over TCP
- Retrieves encrypted/pseudonymized tokens
- Uses private states to decode securely
- Returns token outputs or intermediate results
D. Shared modules
- prompt.py         – canonical parser for prompt structure
- attention.py/gqs.py – core SPD logic
- logger.py         – logs events, timings, token exchanges

##### Architectural patterns & design principles
- Micro-process architecture: separate Obfuscator, Master, and Worker
- Client-server pattern: smd.py master (server) & workers (clients) communicate over sockets
- Modular Python packages: clear separation of prompt parsing, model inference, privacy logic, logging
- Directory-based state persistence: “states_dir” as shared storage for attention states
- Configurable privacy parameters: γ, ε, temperature, prob_dist

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Input
    IN["input.txt"]:::data

    %% External Dependency
    HF["Hugging Face Model\n(on GPU)"]:::external

    %% Components
    subgraph "Prompt Obfuscator" 
        direction TB
        PO["po.py"]:::obf
        PP["prompt.py"]:::module
        ATT["attention.py"]:::module
        GQS["gqs.py"]:::module
    end

    subgraph "States Directory" 
        direction TB
        ST[(states_dir/)]:::storage
    end

    subgraph "LLM Serving Server (Master)" 
        direction TB
        SM["smd.py\n(master mode)"]:::server
        LL["llama.py"]:::module
        LOG["logger.py"]:::module
        ATT2["attention.py"]:::module
        GQS2["gqs.py"]:::module
    end

    subgraph "Secure Prompt Vault Workers" 
        direction TB
        W1["smd.py\n(worker mode)"]:::worker
        ATT3["attention.py"]:::module
        GQS3["gqs.py"]:::module
        LOG2["logger.py"]:::module
        W2["smd.py\n(worker mode)"]:::worker
        ATT4["attention.py"]:::module
        GQS4["gqs.py"]:::module
        LOG3["logger.py"]:::module
    end

    %% Data Flows
    IN -->|"reads prompt"| PO
    PO -->|"writes states"| ST
    ST -->|"public states"| SM
    HF -->|"loads model"| SM
    SM -->|"gQS query"| W1
    W1 -->|"token response"| SM
    SM -->|"gQS query"| W2
    W2 -->|"token response"| SM
    ST -->|"private states"| W1
    ST -->|"private states"| W2

    %% Click Events
    click PO "https://github.com/yale-sys/confidential-prompting/blob/main/po.py"
    click PP "https://github.com/yale-sys/confidential-prompting/blob/main/prompt.py"
    click ATT "https://github.com/yale-sys/confidential-prompting/blob/main/attention.py"
    click GQS "https://github.com/yale-sys/confidential-prompting/blob/main/gqs.py"
    click SM "https://github.com/yale-sys/confidential-prompting/blob/main/smd.py"
    click LL "https://github.com/yale-sys/confidential-prompting/blob/main/llama.py"
    click LOG "https://github.com/yale-sys/confidential-prompting/blob/main/logger.py"
    click ATT2 "https://github.com/yale-sys/confidential-prompting/blob/main/attention.py"
    click GQS2 "https://github.com/yale-sys/confidential-prompting/blob/main/gqs.py"
    click W1 "https://github.com/yale-sys/confidential-prompting/blob/main/smd.py"
    click ATT3 "https://github.com/yale-sys/confidential-prompting/blob/main/attention.py"
    click GQS3 "https://github.com/yale-sys/confidential-prompting/blob/main/gqs.py"
    click LOG2 "https://github.com/yale-sys/confidential-prompting/blob/main/logger.py"
    click W2 "https://github.com/yale-sys/confidential-prompting/blob/main/smd.py"
    click ATT4 "https://github.com/yale-sys/confidential-prompting/blob/main/attention.py"
    click GQS4 "https://github.com/yale-sys/confidential-prompting/blob/main/gqs.py"
    click LOG3 "https://github.com/yale-sys/confidential-prompting/blob/main/logger.py"
    click IN "https://github.com/yale-sys/confidential-prompting/blob/main/input.txt"
    click HF "https://github.com/yale-sys/confidential-prompting/blob/main/requirements.txt"

    %% Styles
    classDef obf fill:#cce5ff,stroke:#004085,color:#004085
    classDef server fill:#d4edda,stroke:#155724,color:#155724
    classDef worker fill:#fff3cd,stroke:#856404,color:#856404
    classDef storage fill:#e2e3e5,stroke:#6c757d,color:#6c757d
    classDef module fill:#d1ecf1,stroke:#0c5460,color:#0c5460
    classDef external fill:#cfe2ff,stroke:#084298,color:#084298
    classDef data fill:#f5c6cb,stroke:#721c24,color:#721c24
```
