How yale-sys/confidential-prompting works
Project type and purpose
• This is an open-source proof-of-concept CLI tool for Confidential Prompting.
• It implements Secure Partitioned Decoding (SPD) and Prompt Obfuscation (PO) around Llama models.
• Key use cases:
• Obfuscate sensitive tokens in prompts
• Spawn a multi-process “LLM Serving Server” and “Secure Prompt Vault” workers to simulate CVMs
• Test privacy budgets and decoding protocols in a non-CVM environment
File structure
• Top-level scripts:
• po.py → Prompt Obfuscation
• smd.py → LLM Serving Daemon (master & worker modes)
• prompt.py → Prompt parser / loader
• llama.py → Llama model wrapper (loads HF model, runs inference)
• attention.py & gqs.py → Modules for attention‐state handling and gradient queries
• logger.py → Logging utility
• Single folder: no frontend/backend separation—this is a set of cooperating microservices/processes.
• Configuration & data:
• input.txt → Sample prompt file
• states_dir/→ Directory where public/private attention states are saved
• requirements.txt, .gitignore, LICENSE, README.md — standard metadata
README for additional insights
• Three phases:
1) Prompt Obfuscation (po.py)
2) Start LLM Server (smd.py --standalone_master)
3) Spawn Secure Vaults (smd.py --standalone_worker or multi‐user)
• Parameters control γ (gamma), ε (epsilon), temperature, model, device, etc.
• Server & workers communicate over TCP (MASTER_ADDR, MASTER_PORT)
Main components & interactions
A. Prompt Obfuscator (po.py)
• Reads input.txt via prompt.py
• Generates γ virtual prompts, obfuscates <redacted> tokens
• Calls llama.py/attention.py to sample attention states
• Writes public/private states to states_dir/
B. LLM Serving Server (smd.py master)
• Loads HF model via llama.py
• Reads states_dir/
• Listens on TCP port for worker connections
• Distributes token‐generation jobs (SPD protocol)
C. Secure Prompt Vault Worker (smd.py worker)
• Connects to master over TCP
• Retrieves encrypted/pseudonymized tokens
• Uses private states to decode securely
• Returns token outputs or intermediate results
D. Shared modules
• prompt.py – canonical parser for prompt structure
• attention.py/gqs.py – core SPD logic
• logger.py – logs events, timings, token exchanges
Architectural patterns & design principles
• Micro-process architecture: separate Obfuscator, Master, and Worker
• Client-server pattern: smd.py master (server) & workers (clients) communicate over sockets
• Modular Python packages: clear separation of prompt parsing, model inference, privacy logic, logging
• Directory-based state persistence: “states_dir” as shared storage for attention states
• Configurable privacy parameters: γ, ε, temperature, prob_dist