How zorrat/fuzzy-private-entity-set-intersection works
Project Type and Purpose
• Type: A Go-based open-source library/tool implementing a secure, fuzzy PSI (Private Set Intersection) protocol.
• Purpose: Ingest two sets of entities, clean & vectorize them, cluster similar entities (fuzzy matching), compress results, perform threshold‐decryption based secure multi-party computation, and finally serialize results.
Main Architectural Components
A. Data Ingestion & Preparation
– data.Dataloader
– data.Cleaner
– data.Vectorizer
Input: JSON or TXT entity sets
B. Fuzzy Matching (Clustering)
– clustering.Clusterer
Implements approximate matching algorithm
C. Compression
– compression.Compressor
Reduces payload size before encryption/transmission
D. Encryption & Secure Compute
– encryption.Encryptor
– encryption.Decryptor
Represents threshold decryption, multiparty compute layer (external)
E. Serialization
– serialization.Serializer
Formats final results (e.g., JSON)
F. Utilities & Orchestration
– utils.Utils: common helper functions
– fpsi.go: top-level orchestration entry point
G. Examples / CLI
– examples/examples.go: demonstrates usage
Data Flows and Interactions
Data Ingestion → Cleaning → Vectorization → Clustering → Compression → Encryption → (External MPC) → Decryption → Serialization → Output
Architectural Patterns & Design Principles
• Pipeline Pattern: linear sequence of transformations.
• Modularity: each sub‐package (data, clustering, compression, encryption, serialization, utils) is a separate component.
• Separation of Concerns: data prep, algorithmic matching, compression, crypto, and I/O are distinctly modular.
Final Checklist
[ ] All six major modules (data, clustering, compression, encryption, serialization, utils) clearly labeled
[ ] External MPC/threshold decryption drawn as a separate entity (cloud or service)