# zorrat/fuzzy-private-entity-set-intersection architecture

> Architecture diagram of the GitHub repository zorrat/fuzzy-private-entity-set-intersection, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zorrat/fuzzy-private-entity-set-intersection
- Repository: https://github.com/zorrat/fuzzy-private-entity-set-intersection

## Overview

##### Project Type and Purpose
- Type: A Go-based open-source library/tool implementing a secure, fuzzy PSI (Private Set Intersection) protocol.
- Purpose: Ingest two sets of entities, clean & vectorize them, cluster similar entities (fuzzy matching), compress results, perform threshold‐decryption based secure multi-party computation, and finally serialize results.

##### Main Architectural Components
A. Data Ingestion & Preparation
– data.Dataloader
– data.Cleaner
– data.Vectorizer
Input: JSON or TXT entity sets
B. Fuzzy Matching (Clustering)
– clustering.Clusterer
Implements approximate matching algorithm
C. Compression
– compression.Compressor
Reduces payload size before encryption/transmission
D. Encryption & Secure Compute
– encryption.Encryptor
– encryption.Decryptor
Represents threshold decryption, multiparty compute layer (external)
E. Serialization
– serialization.Serializer
Formats final results (e.g., JSON)
F. Utilities & Orchestration
– utils.Utils: common helper functions
– fpsi.go: top-level orchestration entry point
G. Examples / CLI
– examples/examples.go: demonstrates usage

##### Data Flows and Interactions
Data Ingestion → Cleaning → Vectorization → Clustering → Compression → Encryption → (External MPC) → Decryption → Serialization → Output

##### Architectural Patterns & Design Principles
- Pipeline Pattern: linear sequence of transformations.
- Modularity: each sub‐package (data, clustering, compression, encryption, serialization, utils) is a separate component.
- Separation of Concerns: data prep, algorithmic matching, compression, crypto, and I/O are distinctly modular.

##### Final Checklist
[ ] All six major modules (data, clustering, compression, encryption, serialization, utils) clearly labeled
[ ] External MPC/threshold decryption drawn as a separate entity (cloud or service)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Ingestion & Preparation
    subgraph "Data Ingestion & Preparation"
        DL["data.Dataloader"]:::blue
        CL["data.Cleaner"]:::blue
        V["data.Vectorizer"]:::blue
    end

    %% Fuzzy Matching
    subgraph "Fuzzy Matching (Clustering)"
        CU["clustering.Clusterer"]:::green
    end

    %% Compression
    subgraph "Compression"
        CO["compression.Compressor"]:::orange
    end

    %% Encryption & Secure Compute
    subgraph "Encryption & Secure Compute"
        EN["encryption.Encryptor"]:::red
        CLOUD("External MPC Service"):::cloud
        DE["encryption.Decryptor"]:::red
    end

    %% Serialization
    subgraph "Serialization"
        SE["serialization.Serializer"]:::purple
    end

    %% Utilities & Orchestration
    subgraph "Utilities & Orchestration"
        UT["utils.Utils"]:::gray
        OR["fpsi.go"]:::gray
    end

    %% Examples & CI
    EX["examples/examples.go"]:::gray
    CI[".github/workflows/main.yml"]:::ci

    %% Pipeline Flow
    OR -->|"start pipeline"| DL
    DL -->|"raw JSON/TXT"| CL
    CL -->|"cleaned data"| V
    V -->|"vectors"| CU
    CU -->|"clusters"| CO
    CO -->|"compressed bytes"| EN
    EN -->|"ciphertext"| CLOUD
    CLOUD -->|"computation results"| DE
    DE -->|"plaintext results"| SE
    SE -->|"serialized JSON"| OR
    OR -->|"demonstrates usage"| EX

    %% Utilities usage
    UT -.-> DL
    UT -.-> CL
    UT -.-> V
    UT -.-> CU
    UT -.-> CO
    UT -.-> EN
    UT -.-> DE
    UT -.-> SE
    UT -.-> OR

    %% CI integration
    CI -.-> OR

    %% Click Events
    click DL "https://github.com/zorrat/fuzzy-private-entity-set-intersection/blob/main/data/dataloader.go"
    click CL "https://github.com/zorrat/fuzzy-private-entity-set-intersection/blob/main/data/cleaner.go"
    click V "https://github.com/zorrat/fuzzy-private-entity-set-intersection/blob/main/data/vectorizer.go"
    click CU "https://github.com/zorrat/fuzzy-private-entity-set-intersection/blob/main/clustering/clusterer.go"
    click CO "https://github.com/zorrat/fuzzy-private-entity-set-intersection/blob/main/compression/compressor.go"
    click EN "https://github.com/zorrat/fuzzy-private-entity-set-intersection/blob/main/encryption/encryptor.go"
    click DE "https://github.com/zorrat/fuzzy-private-entity-set-intersection/blob/main/encryption/decryptor.go"
    click SE "https://github.com/zorrat/fuzzy-private-entity-set-intersection/blob/main/serialization/serializer.go"
    click UT "https://github.com/zorrat/fuzzy-private-entity-set-intersection/blob/main/utils/utils.go"
    click OR "https://github.com/zorrat/fuzzy-private-entity-set-intersection/blob/main/fpsi.go"
    click EX "https://github.com/zorrat/fuzzy-private-entity-set-intersection/blob/main/examples/examples.go"
    click CI "https://github.com/zorrat/fuzzy-private-entity-set-intersection/blob/main/.github/workflows/main.yml"

    %% Styles
    classDef blue fill:#CCE5FF,stroke:#2E6BF1;
    classDef green fill:#CCFFCC,stroke:#28A745;
    classDef orange fill:#FFE5CC,stroke:#FF8C00;
    classDef red fill:#FFCCCC,stroke:#DC3545;
    classDef purple fill:#E5CCFF,stroke:#6F42C1;
    classDef gray fill:#F0F0F0,stroke:#6C757D;
    classDef cloud fill:#F8D7DA,stroke:#DC3545,stroke-dasharray:5 5;
    classDef ci fill:#FFF3CD,stroke:#856404,stroke-dasharray:2 2;
```
