# agishaalbert/personal-data-detection-llms architecture

> Architecture diagram of the GitHub repository agishaalbert/personal-data-detection-llms, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/agishaalbert/personal-data-detection-llms
- Repository: https://github.com/agishaalbert/personal-data-detection-llms

## Overview

This repository is a research project for detecting personal and other sensitive data in structured datasets, with CASSED, GPT-based detection, and Presidio experiments represented in the tree. A CASSED workflow prepares column values and labels as sentence-classification data, then trains and evaluates a transformer-backed classifier. GPT provides direct and batch OpenAI requests; prediction artifacts and evaluation notebooks support comparisons across methods. The source excerpts establish the CASSED training and GPT request behavior, but not the wiring of the Presidio notebooks or most stored results, so those relationships are omitted or marked unverified.

Main entry points: [`CASSED-main/train_classifier.py`](https://github.com/agishaalbert/personal-data-detection-llms/blob/main/CASSED-main/train_classifier.py) (Train and evaluate), [`CASSED-main/prepare_data.py`](https://github.com/agishaalbert/personal-data-detection-llms/blob/main/CASSED-main/prepare_data.py) (Prepare sentences), [`Personal-Detection/GPT/gpt.py`](https://github.com/agishaalbert/personal-data-detection-llms/blob/main/Personal-Detection/GPT/gpt.py) (GPT request client), [`Personal-Detection/comparison`](https://github.com/agishaalbert/personal-data-detection-llms/tree/main/Personal-Detection/comparison) (Evaluation notebooks)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_cassed["CASSED classifier"]
  node_prepare["Prepare sentences<br/>[prepare_data.py]"]
  node_config["Run configuration<br/>[run_config.py]"]
  node_corpus["Classification corpus"]
  node_sentence_parts["Sentence splitting"]
  node_classifier["Text classifier"]
  node_training["Train and evaluate"]
end

subgraph group_gpt["GPT detection"]
  node_gpt_client["GPT request client<br/>[gpt.py]"]
  node_openai{{"OpenAI API"}}
end

subgraph group_data["Data and evaluation"]
  node_structured_data[("Structured datasets")]
  node_presidio["Presidio experiments"]
  node_predictions["Prediction artifacts"]
  node_evaluation["Evaluation notebooks"]
end

node_researcher(("Researcher"))

node_researcher -->|"prepares data"| node_prepare
node_structured_data -->|"reads CSVs"| node_prepare
node_prepare -.->|"creates training files"| node_corpus
node_config -->|"sets data paths"| node_prepare
node_config -->|"sets model paths"| node_training
node_researcher -->|"runs training"| node_training
node_training -->|"loads corpus"| node_corpus
node_training -->|"constructs classifier"| node_classifier
node_classifier -->|"splits sentences"| node_sentence_parts
node_training -.->|"evaluates model"| node_evaluation
node_researcher -->|"submits prompts"| node_gpt_client
node_gpt_client -->|"requests completions"| node_openai
node_gpt_client -->|"writes batch results"| node_predictions
node_presidio -.->|"compared in notebooks"| node_evaluation
node_predictions -.->|"evaluated in notebooks"| node_evaluation

click node_structured_data "https://github.com/agishaalbert/personal-data-detection-llms/tree/main/Personal-Detection/datasets"
click node_prepare "https://github.com/agishaalbert/personal-data-detection-llms/blob/main/CASSED-main/prepare_data.py"
click node_config "https://github.com/agishaalbert/personal-data-detection-llms/blob/main/CASSED-main/config/run_config.py"
click node_corpus "https://github.com/agishaalbert/personal-data-detection-llms/blob/main/CASSED-main/train_classifier.py"
click node_sentence_parts "https://github.com/agishaalbert/personal-data-detection-llms/blob/main/CASSED-main/extended_flair/extended_sentence.py"
click node_classifier "https://github.com/agishaalbert/personal-data-detection-llms/blob/main/CASSED-main/extended_flair/extended_text_classifier.py"
click node_training "https://github.com/agishaalbert/personal-data-detection-llms/blob/main/CASSED-main/train_classifier.py"
click node_presidio "https://github.com/agishaalbert/personal-data-detection-llms/tree/main/Personal-Detection/Presidio"
click node_gpt_client "https://github.com/agishaalbert/personal-data-detection-llms/blob/main/Personal-Detection/GPT/gpt.py"
click node_predictions "https://github.com/agishaalbert/personal-data-detection-llms/tree/main/Personal-Detection/GPT"
click node_evaluation "https://github.com/agishaalbert/personal-data-detection-llms/tree/main/Personal-Detection/comparison"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_prepare,node_config,node_corpus,node_sentence_parts,node_classifier,node_training toneBlue
class node_gpt_client,node_openai toneAmber
class node_structured_data,node_presidio,node_predictions,node_evaluation toneMint
class node_researcher toneIndigo
```

## Components

### CASSED classifier

- [Prepare sentences](https://github.com/agishaalbert/personal-data-detection-llms/blob/main/CASSED-main/prepare_data.py): `CASSED-main/prepare_data.py`
- [Run configuration](https://github.com/agishaalbert/personal-data-detection-llms/blob/main/CASSED-main/config/run_config.py): `CASSED-main/config/run_config.py`
- [Classification corpus](https://github.com/agishaalbert/personal-data-detection-llms/blob/main/CASSED-main/train_classifier.py): `CASSED-main/train_classifier.py`
- [Sentence splitting](https://github.com/agishaalbert/personal-data-detection-llms/blob/main/CASSED-main/extended_flair/extended_sentence.py): `CASSED-main/extended_flair/extended_sentence.py`
- [Text classifier](https://github.com/agishaalbert/personal-data-detection-llms/blob/main/CASSED-main/extended_flair/extended_text_classifier.py): `CASSED-main/extended_flair/extended_text_classifier.py`
- [Train and evaluate](https://github.com/agishaalbert/personal-data-detection-llms/blob/main/CASSED-main/train_classifier.py): `CASSED-main/train_classifier.py`

### GPT detection

- [GPT request client](https://github.com/agishaalbert/personal-data-detection-llms/blob/main/Personal-Detection/GPT/gpt.py): `Personal-Detection/GPT/gpt.py`
- **OpenAI API**

### Data and evaluation

- [Structured datasets](https://github.com/agishaalbert/personal-data-detection-llms/tree/main/Personal-Detection/datasets): `Personal-Detection/datasets`
- [Presidio experiments](https://github.com/agishaalbert/personal-data-detection-llms/tree/main/Personal-Detection/Presidio): `Personal-Detection/Presidio`
- [Prediction artifacts](https://github.com/agishaalbert/personal-data-detection-llms/tree/main/Personal-Detection/GPT): `Personal-Detection/GPT`
- [Evaluation notebooks](https://github.com/agishaalbert/personal-data-detection-llms/tree/main/Personal-Detection/comparison): `Personal-Detection/comparison`

### Other components

- **Researcher**

## Connections

- Researcher → Prepare sentences: prepares data (evidence: `CASSED-main/prepare_data.py`)
- Structured datasets → Prepare sentences: reads CSVs (evidence: `CASSED-main/prepare_data.py`)
- Prepare sentences → Classification corpus: creates training files
- Run configuration → Prepare sentences: sets data paths (evidence: `CASSED-main/prepare_data.py`)
- Run configuration → Train and evaluate: sets model paths (evidence: `CASSED-main/train_classifier.py`)
- Researcher → Train and evaluate: runs training (evidence: `CASSED-main/train_classifier.py`)
- Train and evaluate → Classification corpus: loads corpus (evidence: `CASSED-main/train_classifier.py`)
- Train and evaluate → Text classifier: constructs classifier (evidence: `CASSED-main/train_classifier.py`)
- Text classifier → Sentence splitting: splits sentences (evidence: `CASSED-main/extended_flair/extended_text_classifier.py`)
- Train and evaluate → Evaluation notebooks: evaluates model
- Researcher → GPT request client: submits prompts (evidence: `Personal-Detection/GPT/gpt.py`)
- GPT request client → OpenAI API: requests completions (evidence: `Personal-Detection/GPT/gpt.py`)
- GPT request client → Prediction artifacts: writes batch results (evidence: `Personal-Detection/GPT/gpt.py`)
- Presidio experiments → Evaluation notebooks: compared in notebooks
- Prediction artifacts → Evaluation notebooks: evaluated in notebooks
