How agishaalbert/personal-data-detection-llms works
This repository is a research project for detecting personal and other sensitive data in structured datasets, with CASSED, GPT-based detection, and Presidio experiments represented in the tree. A CASSED workflow prepares column values and labels as sentence-classification data, then trains and evaluates a transformer-backed classifier. GPT provides direct and batch OpenAI requests; prediction artifacts and evaluation notebooks support comparisons across methods. The source excerpts establish the CASSED training and GPT request behavior, but not the wiring of the Presidio notebooks or most stored results, so those relationships are omitted or marked unverified.
Main entry points: train_classifier.py Train and evaluate · prepare_data.py Prepare sentences · gpt.py GPT request client · comparison Evaluation notebooks
Components 13
CASSED classifier
- Prepare sentences
CASSED-main/prepare_data.py - Run configuration
CASSED-main/config/run_config.py - Classification corpus
CASSED-main/train_classifier.py - Sentence splitting
CASSED-main/extended_flair/extended_sentence.py - Text classifier
CASSED-main/extended_flair/extended_text_classifier.py - Train and evaluate
CASSED-main/train_classifier.py
GPT detection
- GPT request client
Personal-Detection/GPT/gpt.py - OpenAI API
Data and evaluation
- Structured datasets
Personal-Detection/datasets - Presidio experiments
Personal-Detection/Presidio - Prediction artifacts
Personal-Detection/GPT - Evaluation notebooks
Personal-Detection/comparison
Other components
- Researcher
Connections 15
- Researcher → Prepare sentences: prepares data
CASSED-main/prepare_data.py - Structured datasets → Prepare sentences: reads CSVs
CASSED-main/prepare_data.py - Prepare sentences → Classification corpus: creates training files
- Run configuration → Prepare sentences: sets data paths
CASSED-main/prepare_data.py - Run configuration → Train and evaluate: sets model paths
CASSED-main/train_classifier.py - Researcher → Train and evaluate: runs training
CASSED-main/train_classifier.py - Train and evaluate → Classification corpus: loads corpus
CASSED-main/train_classifier.py - Train and evaluate → Text classifier: constructs classifier
CASSED-main/train_classifier.py - Text classifier → Sentence splitting: splits sentences
CASSED-main/extended_flair/extended_text_classifier.py - Train and evaluate → Evaluation notebooks: evaluates model
- Researcher → GPT request client: submits prompts
Personal-Detection/GPT/gpt.py - GPT request client → OpenAI API: requests completions
Personal-Detection/GPT/gpt.py - GPT request client → Prediction artifacts: writes batch results
Personal-Detection/GPT/gpt.py - Presidio experiments → Evaluation notebooks: compared in notebooks
- Prediction artifacts → Evaluation notebooks: evaluated in notebooks