# ankudat/thesis architecture

> Architecture diagram of the GitHub repository ankudat/thesis, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ankudat/thesis
- Repository: https://github.com/ankudat/thesis

## Overview

This repository is a thesis research project comparing German financial-text anonymization pipelines, with a Flask demonstrator as a user-facing companion. Its main workflow is synthetic-note generation, validation and annotation, shared dataset splits, independent anonymization experiments, then evaluation of detection, leakage, semantic preservation and re-identification risk. The README documents multiple model families and an external API baseline; those are represented as parallel approaches, not a single processing chain. Source excerpts are partial, so the graph omits uncertain internal calls and treats documented but unsampled capabilities at their supported directory paths.

Main entry points: [`results`](https://github.com/ankudat/thesis/tree/main/results) (Pipeline outputs), [`data/splits`](https://github.com/ankudat/thesis/tree/main/data/splits) (Train/dev/test), [`src/data_generation/validate_data.py`](https://github.com/ankudat/thesis/blob/main/src/data_generation/validate_data.py) (Validate notes), [`src/anonymization/datenwertschoepfung_baseline.py`](https://github.com/ankudat/thesis/blob/main/src/anonymization/datenwertschoepfung_baseline.py) (API baseline)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Dataset preparation"]
  node_generate["Generate notes<br/>[generate_data.py]"]
  node_validate["Validate notes<br/>[validate_data.py]"]
  node_transform["Annotation import"]
  node_splits["Split export"]
end

subgraph group_pipelines["Anonymization approaches"]
  node_classical["spaCy + BERT"]
  node_presidio["Presidio"]
  node_bert["Fine-tuned BERT"]
  node_tagreplace["LLM tag-replace"]
  node_llmtrain["QLoRA fine-tune"]
  node_rewrite["Prompt rewriting"]
  node_api["API baseline"]
end

subgraph group_evaluation["Evaluation"]
  node_nermetrics["NER evaluation"]
  node_semantic["Semantic metrics"]
  node_leakage["Leakage analysis"]
  node_judge["Judge and attack"]
  node_misses["Miss analysis"]
end

subgraph group_demo["Web demonstrator"]
  node_webdemo["Flask demo<br/>[app.py]"]
end

subgraph group_state["Data and results"]
  node_rawdata["Raw notes"]
  node_processed["Validated data"]
  node_annotated["Annotated notes"]
  node_splitdata["Train/dev/test"]
  node_predictions["Pipeline outputs"]
end

node_researcher(("Researcher"))
node_gemini{{"Gemini API"}}
node_labelstudio{{"Label Studio"}}
node_anonymizerapi{{"Anonymizer API"}}
node_browser(("Demo user"))

node_researcher -->|"initiates"| node_generate
node_generate -->|"requests generation"| node_gemini
node_generate -->|"writes"| node_rawdata
node_validate -->|"reads"| node_rawdata
node_validate -->|"audits"| node_gemini
node_validate -->|"writes"| node_processed
node_transform -->|"reads"| node_processed
node_transform -->|"prepares import"| node_labelstudio
node_labelstudio -->|"exports annotations"| node_annotated
node_splits -->|"reads"| node_annotated
node_splits -->|"writes"| node_splitdata
node_classical -->|"evaluates data"| node_splitdata
node_presidio -->|"evaluates data"| node_splitdata
node_bert -->|"trains and evaluates"| node_splitdata
node_tagreplace -->|"evaluates data"| node_splitdata
node_llmtrain -->|"trains and evaluates"| node_splitdata
node_rewrite -->|"evaluates data"| node_splitdata
node_api -.->|"evaluates data"| node_splitdata
node_classical -->|"writes outputs"| node_predictions
node_presidio -->|"writes outputs"| node_predictions
node_bert -->|"writes outputs"| node_predictions
node_tagreplace -->|"writes outputs"| node_predictions
node_llmtrain -->|"writes outputs"| node_predictions
node_rewrite -->|"writes outputs"| node_predictions
node_api -.->|"calls"| node_anonymizerapi
node_api -.->|"writes outputs"| node_predictions
node_nermetrics -->|"scores predictions"| node_predictions
node_semantic -->|"measures preservation"| node_predictions
node_leakage -->|"measures leakage"| node_predictions
node_judge -->|"judges outputs"| node_predictions
node_judge -.->|"requests judgments"| node_gemini
node_misses -->|"classifies misses"| node_predictions
node_browser -->|"submits text"| node_webdemo
node_webdemo -->|"returns anonymization"| node_browser

click node_generate "https://github.com/ankudat/thesis/blob/main/src/data_generation/generate_data.py"
click node_rawdata "https://github.com/ankudat/thesis/blob/main/data/raw/german_financial_data_raw.json"
click node_validate "https://github.com/ankudat/thesis/blob/main/src/data_generation/validate_data.py"
click node_processed "https://github.com/ankudat/thesis/blob/main/data/processed/german_financial_data_cleaned.json"
click node_transform "https://github.com/ankudat/thesis/blob/main/src/data_generation/label_studio_transformation.py"
click node_annotated "https://github.com/ankudat/thesis/blob/main/data/label_studio/20260302_Export_Label_Studio_Client_Notes.json"
click node_splits "https://github.com/ankudat/thesis/blob/main/src/data_generation/export_data_splits.py"
click node_splitdata "https://github.com/ankudat/thesis/tree/main/data/splits"
click node_classical "https://github.com/ankudat/thesis/blob/main/src/anonymization/classical_baseline.py"
click node_presidio "https://github.com/ankudat/thesis/blob/main/src/anonymization/presidio_baseline.py"
click node_bert "https://github.com/ankudat/thesis/blob/main/src/anonymization/bert_finetune_ner.py"
click node_tagreplace "https://github.com/ankudat/thesis/blob/main/src/anonymization/llm_tag_and_replace.py"
click node_llmtrain "https://github.com/ankudat/thesis/blob/main/src/anonymization/llm_finetune_ner.py"
click node_rewrite "https://github.com/ankudat/thesis/blob/main/src/anonymization/llm_prompt_anonymize.py"
click node_api "https://github.com/ankudat/thesis/blob/main/src/anonymization/datenwertschoepfung_baseline.py"
click node_predictions "https://github.com/ankudat/thesis/tree/main/results"
click node_nermetrics "https://github.com/ankudat/thesis/blob/main/src/anonymization/evaluation_utils.py"
click node_semantic "https://github.com/ankudat/thesis/blob/main/src/metrics/semantic_preservation.py"
click node_leakage "https://github.com/ankudat/thesis/blob/main/src/metrics/leakage_breakdown.py"
click node_judge "https://github.com/ankudat/thesis/blob/main/src/metrics/llm_judge_gemini.py"
click node_misses "https://github.com/ankudat/thesis/blob/main/src/metrics/classify_ner_misses.py"
click node_webdemo "https://github.com/ankudat/thesis/blob/main/demo/app.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_generate,node_validate,node_transform,node_splits,node_browser toneBlue
class node_classical,node_presidio,node_bert,node_tagreplace,node_llmtrain,node_rewrite,node_api toneAmber
class node_nermetrics,node_semantic,node_leakage,node_judge,node_misses,node_gemini,node_anonymizerapi toneMint
class node_webdemo toneRose
class node_rawdata,node_processed,node_annotated,node_splitdata,node_predictions,node_researcher,node_labelstudio toneIndigo
```

## Components

### Dataset preparation

- [Generate notes](https://github.com/ankudat/thesis/blob/main/src/data_generation/generate_data.py): `src/data_generation/generate_data.py`
- [Validate notes](https://github.com/ankudat/thesis/blob/main/src/data_generation/validate_data.py): `src/data_generation/validate_data.py`
- [Annotation import](https://github.com/ankudat/thesis/blob/main/src/data_generation/label_studio_transformation.py): `src/data_generation/label_studio_transformation.py`
- [Split export](https://github.com/ankudat/thesis/blob/main/src/data_generation/export_data_splits.py): `src/data_generation/export_data_splits.py`

### Anonymization approaches

- [spaCy + BERT](https://github.com/ankudat/thesis/blob/main/src/anonymization/classical_baseline.py): `src/anonymization/classical_baseline.py`
- [Presidio](https://github.com/ankudat/thesis/blob/main/src/anonymization/presidio_baseline.py): `src/anonymization/presidio_baseline.py`
- [Fine-tuned BERT](https://github.com/ankudat/thesis/blob/main/src/anonymization/bert_finetune_ner.py): `src/anonymization/bert_finetune_ner.py`
- [LLM tag-replace](https://github.com/ankudat/thesis/blob/main/src/anonymization/llm_tag_and_replace.py): `src/anonymization/llm_tag_and_replace.py`
- [QLoRA fine-tune](https://github.com/ankudat/thesis/blob/main/src/anonymization/llm_finetune_ner.py): `src/anonymization/llm_finetune_ner.py`
- [Prompt rewriting](https://github.com/ankudat/thesis/blob/main/src/anonymization/llm_prompt_anonymize.py): `src/anonymization/llm_prompt_anonymize.py`
- [API baseline](https://github.com/ankudat/thesis/blob/main/src/anonymization/datenwertschoepfung_baseline.py): `src/anonymization/datenwertschoepfung_baseline.py`

### Evaluation

- [NER evaluation](https://github.com/ankudat/thesis/blob/main/src/anonymization/evaluation_utils.py): `src/anonymization/evaluation_utils.py`
- [Semantic metrics](https://github.com/ankudat/thesis/blob/main/src/metrics/semantic_preservation.py): `src/metrics/semantic_preservation.py`
- [Leakage analysis](https://github.com/ankudat/thesis/blob/main/src/metrics/leakage_breakdown.py): `src/metrics/leakage_breakdown.py`
- [Judge and attack](https://github.com/ankudat/thesis/blob/main/src/metrics/llm_judge_gemini.py): `src/metrics/llm_judge_gemini.py`
- [Miss analysis](https://github.com/ankudat/thesis/blob/main/src/metrics/classify_ner_misses.py): `src/metrics/classify_ner_misses.py`

### Web demonstrator

- [Flask demo](https://github.com/ankudat/thesis/blob/main/demo/app.py): `demo/app.py`

### Data and results

- [Raw notes](https://github.com/ankudat/thesis/blob/main/data/raw/german_financial_data_raw.json): `data/raw/german_financial_data_raw.json`
- [Validated data](https://github.com/ankudat/thesis/blob/main/data/processed/german_financial_data_cleaned.json): `data/processed/german_financial_data_cleaned.json`
- [Annotated notes](https://github.com/ankudat/thesis/blob/main/data/label_studio/20260302_Export_Label_Studio_Client_Notes.json): `data/label_studio/20260302_Export_Label_Studio_Client_Notes.json`
- [Train/dev/test](https://github.com/ankudat/thesis/tree/main/data/splits): `data/splits`
- [Pipeline outputs](https://github.com/ankudat/thesis/tree/main/results): `results`

### Other components

- **Researcher**
- **Gemini API**
- **Label Studio**
- **Anonymizer API**
- **Demo user**

## Connections

- Researcher → Generate notes: initiates
- Generate notes → Gemini API: requests generation
- Generate notes → Raw notes: writes
- Validate notes → Raw notes: reads
- Validate notes → Gemini API: audits
- Validate notes → Validated data: writes
- Annotation import → Validated data: reads
- Annotation import → Label Studio: prepares import
- Label Studio → Annotated notes: exports annotations
- Split export → Annotated notes: reads
- Split export → Train/dev/test: writes
- spaCy + BERT → Train/dev/test: evaluates data
- Presidio → Train/dev/test: evaluates data
- Fine-tuned BERT → Train/dev/test: trains and evaluates
- LLM tag-replace → Train/dev/test: evaluates data
- QLoRA fine-tune → Train/dev/test: trains and evaluates
- Prompt rewriting → Train/dev/test: evaluates data
- API baseline → Train/dev/test: evaluates data
- spaCy + BERT → Pipeline outputs: writes outputs
- Presidio → Pipeline outputs: writes outputs
- Fine-tuned BERT → Pipeline outputs: writes outputs
- LLM tag-replace → Pipeline outputs: writes outputs
- QLoRA fine-tune → Pipeline outputs: writes outputs
- Prompt rewriting → Pipeline outputs: writes outputs
- API baseline → Anonymizer API: calls
- API baseline → Pipeline outputs: writes outputs
- NER evaluation → Pipeline outputs: scores predictions
- Semantic metrics → Pipeline outputs: measures preservation
- Leakage analysis → Pipeline outputs: measures leakage
- Judge and attack → Pipeline outputs: judges outputs
- Judge and attack → Gemini API: requests judgments
- Miss analysis → Pipeline outputs: classifies misses
- Demo user → Flask demo: submits text
- Flask demo → Demo user: returns anonymization