# systemhubtahir/data-cleaning-assistant architecture

> Architecture diagram of the GitHub repository systemhubtahir/data-cleaning-assistant, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/systemhubtahir/data-cleaning-assistant
- Repository: https://github.com/systemhubtahir/data-cleaning-assistant

## Overview

Data Cleaning Assistant profiles a customer-call dataset, sends the profile—not raw rows—to an LLM for suggestions, and leaves decisions to a human before fixes are applied. Its stages are profiling, AI analysis, manual review, and cleaning. The reviewed suggestions are not automatically consumed by the apply script: the documented fixes are implemented directly in that script. The graph reflects documented workflow and sampled file operations; unsampled files are omitted.

Main entry points: [`clean_with_ai.py`](https://github.com/systemhubtahir/data-cleaning-assistant/blob/main/clean_with_ai.py) (AI analysis), [`profile_data.py`](https://github.com/systemhubtahir/data-cleaning-assistant/blob/main/profile_data.py) (Profile data), [`apply_fixes.py`](https://github.com/systemhubtahir/data-cleaning-assistant/blob/main/apply_fixes.py) (Apply fixes), [`reports/ai_suggestions.txt`](https://github.com/systemhubtahir/data-cleaning-assistant/blob/main/reports/ai_suggestions.txt) (AI suggestions)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_profiling["Dataset profiling"]
  node_raw_dataset["Raw dataset"]
  node_profile_script["Profile data<br/>[profile_data.py]"]
end

subgraph group_analysis["AI suggestions"]
  node_profile_report["Profile report<br/>[profile.json]"]
  node_ai_script["AI analysis<br/>[clean_with_ai.py]"]
  node_suggestions["AI suggestions<br/>[ai_suggestions.txt]"]
end

subgraph group_cleaning["Manual cleaning"]
  node_review["Review suggestions"]
  node_apply_script["Apply fixes<br/>[apply_fixes.py]"]
  node_cleaned_dataset["Cleaned dataset"]
end

node_user(("Human operator"))
node_inference_api["Hugging Face API"]

node_user -->|"runs"| node_profile_script
node_profile_script -->|"reads"| node_raw_dataset
node_profile_script -->|"writes"| node_profile_report
node_user -->|"runs"| node_ai_script
node_ai_script -->|"reads"| node_profile_report
node_ai_script -->|"requests suggestions"| node_inference_api
node_ai_script -->|"writes"| node_suggestions
node_user -->|"reviews"| node_review
node_suggestions -->|"informs decisions"| node_review
node_user -->|"runs approved fixes"| node_apply_script
node_apply_script -->|"reads"| node_raw_dataset
node_apply_script -->|"writes"| node_cleaned_dataset

click node_raw_dataset "https://github.com/systemhubtahir/data-cleaning-assistant/blob/main/data/Customer_Call_List.xlsx"
click node_profile_script "https://github.com/systemhubtahir/data-cleaning-assistant/blob/main/profile_data.py"
click node_profile_report "https://github.com/systemhubtahir/data-cleaning-assistant/blob/main/reports/profile.json"
click node_ai_script "https://github.com/systemhubtahir/data-cleaning-assistant/blob/main/clean_with_ai.py"
click node_suggestions "https://github.com/systemhubtahir/data-cleaning-assistant/blob/main/reports/ai_suggestions.txt"
click node_apply_script "https://github.com/systemhubtahir/data-cleaning-assistant/blob/main/apply_fixes.py"
click node_cleaned_dataset "https://github.com/systemhubtahir/data-cleaning-assistant/blob/main/data/Cleaned_Customer_Call_List.csv"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_raw_dataset,node_profile_script toneBlue
class node_profile_report,node_ai_script,node_suggestions toneAmber
class node_review,node_apply_script,node_cleaned_dataset,node_inference_api toneMint
class node_user toneIndigo
```

## Components

### Dataset profiling

- [Raw dataset](https://github.com/systemhubtahir/data-cleaning-assistant/blob/main/data/Customer_Call_List.xlsx): `data/Customer_Call_List.xlsx`
- [Profile data](https://github.com/systemhubtahir/data-cleaning-assistant/blob/main/profile_data.py): `profile_data.py`

### AI suggestions

- [Profile report](https://github.com/systemhubtahir/data-cleaning-assistant/blob/main/reports/profile.json): `reports/profile.json`
- [AI analysis](https://github.com/systemhubtahir/data-cleaning-assistant/blob/main/clean_with_ai.py): `clean_with_ai.py`
- [AI suggestions](https://github.com/systemhubtahir/data-cleaning-assistant/blob/main/reports/ai_suggestions.txt): `reports/ai_suggestions.txt`

### Manual cleaning

- **Review suggestions**
- [Apply fixes](https://github.com/systemhubtahir/data-cleaning-assistant/blob/main/apply_fixes.py): `apply_fixes.py`
- [Cleaned dataset](https://github.com/systemhubtahir/data-cleaning-assistant/blob/main/data/Cleaned_Customer_Call_List.csv): `data/Cleaned_Customer_Call_List.csv`

### Other components

- **Human operator**
- **Hugging Face API**

## Connections

- Human operator → Profile data: runs (evidence: `README.md`)
- Profile data → Raw dataset: reads (evidence: `profile_data.py`)
- Profile data → Profile report: writes (evidence: `profile_data.py`)
- Human operator → AI analysis: runs (evidence: `README.md`)
- AI analysis → Profile report: reads (evidence: `clean_with_ai.py`)
- AI analysis → Hugging Face API: requests suggestions (evidence: `clean_with_ai.py`)
- AI analysis → AI suggestions: writes (evidence: `clean_with_ai.py`)
- Human operator → Review suggestions: reviews (evidence: `README.md`)
- AI suggestions → Review suggestions: informs decisions (evidence: `README.md`)
- Human operator → Apply fixes: runs approved fixes (evidence: `README.md`)
- Apply fixes → Raw dataset: reads (evidence: `apply_fixes.py`)
- Apply fixes → Cleaned dataset: writes (evidence: `apply_fixes.py`)
