# p4zaa/dataprepkit architecture

> Architecture diagram of the GitHub repository p4zaa/dataprepkit, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/p4zaa/dataprepkit
- Repository: https://github.com/p4zaa/dataprepkit

## Overview

##### Project type and purpose
- DataPrepKit is an open‐source Python library for data preprocessing.
- It bundles two domain‐specific subpackages:
- TabularPrepKit – helpers & Polars integration for tabular data
- ThaiTextPrepKit – typo patterns & Polars integration for Thai text

##### File structure
- Top‐level files
– README.md (project description & usage)
– setup.py (packaging)
– Jupyter notebooks & scratch.py (exploration/demos)
- TabularPrepKit/
– tablekit.py (public façade API)
– polars_tablekit.py (Polars adapter)
– helper_functions.py (core clean/transform logic)
- ThaiTextPrepKit/
– pretextkit.py (public façade API)
– polars_pretextkit.py (Polars adapter)
– typo_patterns.py, _typo_patterns.py (pattern definitions)

##### Extract README insights
- Installs via pip
- Usage: import TabularPrepKit.tablekit and ThaiTextPrepKit.pretextkit, then call on Polars DataFrame/Series (e.g. .map_batches)
- Core dependency: Polars

a. Main components
– Client (script or Jupyter notebook)
– TabularPrepKit package
- API layer (tablekit.py)
- Core logic (helper_functions.py)
- Polars adapter (polars_tablekit.py)
– ThaiTextPrepKit package
- API layer (pretextkit.py)
- Core logic (typo_patterns.py/_typo_patterns.py)
- Polars adapter (polars_pretextkit.py)
– External dependencies: Polars, Python runtime, pip registry
b. Relationships & data flow
1. Client → TabularPrepKit.tablekit → polars_tablekit → Polars DataFrame
2. Client → ThaiTextPrepKit.pretextkit → polars_pretextkit → Polars Series
3. Core logic modules supply rules and helpers to adapters
4. setup.py packages everything for pip distribution
c. Architectural patterns & principles
– Modular packaging by domain (tabular vs text)
– Facade/API layer for user‐facing methods
– Adapter layer to isolate external library (Polars)
– Single responsibility: pattern definitions separated from API
d. Key technologies
– Python 3.x, Polars, pip, Jupyter

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client
    Client["Client (Script / Notebook)"]

    %% Packaging
    Setup["setup.py"]:::packaging
    README["README.md"]:::packaging
    PipRegistry["PyPI / pip Registry"]:::ext

    %% API Layer
    subgraph "API Layer"
        direction TB
        TableAPI["TabularPrepKit/tablekit.py"]:::api
        TextAPI["ThaiTextPrepKit/pretextkit.py"]:::api
    end

    %% Core Logic Layer
    subgraph "Core Logic Layer"
        direction TB
        Helpers["TabularPrepKit/helper_functions.py"]:::logic
        Typo1["ThaiTextPrepKit/typo_patterns.py"]:::logic
        Typo2["ThaiTextPrepKit/_typo_patterns.py"]:::logic
    end

    %% Adapter Layer
    subgraph "Adapter Layer"
        direction TB
        TableAdapter["TabularPrepKit/polars_tablekit.py"]:::adapter
        TextAdapter["ThaiTextPrepKit/polars_pretextkit.py"]:::adapter
    end

    %% External Dependencies
    subgraph "External Dependencies"
        direction TB
        Polars["Polars Library"]:::ext
        Python["Python Runtime"]:::ext
    end

    %% Data Flows
    Client -->|"map_batches()"| TableAPI
    Client -->|"preprocess_text_polars()"| TextAPI

    TableAPI --> TableAdapter
    TextAPI --> TextAdapter

    TableAdapter --> Polars
    TextAdapter --> Polars

    Helpers --> TableAdapter
    Typo1 --> TextAdapter
    Typo2 --> TextAdapter

    Setup --> PipRegistry
    README --> Client

    %% Click Events
    click README "https://github.com/p4zaa/dataprepkit/blob/main/README.md"
    click Setup "https://github.com/p4zaa/dataprepkit/blob/main/setup.py"
    click TableAPI "https://github.com/p4zaa/dataprepkit/blob/main/TabularPrepKit/tablekit.py"
    click Helpers "https://github.com/p4zaa/dataprepkit/blob/main/TabularPrepKit/helper_functions.py"
    click TableAdapter "https://github.com/p4zaa/dataprepkit/blob/main/TabularPrepKit/polars_tablekit.py"
    click TextAPI "https://github.com/p4zaa/dataprepkit/blob/main/ThaiTextPrepKit/pretextkit.py"
    click Typo1 "https://github.com/p4zaa/dataprepkit/blob/main/ThaiTextPrepKit/typo_patterns.py"
    click Typo2 "https://github.com/p4zaa/dataprepkit/blob/main/ThaiTextPrepKit/_typo_patterns.py"
    click TextAdapter "https://github.com/p4zaa/dataprepkit/blob/main/ThaiTextPrepKit/polars_pretextkit.py"

    %% Styles
    classDef api fill:#AED6F1,stroke:#2980B9,color:#1B4F72;
    classDef logic fill:#A9DFBF,stroke:#27AE60,color:#1D8348,stroke-dasharray: 5 5;
    classDef adapter fill:#F9E79F,stroke:#B7950B,color:#7D6608;
    classDef ext fill:#D5DBDB,stroke:#566573,color:#2C3E50,stroke-dasharray: 2 2;
    classDef packaging fill:#E8DAEF,stroke:#9B59B6,color:#4A235A,stroke-dasharray: 5 2;
```
