How p4zaa/dataprepkit works
Project type and purpose
• DataPrepKit is an open‐source Python library for data preprocessing.
• It bundles two domain‐specific subpackages:
• TabularPrepKit – helpers & Polars integration for tabular data
• ThaiTextPrepKit – typo patterns & Polars integration for Thai text
File structure
• Top‐level files
– README.md (project description & usage)
– setup.py (packaging)
– Jupyter notebooks & scratch.py (exploration/demos)
• TabularPrepKit/
– tablekit.py (public façade API)
– polars_tablekit.py (Polars adapter)
– helper_functions.py (core clean/transform logic)
• ThaiTextPrepKit/
– pretextkit.py (public façade API)
– polars_pretextkit.py (Polars adapter)
– typo_patterns.py, _typo_patterns.py (pattern definitions)
Extract README insights
• Installs via pip
• Usage: import TabularPrepKit.tablekit and ThaiTextPrepKit.pretextkit, then call on Polars DataFrame/Series (e.g. .map_batches)
• Core dependency: Polars
a. Main components
– Client (script or Jupyter notebook)
– TabularPrepKit package
• API layer (tablekit.py)
• Core logic (helper_functions.py)
• Polars adapter (polars_tablekit.py)
– ThaiTextPrepKit package
• API layer (pretextkit.py)
• Core logic (typo_patterns.py/_typo_patterns.py)
• Polars adapter (polars_pretextkit.py)
– External dependencies: Polars, Python runtime, pip registry
b. Relationships & data flow
1. Client → TabularPrepKit.tablekit → polars_tablekit → Polars DataFrame
2. Client → ThaiTextPrepKit.pretextkit → polars_pretextkit → Polars Series
3. Core logic modules supply rules and helpers to adapters
4. setup.py packages everything for pip distribution
c. Architectural patterns & principles
– Modular packaging by domain (tabular vs text)
– Facade/API layer for user‐facing methods
– Adapter layer to isolate external library (Polars)
– Single responsibility: pattern definitions separated from API
d. Key technologies
– Python 3.x, Polars, pip, Jupyter