How pierregr7/data_factory_x works
Data_Factory is a CSV profiling and dimensional-modeling tool. Its primary workflow loads CSV tables, profiles them, discovers likely joins, chooses a fact-and-dimension schema, and materializes an enriched DuckDB table with run summaries. The first_py/ scripts provide separate, simpler profiling, cleaning, join-discovery, and schema-proposal utilities; the available excerpts do not show them integrated with the main runner. Edges for the main workflow are grounded in dfx_run.py; standalone utility relationships are omitted where no call is shown.
Main entry points: dfx_run.py Pipeline runner · outputs DuckDB model
Components 20
Main pipeline
- Pipeline runner
dfx_run.py - CSV loader
dfx_run.py - Table profiler
dfx_run.py
Schema modeling
- Join discovery
dfx_run.py - Schema selection
dfx_run.py - Fact-dimension scoring
dfx_run.py - DuckDB builder
dfx_run.py
Standalone utilities
- CSV profiling
first_py/profiling.py - Multi-CSV profiling
first_py/multi_csv_profiling.py - Cleaning suggestions
first_py/propose_cleaning.py - Cleaning application
first_py/apply_cleaning.py - Join utility
first_py/find_joins.py - Fact-dim utility
first_py/fact_dim_guess.py - Schema proposal
first_py/star_schema_proposal.py - CSV display
first_py/read_csv.py
Run artifacts
- Run directory
dfx_run.py - DuckDB model
outputs - Run summary
outputs
Other components
- Operator
- CSV inputs
Connections 13
- Operator → Pipeline runner: runs
dfx_run.py - Pipeline runner → Run directory: creates
dfx_run.py - Pipeline runner → CSV loader: calls
dfx_run.py - CSV loader → CSV inputs: reads
dfx_run.py - Pipeline runner → Table profiler: profiles
dfx_run.py - Pipeline runner → Join discovery: discovers joins
dfx_run.py - Pipeline runner → Schema selection: selects schema
dfx_run.py - Schema selection → Fact-dimension scoring: scores tables
dfx_run.py - Pipeline runner → DuckDB builder: builds model
dfx_run.py - DuckDB builder → DuckDB model: writes
dfx_run.py - Pipeline runner → Run summary: writes
dfx_run.py - Run directory → DuckDB model: contains
- Run directory → Run summary: contains