How arashdn/tabulax works
Project type and purpose
• This is a research‐oriented, command‐line data‐processing pipeline (“TabulaX”) that automates multi‐class table transformations using large language models (LLMs).
• It ingests labeled CSV datasets, classifies each table’s transformation type (e.g., algorithmic, string, numeric, general), invokes an LLM (either OpenAI’s GPT family or a local LLaMA 3 served via vLLM) to generate the transformed output, and evaluates results against ground truth.
File structure
• Top‐level directories:
• data/ — contains three datasets (AutoJoin, FlashFill, DataXFormer) organized by table, each with source.csv, target.csv, ground truth.
• src/LLM_pipeline/ — the code under three modules:
– classifier/ — input classifier scripts (gpt_classifier.py, llama_classifier.py), prompt templates, utilities, metrics reporter
– transformers/ — transformation functions grouped by class (basic, algorithmic, general, numeric, string) with their prompt templates
– util/ — dataset loader/sampler and join‐evaluation utilities
• Orchestration script: run_pipeline.py invokes classification and transformation in sequence.
• Config files: environment.yml, requirements.txt, plus several cache‐path constants in the code (PROMPT_CACHE_PATH, ED_CACHE_PATH).
Extract architectural clues from the README
• Two phases: classification of input tables, then transformation of each table.
• Dual model support: OpenAI API vs. local LLaMA via vLLM.
• Caching layers for prompt generation and edit‐distance computations.
• Pluggable “matching” and “classification” strategies configured at runtime.
Essential details
• Run_pipeline.py as the orchestrator at the center.
• Configuration inputs (MODEL_NAME, EXAMPLE_SIZE, MATCHING_TYPE, CLASSIFICATION_TYPE) feeding into the orchestrator.
• Detail prompt paths (classifier_prompts/*, transformers/prompts/*) linking into classifier and transformer modules.
• Evaluation outputs: performance_report.csv, metrics printed by report_metrics.py and JoinEval.