# soda-inria/tabicl architecture

> Architecture diagram of the GitHub repository soda-inria/tabicl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/soda-inria/tabicl
- Repository: https://github.com/soda-inria/tabicl

## Overview

1. **Package shape and public entrypoints**
   - This is a Python library, not a service. The public API is exposed from `src/tabicl/__init__.py` and is centered on a few user-facing classes: `TabICLClassifier`, `TabICLRegressor`, `FinetunedTabICLClassifier`, `FinetunedTabICLRegressor`, `TabICLForecaster`, plus SHAP helpers.
   - The repository is organized into internal subpackages for inference/modeling, sklearn integration, finetuning, forecasting, interpretability, and pretraining.

2. **Core inference path: tabular in-context learning**
   - The main runtime flow is `fit(X_train, y_train) -> predict(X_test)` on a pretrained transformer.
   - `fit()` mainly stores and preprocesses the training context; `predict()` performs the actual in-context forward pass through the model using `(X_train, y_train, X_test)`.
   - This is the central architectural boundary: user tabular data enters, becomes model-ready tensors, then a pretrained transformer emits class probabilities or regression outputs.

3. **Model subsystem**
   - `src/tabicl/_model` contains the Transformer-style architecture and supporting layers: embeddings, attention, encoders, interaction blocks, quantile/distribution heads, rotary embeddings, and learning/inference utilities.
   - `tabicl.py` likely assembles the end-to-end model used by the high-level estimators.
   - `inference.py`, `inference_config.py`, and `kv_cache.py` capture inference-time controls such as checkpoint loading, caching, and memory/offload behavior.
   - This subsystem is where the pretrained checkpoints are consumed.

4. **Preprocessing and tabular normalization**
   - `src/tabicl/_sklearn/preprocessing.py` and related utilities handle the tabular input boundary for both pandas DataFrames and NumPy arrays.
   - The library performs categorical detection/ordinal encoding, missing-value handling, numerical imputation, clipping/outlier handling, scaling/normalization, and feature shuffling for ensemble diversity.
   - This preprocessing layer is important because it bridges heterogeneous user data formats to the model’s fixed tensor expectations.

5. **Scikit-learn compatibility layer**
   - `src/tabicl/_sklearn` wraps the model in sklearn-style estimators and ensures it can be used in pipelines.
   - `base.py`, `classifier.py`, `regressor.py`, and `sklearn_utils.py` provide the fit/predict API, parameter handling, and serialization behavior expected from sklearn estimators.
   - This is the main integration boundary for external ML workflows; the repo is designed to plug into sklearn pipelines rather than expose a custom training framework only.

6. **Checkpointing, persistence, and runtime configuration**
   - The README’s `save()` / `load()` behavior maps to estimator-level persistence, with options to save model weights, training data, and KV cache.
   - Model weights can be downloaded from Hugging Face or loaded from a local `model_path`; this makes pretrained checkpoints an external dependency of the inference path.
   - Runtime knobs such as device selection, mixed precision, Flash Attention 3, offloading, and batch sizing all affect inference execution but do not change the logical model flow.

7. **KV cache and scalability**
   - `kv_cache.py` and the README’s cache option show a distinct performance subsystem: training-context projections can be cached during `fit()` and reused across repeated `predict()` calls.
   - The architecture therefore has two inference modes: fresh full-context inference, and cached-context inference.
   - Offloading support (CPU/disk) is part of the same scalability boundary and matters for large datasets or constrained memory environments.

8. **Fine-tuning subsystem**
   - `src/tabicl/_finetune` provides a separate training loop for adapting the pretrained model to a single downstream dataset.
   - It includes dataset preparation, logging, classifier/regressor wrappers, and a base training workflow.
   - The README indicates this path uses PyTorch training utilities such as AdamW, cosine warmup, gradient clipping, early stopping, and multi-GPU via `torchrun`.
   - Architecturally, this is distinct from zero-shot inference: instead of pure forward-pass prediction, it updates model weights and writes a new checkpoint.

9. **Forecasting subsystem**
   - `src/tabicl/forecast` reuses TabICL as the predictive backend for time series forecasting.
   - It includes a forecasting engine, dispatch logic, a forecaster wrapper, time-series dataframe abstraction, preprocessing, plotting, and transform pipelines for calendar/seasonality features.
   - The flow is: time series data is normalized into a `TimeSeriesDataFrame`, transformed into tabular context, passed through a TabICL regressor-backed forecaster, then optionally visualized.
   - This subsystem is a domain-specific adapter around the core tabular model.

10. **Interpretability subsystem**
    - `src/tabicl/shap` integrates SHAP with TabICL.
    - The key architectural choice is using an all-NaN background row so that feature masking aligns with TabICL’s native missing-value handling instead of substituting arbitrary reference values.
    - This sits outside the core prediction path but consumes a fitted estimator and input samples to produce attribution outputs.

11. **Pretraining / synthetic data generation**
    - `src/tabicl/prior` contains the pretraining-side machinery: dataset generation/loading, prior configuration, hyperparameter sampling, synthetic SCMs (MLP/tree), activation utilities, and regression-to-classification helpers.
    - The scripts in `scripts/train_stage1.sh`, `train_stage2.sh`, and `train_stage3.sh` indicate a staged curriculum-learning pipeline for v1 pretraining.
    - This subsystem is upstream of the shipped checkpoints and is separate from end-user inference.

12. **Training orchestration for pretraining**
    - `src/tabicl/train` provides training configuration, optimization, and run orchestration for the pretraining workflow.
    - It is distinct from `_finetune`: pretraining operates on generated/synthetic data and curriculum stages, while finetuning adapts to a specific real dataset.
    - Together, these two training subsystems form the model-development side of the repository.

13. **Tests, docs, and tutorials as support layers**
    - `tests/` focuses on API compatibility and input handling, especially NumPy inputs and string/categorical data.
    - `tutorials/` demonstrates the main architectural use cases: classification, regression, fine-tuning, interpretability, forecasting, string handling, and unsupervised learning.
    - `docs/` and CI workflows are support infrastructure; they document and validate the library but are not part of runtime execution.

14. **Overall architecture in one line**
    - The repo is a pretrained-transformer tabular learning platform with a shared core model, thin sklearn-compatible estimators, optional fine-tuning, domain adapters for forecasting and SHAP, and a separate synthetic-data pretraining pipeline that produced the distributed checkpoints.

Main entry points: [`src/tabicl/_sklearn/base.py`](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_sklearn/base.py) (Base estimator), [`src/tabicl/_model/tabicl.py`](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_model/tabicl.py) (TabICL model), [`src/tabicl/_model/inference.py`](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_model/inference.py) (Inference), [`src/tabicl/__init__.py`](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/__init__.py) (tabicl)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_public["Public API"]
  node_public_api["tabicl<br/>package API<br/>[__init__.py]"]
  node_clf_api["Classifier API<br/>estimator<br/>[classifier.py]"]
  node_reg_api["Regressor API<br/>estimator<br/>[regressor.py]"]
end

subgraph group_group_core["Core model"]
  node_tabicl_model["TabICL model<br/>transformer core<br/>[tabicl.py]"]
  node_model_layers["Model blocks<br/>layers<br/>[layers.py]"]
  node_model_infer["Inference<br/>runtime<br/>[inference.py]"]
  node_infer_cfg["Inference cfg<br/>config"]
  node_kv_cache[("KV cache<br/>[kv_cache.py]")]
end

subgraph group_group_sklearn["Sklearn layer"]
  node_core_estimator["Base estimator<br/>sklearn base<br/>[base.py]"]
  node_preprocess["Preprocessing<br/>tabular prep<br/>[preprocessing.py]"]
  node_sk_utils["Sklearn utils<br/>compat helpers<br/>[sklearn_utils.py]"]
end

subgraph group_group_finetune["Fine-tuning"]
  node_finetune_base["Finetune base<br/>training loop<br/>[base.py]"]
  node_finetune_data["Finetune data<br/>dataset prep<br/>[data.py]"]
end

subgraph group_group_forecast["Forecasting"]
  node_forecaster["Forecaster<br/>forecast adapter<br/>[_forecaster.py]"]
  node_ts_frame["TS frame<br/>time-series data<br/>[_ts_dataframe.py]"]
end

subgraph group_group_interp["Interpretability"]
  node_shap_api["SHAP<br/>interpretability<br/>[_shap.py]"]
end

subgraph group_group_dev["Model development"]
  node_prior_pipe["Prior pipeline<br/>pretraining data<br/>[_dataset.py]"]
  node_train_run["Train run<br/>pretraining runner<br/>[_run.py]"]
end

subgraph group_group_support["Support"]
  node_unsup["Unsupervised<br/>auxiliary mode<br/>[unsupervised.py]"]
  node_tests_api["Tests<br/>validation<br/>[test_sklearn.py]"]
end

node_public_api -->|"exports"| node_clf_api
node_public_api -->|"exports"| node_reg_api
node_clf_api -->|"uses"| node_core_estimator
node_reg_api -->|"uses"| node_core_estimator
node_core_estimator -->|"prepares data"| node_preprocess
node_core_estimator -->|"depends on"| node_sk_utils
node_core_estimator -->|"loads model"| node_tabicl_model
node_tabicl_model -->|"assembled from"| node_model_layers
node_tabicl_model -->|"served by"| node_model_infer
node_model_infer -->|"configured by"| node_infer_cfg
node_model_infer -->|"reuses"| node_kv_cache
node_core_estimator -.->|"optionally persists"| node_kv_cache
node_finetune_base -->|"trains on"| node_finetune_data
node_finetune_base -->|"updates"| node_tabicl_model
node_forecaster -->|"normalizes"| node_ts_frame
node_forecaster -->|"delegates to"| node_reg_api
node_shap_api -->|"explains"| node_core_estimator
node_prior_pipe -->|"feeds"| node_train_run
node_train_run -->|"produces"| node_tabicl_model
node_unsup -.->|"shares core"| node_core_estimator
node_tests_api -.->|"validates"| node_public_api
node_tests_api -.->|"covers"| node_preprocess

click node_public_api "https://github.com/soda-inria/tabicl/blob/main/src/tabicl/__init__.py"
click node_clf_api "https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_sklearn/classifier.py"
click node_reg_api "https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_sklearn/regressor.py"
click node_core_estimator "https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_sklearn/base.py"
click node_preprocess "https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_sklearn/preprocessing.py"
click node_sk_utils "https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_sklearn/sklearn_utils.py"
click node_tabicl_model "https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_model/tabicl.py"
click node_model_layers "https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_model/layers.py"
click node_model_infer "https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_model/inference.py"
click node_infer_cfg "https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_model/inference_config.py"
click node_kv_cache "https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_model/kv_cache.py"
click node_finetune_base "https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_finetune/base.py"
click node_finetune_data "https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_finetune/data.py"
click node_forecaster "https://github.com/soda-inria/tabicl/blob/main/src/tabicl/forecast/_forecaster.py"
click node_ts_frame "https://github.com/soda-inria/tabicl/blob/main/src/tabicl/forecast/_ts_dataframe.py"
click node_shap_api "https://github.com/soda-inria/tabicl/blob/main/src/tabicl/shap/_shap.py"
click node_prior_pipe "https://github.com/soda-inria/tabicl/blob/main/src/tabicl/prior/_dataset.py"
click node_train_run "https://github.com/soda-inria/tabicl/blob/main/src/tabicl/train/_run.py"
click node_unsup "https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_unsupervised/unsupervised.py"
click node_tests_api "https://github.com/soda-inria/tabicl/blob/main/tests/test_sklearn.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_public_api,node_clf_api,node_reg_api,node_prior_pipe,node_train_run toneBlue
class node_tabicl_model,node_model_layers,node_model_infer,node_infer_cfg,node_kv_cache,node_unsup,node_tests_api toneAmber
class node_core_estimator,node_preprocess,node_sk_utils toneMint
class node_finetune_base,node_finetune_data toneRose
class node_forecaster,node_ts_frame toneIndigo
class node_shap_api toneTeal
```

## Components

### Public API

User-facing imports and estimator entrypoints.

- [tabicl](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/__init__.py): package API · Top-level module exporting the main estimators and helpers. · `src/tabicl/__init__.py`
- [Classifier API](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_sklearn/classifier.py): estimator · High-level tabular classification wrapper. · `src/tabicl/_sklearn/classifier.py`
- [Regressor API](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_sklearn/regressor.py): estimator · High-level tabular regression wrapper. · `src/tabicl/_sklearn/regressor.py`

### Core model

Pretrained transformer, inference, and cache behavior.

- [TabICL model](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_model/tabicl.py): transformer core · End-to-end pretrained tabular in-context learning model assembly. · `src/tabicl/_model/tabicl.py`
- [Model blocks](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_model/layers.py): layers · Embeddings, attention, encoders, interaction blocks, and heads. · `src/tabicl/_model/layers.py`
- [Inference](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_model/inference.py): runtime · Checkpoint loading, forward-pass execution, and inference-time control flow. · `src/tabicl/_model/inference.py`
- [Inference cfg](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_model/inference_config.py): config · Device, precision, offload, and batch-size settings for inference. · `src/tabicl/_model/inference_config.py`
- [KV cache](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_model/kv_cache.py): cache · Cached training-context projections reused across repeated predictions. · `src/tabicl/_model/kv_cache.py`

### Sklearn layer

Tabular preprocessing and sklearn-compatible estimators.

- [Base estimator](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_sklearn/base.py): sklearn base · Shared fit/predict lifecycle, persistence, and model wiring. · `src/tabicl/_sklearn/base.py`
- [Preprocessing](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_sklearn/preprocessing.py): tabular prep · Input normalization for pandas/NumPy, categorical handling, imputation, scaling, and shuffling. · `src/tabicl/_sklearn/preprocessing.py`
- [Sklearn utils](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_sklearn/sklearn_utils.py): compat helpers · Shared estimator utilities and parameter/serialization helpers. · `src/tabicl/_sklearn/sklearn_utils.py`

### Fine-tuning

Dataset adaptation and weight-updating training loop.

- [Finetune base](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_finetune/base.py): training loop · Shared downstream fine-tuning workflow and checkpoint writing. · `src/tabicl/_finetune/base.py`
- [Finetune data](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_finetune/data.py): dataset prep · Dataset preparation and batching for downstream adaptation. · `src/tabicl/_finetune/data.py`

### Forecasting

Time-series adapter built on the core regressor.

- [Forecaster](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/forecast/_forecaster.py): forecast adapter · Time-series wrapper that routes forecasting through TabICL. · `src/tabicl/forecast/_forecaster.py`
- [TS frame](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/forecast/_ts_dataframe.py): time-series data · Time-series dataframe abstraction used by the forecasting pipeline. · `src/tabicl/forecast/_ts_dataframe.py`

### Interpretability

SHAP-based attribution utilities.

- [SHAP](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/shap/_shap.py): interpretability · Feature attribution integration aligned with TabICL missing-value behavior. · `src/tabicl/shap/_shap.py`

### Model development

Synthetic pretraining and training orchestration for released checkpoints.

- [Prior pipeline](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/prior/_dataset.py): pretraining data · Synthetic dataset generation, priors, and curriculum data sources. · `src/tabicl/prior/_dataset.py`
- [Train run](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/train/_run.py): pretraining runner · Pretraining orchestration, optimization, and staged run control. · `src/tabicl/train/_run.py`

### Support

Unsupervised helpers and tests that exercise the public API.

- [Unsupervised](https://github.com/soda-inria/tabicl/blob/main/src/tabicl/_unsupervised/unsupervised.py): auxiliary mode · Unsupervised learning entrypoint built on the shared library. · `src/tabicl/_unsupervised/unsupervised.py`
- [Tests](https://github.com/soda-inria/tabicl/blob/main/tests/test_sklearn.py): validation · Repository tests covering NumPy, strings, and sklearn behavior. · `tests/test_sklearn.py`

## Connections

- tabicl → Classifier API: exports · Top-level module exposes classifier entrypoints.
- tabicl → Regressor API: exports · Top-level module exposes regressor entrypoints.
- Classifier API → Base estimator: uses · Classifier wraps the shared sklearn base lifecycle.
- Regressor API → Base estimator: uses · Regressor wraps the shared sklearn base lifecycle.
- Base estimator → Preprocessing: prepares data · Fit/predict boundaries normalize heterogeneous tabular inputs.
- Base estimator → Sklearn utils: depends on · Shared estimator utilities support parameters and persistence.
- Base estimator → TabICL model: loads model · Estimator binds to the pretrained TabICL backbone.
- TabICL model → Model blocks: assembled from · Model composition is built from embeddings, attention, encoders, and heads.
- TabICL model → Inference: served by · Inference module executes the model with loaded checkpoints.
- Inference → Inference cfg: configured by · Runtime controls select precision, device, batching, and offload.
- Inference → KV cache: reuses · Cached context is reused for repeated predictions.
- Base estimator → KV cache: optionally persists · Estimator-level save/load can include cached context.
- Finetune base → Finetune data: trains on · Fine-tuning uses adapted downstream datasets and loaders.
- Finetune base → TabICL model: updates · Fine-tuning adjusts pretrained weights to a single dataset.
- Forecaster → TS frame: normalizes · Forecasting begins by converting series data into a time-series frame.
- Forecaster → Regressor API: delegates to · Forecasting reuses the regression pathway as its predictive backend.
- SHAP → Base estimator: explains · Attribution utilities consume fitted estimators and samples.
- Prior pipeline → Train run: feeds · Synthetic data and priors drive the staged pretraining workflow.
- Train run → TabICL model: produces · Pretraining writes checkpoints consumed by inference.
- Unsupervised → Base estimator: shares core · Unsupervised mode reuses the common library stack.
- Tests → tabicl: validates · Tests exercise the public API and input handling.
- Tests → Preprocessing: covers · Tests check NumPy and string/categorical preprocessing paths.
