# priorlabs/tabpfn architecture

> Architecture diagram of the GitHub repository priorlabs/tabpfn, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/priorlabs/tabpfn
- Repository: https://github.com/priorlabs/tabpfn

## Overview

##### Project type and purpose
- This is an open-source Python library (package) implementing a foundation model for tabular data (TabPFN).
- It provides both classification (TabPFNClassifier) and regression (TabPFNRegressor) estimators with GPU-optimized inference, plus finetuning utilities, model loading/saving, and preprocessing.
- Your focus: the full “regressor inference” pipeline, from user API call to final numeric predictions.

##### File structure
- Top-level directories:
- src/tabpfn – core Python package
– base.py, classifier.py, regressor.py – high-level Estimator classes
– inference.py – orchestrates predict/predict_proba logic
– preprocessing.py – feature transformation utilities
– model/ – transformer architecture implementation (encoders, attention, MLP, distributions, loading)
- scripts/download_all_models.py – fetches model checkpoint files into the cache
- examples/ – sample code showing usage for regression and classification
- tests/ – unit tests covering interface, model internals, preprocessing, scripts
- key config: src/tabpfn/config.py, constants.py; model configuration in src/tabpfn/model/config.py
- external dependencies: PyTorch, CUDA, HuggingFace model storage

##### Extract README insights
- The Regressor class uses the same Transformer backbone as the classifier, outputting continuous predictions.
- Model weights auto-download or can be loaded from local cache via environment variable or explicit model_path.
- Inference speed is vital—most logic lives in inference.py, model/loading.py, and the model subpackage.

a. Components
1. Client Code (User Script / Notebook)
- Calls TabPFNRegressor.fit() or .predict()
2. API Layer (src/tabpfn/regressor.py)
- TabPFNRegressor class wraps base Estimator logic
3. Inference Orchestrator (src/tabpfn/inference.py)
- Accepts raw X, y; routes through preprocessing & model
4. Preprocessing Engine (src/tabpfn/preprocessing.py + model/preprocessing.py)
- Feature scaling, encoding, missing-value handling, batched slicing
5. Model Loader (src/tabpfn/model/loading.py)
- Downloads or loads .ckpt weights; constructs Transformer with config
6. Transformer Core (src/tabpfn/model/transformer.py + attention, mlp, encoders, memory)
- Multi-head attention, positional encodings, MLP stacks
7. Output Distribution (src/tabpfn/model/bar_distribution.py)
- Converts final logits into continuous regression outputs
8. Postprocessing (in inference.py)
- Aggregates batched outputs, applies any smoothing or ensemble logic
9. Model Cache / External Storage
- Local cache directory (~/.cache/tabpfn/) or HuggingFace remote
10. Optional: GPU Runtime (CUDA) vs. CPU fallback

b. Relationships & data flow
- Control flow: Model Loader invoked once per session; Preprocessing & Core inference loop per batch

##### Architectural patterns & design principles
- Modular layering: clear separation between API, orchestration, preprocessing, model internals
- Lazy loading / caching of heavy assets (model weights)
- Single Responsibility: each submodule (encoders, attention, distribution) focuses on one aspect
- Batch-first tensor processing for GPU efficiency

##### Relevant technologies, frameworks, libraries
- PyTorch (tensors, CUDA kernels)
- HuggingFace Hub (optional remote storage)
- Numpy / Pandas for CPU data structures
- Pre-commit, pytest in dev pipeline

##### Be comprehensive but not overcrowded
- Core components listed above.
- Low-level modules (attention, mlp, encoders) under a single “Transformer Core” cluster, with an internal note listing submodules.
- Inference path (Client → API → Orchestrator → Preprocess → Model → Postprocess → Return) is prominent.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Side
    subgraph "User Side"
        direction TB
        Client["User Script / Notebook"]:::data
    end

    %% API & Orchestration
    subgraph "API & Orchestration"
        direction TB
        API["TabPFNRegressor (API Layer)"]:::api
        Base["Base Estimator Logic"]:::api
        Orchestrator["Inference Orchestrator"]:::api
    end

    %% Preprocessing Layer
    subgraph "Preprocessing Layer"
        direction TB
        PreprocUser["Preprocessing (User-side)"]:::process
        PreprocModel["Preprocessing (Model utilities)"]:::process
    end

    %% Model Internals
    subgraph "Model Internals"
        direction TB
        Loader["Model Loader"]:::process

        subgraph "Transformer Core"
            direction TB
            Transformer["Transformer (Entry Point)"]:::process
            FullAttention["Full Attention"]:::process
            MultiHeadAttention["Multi-Head Attention"]:::process
            MLP["Feed-Forward (MLP)"]:::process
            Encoders["Positional / Feature Encoders"]:::process
            Memory["Memory / Caching Layers"]:::process
            LayerUtil["Layer Utilities"]:::process
        end

        Dist["Output Distribution"]:::process
    end

    %% Postprocessing & Return
    subgraph "Postprocessing & Return"
        direction TB
        Postproc["Postprocessing Logic"]:::api
        Output["Predictions (ŷ)"]:::data
    end

    %% External Resources
    subgraph "External Resources"
        direction TB
        Cache[(Model Cache / HF)]:::external
        Script["Download Script"]:::external
        GlobalConfig["Global Config"]:::external
        ModelConfig["Model Config"]:::external
        Constants["Constants"]:::external
    end

    %% Connections
    Client -->|"calls predict()"| API
    API -->|"uses"| Base
    API -->|"calls"| Orchestrator
    Orchestrator -->|"raw X, y"| PreprocUser
    PreprocUser -->|"transformed X"| PreprocModel
    PreprocModel -->|"preprocessed batch"| Orchestrator
    Orchestrator -->|"load_checkpoint()"| Loader
    Loader -->|"loads weights"| Cache
    Script -->|"downloads to"| Cache
    Loader -->|"builds model"| Transformer
    Transformer --> FullAttention
    Transformer --> MultiHeadAttention
    Transformer --> MLP
    Transformer --> Encoders
    Transformer --> Memory
    Transformer --> LayerUtil
    Transformer -->|"forward() logits"| Dist
    Dist -->|"continuous outputs"| Postproc
    Postproc -->|"aggregated ŷ"| Output
    Output -->|"returned to"| API
    API -->|"returns predictions"| Client
    Loader -->|"reads config"| ModelConfig
    Loader -->|"uses constants"| Constants
    API -->|"reads config"| GlobalConfig

    %% Click Events
    click Client "https://github.com/priorlabs/tabpfn/blob/main/examples/tabpfn_for_regression.py"
    click API "https://github.com/priorlabs/tabpfn/blob/main/src/tabpfn/regressor.py"
    click Base "https://github.com/priorlabs/tabpfn/blob/main/src/tabpfn/base.py"
    click Orchestrator "https://github.com/priorlabs/tabpfn/blob/main/src/tabpfn/inference.py"
    click PreprocUser "https://github.com/priorlabs/tabpfn/blob/main/src/tabpfn/preprocessing.py"
    click PreprocModel "https://github.com/priorlabs/tabpfn/blob/main/src/tabpfn/model/preprocessing.py"
    click Loader "https://github.com/priorlabs/tabpfn/blob/main/src/tabpfn/model/loading.py"
    click Transformer "https://github.com/priorlabs/tabpfn/blob/main/src/tabpfn/model/transformer.py"
    click FullAttention "https://github.com/priorlabs/tabpfn/blob/main/src/tabpfn/model/attention/full_attention.py"
    click MultiHeadAttention "https://github.com/priorlabs/tabpfn/blob/main/src/tabpfn/model/multi_head_attention.py"
    click MLP "https://github.com/priorlabs/tabpfn/blob/main/src/tabpfn/model/mlp.py"
    click Encoders "https://github.com/priorlabs/tabpfn/blob/main/src/tabpfn/model/encoders.py"
    click Memory "https://github.com/priorlabs/tabpfn/blob/main/src/tabpfn/model/memory.py"
    click LayerUtil "https://github.com/priorlabs/tabpfn/blob/main/src/tabpfn/model/layer.py"
    click Dist "https://github.com/priorlabs/tabpfn/blob/main/src/tabpfn/model/bar_distribution.py"
    click Postproc "https://github.com/priorlabs/tabpfn/blob/main/src/tabpfn/inference.py"
    click Script "https://github.com/priorlabs/tabpfn/blob/main/scripts/download_all_models.py"
    click GlobalConfig "https://github.com/priorlabs/tabpfn/blob/main/src/tabpfn/config.py"
    click ModelConfig "https://github.com/priorlabs/tabpfn/blob/main/src/tabpfn/model/config.py"
    click Constants "https://github.com/priorlabs/tabpfn/blob/main/src/tabpfn/constants.py"

    %% Styles
    classDef api fill:#D0E1F9,stroke:#000,stroke-width:1px
    classDef process fill:#D2F8D2,stroke:#000,stroke-width:1px
    classDef external fill:#FFF5BA,stroke:#000,stroke-width:1px
    classDef data fill:#F0F0F0,stroke:#000,stroke-width:1px,stroke-dasharray: 5 5
```
