# alteryx/featuretools architecture

> Architecture diagram of the GitHub repository alteryx/featuretools, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/alteryx/featuretools
- Repository: https://github.com/alteryx/featuretools

## Overview

1. **What this repository is**
   Featuretools is a Python library for automated feature engineering. Its core job is to turn relational or time-based data stored in an `EntitySet` into a machine-learning-ready feature matrix using Deep Feature Synthesis (DFS).

2. **Primary user-facing entry points**
   The top-level `featuretools` package exposes the main API, including `dfs`, `EntitySet`, demo data loaders, and feature/primitive APIs. The package can also be run as a module via `featuretools/__main__.py`, but the main architecture is library-first rather than service-based.

3. **Core data model: EntitySet and relationships**
   The `entityset` subsystem defines the project’s central data abstraction: tables, relationships, timestamps, and last-time-index logic. It also includes serialization/deserialization so entity sets can be saved and reloaded across sessions or environments.

4. **Feature definition layer**
   The `feature_base` subsystem represents features as composable objects rather than simple column names. It handles feature construction, caching, descriptions, visualization, and serialization, which makes feature definitions portable and reusable.

5. **Primitive system**
   The `primitives` package provides the building blocks for feature generation. It contains base classes for aggregation and transform primitives, plus a large catalog of standard primitives grouped by type: numeric, datetime, email, URL, natural language, lat/long, time series, cumulative, and binary operations. This is the main extensibility boundary for custom feature logic.

6. **Feature synthesis pipeline**
   The `synthesis` subsystem implements DFS and related orchestration. It selects valid primitives, combines features according to entity relationships, and produces the final feature definitions. This is the layer that turns entity graph structure plus primitive catalog into candidate features.

7. **Feature matrix computation**
   The `computational_backends` subsystem executes the synthesized features against actual data. It builds feature sets, calculates the feature matrix, and manages backend-specific utilities. This is the separation point between symbolic feature definitions and numeric execution over tables.

8. **Feature selection and reduction**
   The `selection` package provides post-synthesis filtering and selection logic. It acts after feature generation to reduce feature sets based on criteria such as redundancy or usefulness, helping keep outputs tractable for downstream ML pipelines.

9. **Feature discovery utilities**
   The `feature_discovery` package contains lightweight discovery and conversion tools, including `FeatureCollection` and `LiteFeature`. This appears to support importing, comparing, and managing feature definitions outside the full DFS pipeline.

10. **Demo and example datasets**
   The `demo` package supplies ready-made sample datasets such as retail, flight, weather, and mock customer data. These are used to illustrate the library’s flow: load an `EntitySet`, define a target dataframe, run DFS, and inspect the resulting feature matrix.

11. **Utilities and supporting services**
   The `utils` package contains cross-cutting helpers for configuration, time handling, schema/description utilities, plugin entry points, S3 helpers, plotting, and primitive recommendation. These are shared support functions rather than a separate domain layer.

12. **Documentation and notebooks as a major repo component**
   The `docs` tree is substantial and includes Sphinx docs plus many Jupyter notebooks showing getting started, guides, and resource material. For this repo, documentation is not ancillary; it is part of the primary product experience and explains the library’s multi-step workflow.

13. **Testing and compatibility boundaries**
   The `tests` tree mirrors the library architecture: entity sets, primitives, synthesis, selection, discovery, utilities, entry points, and compatibility with add-ons. There are also requirement files for minimum/latest dependency matrices, indicating the repo carefully manages behavior across dependency versions and optional extras like Dask.

14. **Operational/runtime characteristics**
   This is a pure Python package distributed via pip/conda, with optional dependency-based features such as Dask-backed parallelism and add-on primitive families. CI workflows cover linting, docs, release automation, dependency testing, and performance checks, but there is no standalone server or persistent runtime in the repository.

15. **End-to-end data flow**
   The typical flow is: load or build an `EntitySet` → define target dataframe and relationships → DFS synthesizes candidate features from primitives → computational backend evaluates them into a feature matrix → optional selection/reduction and serialization/export for downstream ML use.

Main entry points: [`featuretools/tests`](https://github.com/alteryx/featuretools/tree/main/featuretools/tests) (Core tests), [`featuretools/__init__.py`](https://github.com/alteryx/featuretools/blob/main/featuretools/__init__.py) (featuretools), [`featuretools/synthesis/deep_feature_synthesis.py`](https://github.com/alteryx/featuretools/blob/main/featuretools/synthesis/deep_feature_synthesis.py) (DFS synthesis), [`featuretools/primitives/standard/__init__.py`](https://github.com/alteryx/featuretools/blob/main/featuretools/primitives/standard/__init__.py) (Standard primitives)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_core["Core library"]
  node_featuretools_pkg["featuretools<br/>top-level API<br/>[__init__.py]"]
  node_featuretools_main["Module entry<br/>CLI entry<br/>[__main__.py]"]
  node_entityset[("EntitySet<br/>data model<br/>[entityset.py]")]
  node_feature_base["Feature base<br/>feature model<br/>[feature_base.py]"]
  node_synthesis["DFS synthesis<br/>feature generation"]
  node_computational_backends["Execution backend<br/>matrix compute"]
  node_selection["Selection<br/>post-processing<br/>[selection.py]"]
  node_feature_discovery["Discovery<br/>feature registry"]
end

subgraph group_group_primitives["Primitives"]
  node_primitives_base["Primitive base<br/>extensibility base<br/>[primitive_base.py]"]
  node_primitives_standard["Standard primitives<br/>primitive catalog<br/>[__init__.py]"]
  node_primitives_datetime["Datetime<br/>primitive family<br/>[__init__.py]"]
  node_primitives_numeric["Numeric<br/>primitive family<br/>[__init__.py]"]
  node_primitives_text["Text & IDs<br/>primitive family<br/>[__init__.py]"]
  node_primitives_timeseries["Time series<br/>primitive family<br/>[__init__.py]"]
end

subgraph group_group_support["Support"]
  node_demo[("Demo data<br/>sample datasets<br/>[api.py]")]
  node_utils["Utilities<br/>shared helpers<br/>[api.py]"]
end

subgraph group_group_docs["Docs"]
  node_docs["Documentation<br/>docs site<br/>[index.ipynb]"]
end

subgraph group_group_tests["Tests"]
  node_tests_core["Core tests<br/>test suite"]
  node_tests_addons["Add-on tests<br/>compatibility tests"]
  node_tests_requirements["Req matrices<br/>dependency matrix"]
end

subgraph group_group_ops["Tooling"]
  node_tooling["CI & release<br/>repo tooling<br/>[pyproject.toml]"]
end

node_featuretools_pkg -->|"exports"| node_entityset
node_featuretools_pkg -->|"exports"| node_feature_base
node_featuretools_pkg -->|"exports"| node_synthesis
node_featuretools_pkg -->|"exports"| node_computational_backends
node_featuretools_pkg -->|"exports"| node_selection
node_featuretools_pkg -->|"exports"| node_demo
node_featuretools_main -->|"uses"| node_featuretools_pkg
node_synthesis -->|"analyzes"| node_entityset
node_synthesis -->|"consumes"| node_primitives_base
node_synthesis -->|"selects"| node_primitives_standard
node_synthesis -->|"builds"| node_feature_base
node_computational_backends -->|"executes"| node_feature_base
node_computational_backends -->|"reads"| node_entityset
node_selection -->|"filters"| node_feature_base
node_feature_discovery -->|"converts"| node_feature_base
node_primitives_standard -->|"implements"| node_primitives_base
node_primitives_standard -->|"groups"| node_primitives_datetime
node_primitives_standard -->|"groups"| node_primitives_numeric
node_primitives_standard -->|"groups"| node_primitives_text
node_primitives_standard -->|"groups"| node_primitives_timeseries
node_demo -->|"loads"| node_entityset
node_demo -->|"illustrates"| node_synthesis
node_utils -->|"supports"| node_entityset
node_utils -->|"supports"| node_feature_base
node_utils -->|"supports"| node_primitives_base
node_docs -->|"documents"| node_featuretools_pkg
node_docs -->|"teaches"| node_entityset
node_docs -->|"teaches"| node_synthesis
node_docs -->|"teaches"| node_selection
node_tests_core -->|"verifies"| node_entityset
node_tests_core -->|"verifies"| node_feature_base
node_tests_core -->|"verifies"| node_primitives_standard
node_tests_core -->|"verifies"| node_synthesis
node_tests_core -->|"verifies"| node_computational_backends
node_tests_core -->|"verifies"| node_selection
node_tests_core -->|"verifies"| node_feature_discovery
node_tests_core -->|"verifies"| node_utils
node_tests_addons -->|"verifies"| node_utils
node_tests_requirements -->|"supports"| node_tooling
node_tooling -->|"builds"| node_docs
node_tooling -->|"runs"| node_tests_core
node_tooling -->|"checks"| node_tests_requirements

click node_featuretools_pkg "https://github.com/alteryx/featuretools/blob/main/featuretools/__init__.py"
click node_featuretools_main "https://github.com/alteryx/featuretools/blob/main/featuretools/__main__.py"
click node_entityset "https://github.com/alteryx/featuretools/blob/main/featuretools/entityset/entityset.py"
click node_feature_base "https://github.com/alteryx/featuretools/blob/main/featuretools/feature_base/feature_base.py"
click node_synthesis "https://github.com/alteryx/featuretools/blob/main/featuretools/synthesis/deep_feature_synthesis.py"
click node_computational_backends "https://github.com/alteryx/featuretools/blob/main/featuretools/computational_backends/calculate_feature_matrix.py"
click node_selection "https://github.com/alteryx/featuretools/blob/main/featuretools/selection/selection.py"
click node_feature_discovery "https://github.com/alteryx/featuretools/blob/main/featuretools/feature_discovery/feature_discovery.py"
click node_primitives_base "https://github.com/alteryx/featuretools/blob/main/featuretools/primitives/base/primitive_base.py"
click node_primitives_standard "https://github.com/alteryx/featuretools/blob/main/featuretools/primitives/standard/__init__.py"
click node_primitives_datetime "https://github.com/alteryx/featuretools/blob/main/featuretools/primitives/standard/transform/datetime/__init__.py"
click node_primitives_numeric "https://github.com/alteryx/featuretools/blob/main/featuretools/primitives/standard/transform/numeric/__init__.py"
click node_primitives_text "https://github.com/alteryx/featuretools/blob/main/featuretools/primitives/standard/transform/natural_language/__init__.py"
click node_primitives_timeseries "https://github.com/alteryx/featuretools/blob/main/featuretools/primitives/standard/transform/time_series/__init__.py"
click node_demo "https://github.com/alteryx/featuretools/blob/main/featuretools/demo/api.py"
click node_utils "https://github.com/alteryx/featuretools/blob/main/featuretools/utils/api.py"
click node_docs "https://github.com/alteryx/featuretools/blob/main/docs/source/index.ipynb"
click node_tests_core "https://github.com/alteryx/featuretools/tree/main/featuretools/tests"
click node_tests_addons "https://github.com/alteryx/featuretools/tree/main/featuretools/tests/entry_point_tests"
click node_tests_requirements "https://github.com/alteryx/featuretools/tree/main/featuretools/tests/requirement_files"
click node_tooling "https://github.com/alteryx/featuretools/blob/main/pyproject.toml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_featuretools_pkg,node_featuretools_main,node_entityset,node_feature_base,node_synthesis,node_computational_backends,node_selection,node_feature_discovery toneBlue
class node_primitives_base,node_primitives_standard,node_primitives_datetime,node_primitives_numeric,node_primitives_text,node_primitives_timeseries toneAmber
class node_demo,node_utils toneMint
class node_docs toneRose
class node_tests_core,node_tests_addons,node_tests_requirements toneIndigo
class node_tooling toneTeal
```

## Components

### Core library

Main runtime packages for entity modeling, feature definitions, synthesis, and execution.

- [featuretools](https://github.com/alteryx/featuretools/blob/main/featuretools/__init__.py): top-level API · Public package surface exposing DFS, EntitySet, primitives, and demos. · `featuretools/__init__.py`
- [Module entry](https://github.com/alteryx/featuretools/blob/main/featuretools/__main__.py): CLI entry · Allows running the library as a module. · `featuretools/__main__.py`
- [EntitySet](https://github.com/alteryx/featuretools/blob/main/featuretools/entityset/entityset.py): data model · Tables, relationships, timestamps, and serialization for relational/time-based data. · `featuretools/entityset/entityset.py`
- [Feature base](https://github.com/alteryx/featuretools/blob/main/featuretools/feature_base/feature_base.py): feature model · Composable feature objects with caching, descriptions, visualization, and serialization. · `featuretools/feature_base/feature_base.py`
- [DFS synthesis](https://github.com/alteryx/featuretools/blob/main/featuretools/synthesis/deep_feature_synthesis.py): feature generation · Deep Feature Synthesis orchestration and primitive selection. · `featuretools/synthesis/deep_feature_synthesis.py`
- [Execution backend](https://github.com/alteryx/featuretools/blob/main/featuretools/computational_backends/calculate_feature_matrix.py): matrix compute · Builds feature sets and calculates feature matrices over data. · `featuretools/computational_backends/calculate_feature_matrix.py`
- [Selection](https://github.com/alteryx/featuretools/blob/main/featuretools/selection/selection.py): post-processing · Filters and reduces generated feature sets. · `featuretools/selection/selection.py`
- [Discovery](https://github.com/alteryx/featuretools/blob/main/featuretools/feature_discovery/feature_discovery.py): feature registry · Lightweight feature collection and conversion utilities. · `featuretools/feature_discovery/feature_discovery.py`

### Primitives

Base classes and standard feature primitives used by DFS.

- [Primitive base](https://github.com/alteryx/featuretools/blob/main/featuretools/primitives/base/primitive_base.py): extensibility base · Base classes for aggregation and transform primitives. · `featuretools/primitives/base/primitive_base.py`
- [Standard primitives](https://github.com/alteryx/featuretools/blob/main/featuretools/primitives/standard/__init__.py): primitive catalog · Built-in aggregation, transform, and time-series primitive families. · `featuretools/primitives/standard/__init__.py`
- [Datetime](https://github.com/alteryx/featuretools/blob/main/featuretools/primitives/standard/transform/datetime/__init__.py): primitive family · Calendar, holiday, and time-difference transforms. · `featuretools/primitives/standard/transform/datetime/__init__.py`
- [Numeric](https://github.com/alteryx/featuretools/blob/main/featuretools/primitives/standard/transform/numeric/__init__.py): primitive family · Numeric transforms such as absolute value, logs, percent change, and lag-like operations. · `featuretools/primitives/standard/transform/numeric/__init__.py`
- [Text & IDs](https://github.com/alteryx/featuretools/blob/main/featuretools/primitives/standard/transform/natural_language/__init__.py): primitive family · Email, URL, natural language, postal, name, and file-derived transforms. · `featuretools/primitives/standard/transform/natural_language/__init__.py`
- [Time series](https://github.com/alteryx/featuretools/blob/main/featuretools/primitives/standard/transform/time_series/__init__.py): primitive family · Rolling, expanding, cumulative, and lag primitives for ordered data. · `featuretools/primitives/standard/transform/time_series/__init__.py`

### Support

Shared utilities, demo data, and package entry points.

- [Demo data](https://github.com/alteryx/featuretools/blob/main/featuretools/demo/api.py): sample datasets · Retail, flight, weather, and mock customer datasets for tutorials and examples. · `featuretools/demo/api.py`
- [Utilities](https://github.com/alteryx/featuretools/blob/main/featuretools/utils/api.py): shared helpers · Cross-cutting helpers for time, schema, entry points, plotting, S3, and recommendations. · `featuretools/utils/api.py`

### Docs

Sphinx docs, guides, and notebooks used as part of the product experience.

- [Documentation](https://github.com/alteryx/featuretools/blob/main/docs/source/index.ipynb): docs site · Sphinx docs and notebooks demonstrating the library workflow. · `docs/source/index.ipynb`

### Tests

Architecture-mirroring test suites and compatibility checks.

- [Core tests](https://github.com/alteryx/featuretools/tree/main/featuretools/tests): test suite · Tests covering entity sets, primitives, synthesis, selection, discovery, and utilities. · `featuretools/tests`
- [Add-on tests](https://github.com/alteryx/featuretools/tree/main/featuretools/tests/entry_point_tests): compatibility tests · Entry-point and plugin tests for external primitive and plugin packages. · `featuretools/tests/entry_point_tests`
- [Req matrices](https://github.com/alteryx/featuretools/tree/main/featuretools/tests/requirement_files): dependency matrix · Minimum/latest/core/dask requirement files used to validate supported environments. · `featuretools/tests/requirement_files`

### Tooling

Packaging, CI, release, and developer workflow files.

- [CI & release](https://github.com/alteryx/featuretools/blob/main/pyproject.toml): repo tooling · Workflow, packaging, and development automation files. · `pyproject.toml`

## Connections

- featuretools → EntitySet: exports · Public API exposes EntitySet and related helpers.
- featuretools → Feature base: exports · Public API exposes feature objects and feature construction helpers.
- featuretools → DFS synthesis: exports · Public API exposes DFS entry points.
- featuretools → Execution backend: exports · Public API exposes feature matrix computation.
- featuretools → Selection: exports · Public API exposes feature selection helpers.
- featuretools → Demo data: exports · Public API exposes demo datasets.
- Module entry → featuretools: uses · Module entry delegates to library API.
- DFS synthesis → EntitySet: analyzes · DFS uses entity tables and relationships to build candidate features.
- DFS synthesis → Primitive base: consumes · Synthesis relies on primitive interfaces to enumerate valid features.
- DFS synthesis → Standard primitives: selects · DFS selects from the built-in primitive catalog.
- DFS synthesis → Feature base: builds · Synthesized outputs are feature objects.
- Execution backend → Feature base: executes · Backends evaluate feature definitions against data.
- Execution backend → EntitySet: reads · Matrix calculation reads from entity sets and relationships.
- Selection → Feature base: filters · Selection operates on generated feature objects.
- Discovery → Feature base: converts · Discovery utilities import and transform feature definitions.
- Standard primitives → Primitive base: implements · Standard primitives extend the base primitive abstractions.
- Standard primitives → Datetime: groups · Datetime family is part of the standard catalog.
- Standard primitives → Numeric: groups · Numeric family is part of the standard catalog.
- Standard primitives → Text & IDs: groups · Text and identifier family is part of the standard catalog.
- Standard primitives → Time series: groups · Time-series family is part of the standard catalog.
- Demo data → EntitySet: loads · Demo datasets are used to construct example entity sets.
- Demo data → DFS synthesis: illustrates · Demo flows demonstrate DFS usage.
- Utilities → EntitySet: supports · Shared utilities support entity set handling and metadata.
- Utilities → Feature base: supports · Shared utilities support feature descriptions and serialization.
- Utilities → Primitive base: supports · Utilities support primitive discovery and recommendation.
- Documentation → featuretools: documents · Docs explain the main public API and workflow.
- Documentation → EntitySet: teaches · Guides cover building and using entity sets.
- Documentation → DFS synthesis: teaches · Guides cover DFS tuning and primitive options.
- Documentation → Selection: teaches · Guides cover feature selection and reduction.
- Core tests → EntitySet: verifies · Tests cover entity set behavior and serialization.
- Core tests → Feature base: verifies · Tests cover feature modeling, descriptions, and serialization.
- Core tests → Standard primitives: verifies · Tests cover standard primitive behavior.
- Core tests → DFS synthesis: verifies · Tests cover DFS and primitive selection.
- Core tests → Execution backend: verifies · Tests cover feature matrix calculation.
- Core tests → Selection: verifies · Tests cover feature selection logic.
- Core tests → Discovery: verifies · Tests cover feature discovery utilities.
- Core tests → Utilities: verifies · Tests cover shared utility behavior.
- Add-on tests → Utilities: verifies · Addon tests validate entry-point and plugin loading paths.
- Req matrices → CI & release: supports · Dependency matrices support CI and compatibility testing.
- CI & release → Documentation: builds · Workflow and packaging tooling build and validate docs.
- CI & release → Core tests: runs · CI validates core behavior across dependency sets.
- CI & release → Req matrices: checks · CI uses requirement matrices for minimum/latest coverage.
