# sematic-ai/sematic architecture

> Architecture diagram of the GitHub repository sematic-ai/sematic, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sematic-ai/sematic
- Repository: https://github.com/sematic-ai/sematic

## Overview

Sematic turns Python functions into typed, traceable pipeline graphs, executes them locally or across cloud/Kubernetes resources, persists run state and artifacts, and exposes monitoring and lineage through a web dashboard. The main path is authoring, graph construction, execution/resolution, persistence, then inspection or rerun. The README documents caching, retries, dynamic graphs, resource plugins, and integrations; those branches are represented at subsystem level where their internal wiring was not sampled. Exact paths come from the tree; implementation detail for unsampled areas is limited.

Main entry points: [`sematic/resolver.py`](https://github.com/sematic-ai/sematic/blob/main/sematic/resolver.py) (Step resolver), [`sematic/runner.py`](https://github.com/sematic-ai/sematic/blob/main/sematic/runner.py) (Runner entry), [`sematic/api/app.py`](https://github.com/sematic-ai/sematic/blob/main/sematic/api/app.py) (HTTP API), [`sematic/api_client.py`](https://github.com/sematic-ai/sematic/blob/main/sematic/api_client.py) (API client)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_authoring["Pipeline authoring"]
  node_sdk["Python SDK<br/>[function.py]"]
  node_graph["Pipeline graph<br/>[graph.py]"]
  node_typing["Types and serialization<br/>[serialization.py]"]
end

subgraph group_execution["Pipeline execution"]
  node_runner["Runner entry<br/>[runner.py]"]
  node_local["Local runner<br/>[local_runner.py]"]
  node_cloud["Cloud runner<br/>[cloud_runner.py]"]
  node_resolver["Step resolver<br/>[resolver.py]"]
  node_scheduler["Job scheduler<br/>[job_scheduler.py]"]
  node_kubernetes["Kubernetes<br/>[kubernetes.py]"]
  node_plugins["Compute plugins"]
  node_cache["Step caching<br/>[caching.py]"]
end

subgraph group_persistence["State and artifacts"]
  node_database[("Platform database<br/>[db.py]")]
  node_storage["Artifact storage"]
  node_s3["S3 backend<br/>[s3_storage.py]"]
  node_metrics["Metrics storage"]
end

subgraph group_service["Platform services"]
  node_cli["CLI<br/>[main.py]"]
  node_api["HTTP API<br/>[app.py]"]
  node_client["API client<br/>[api_client.py]"]
  node_git["Git metadata<br/>[git.py]"]
end

subgraph group_experience["User experience"]
  node_dashboard["Web dashboard<br/>[index.tsx]"]
  node_pipeline_views["Pipeline views<br/>[PipelineList.tsx]"]
  node_reruns["Run inspection<br/>[RunRouter.tsx]"]
end

node_engineer(("ML engineer"))
node_kube_service{{"Kubernetes cluster"}}

node_engineer -->|"defines pipeline"| node_sdk
node_sdk -->|"builds graph"| node_graph
node_sdk -->|"checks and serializes"| node_typing
node_engineer -->|"starts or runs"| node_cli
node_cli -->|"invokes"| node_runner
node_runner -->|"selects local"| node_local
node_runner -->|"selects cloud"| node_cloud
node_local -->|"resolves steps"| node_resolver
node_cloud -->|"resolves steps"| node_resolver
node_cloud -->|"submits jobs"| node_scheduler
node_scheduler -->|"dispatches jobs"| node_kubernetes
node_kubernetes -->|"runs workloads"| node_kube_service
node_resolver -.->|"manages resources"| node_plugins
node_resolver -->|"checks cached steps"| node_cache
node_runner -->|"reports run state"| node_api
node_api -->|"reads and writes"| node_database
node_api -->|"requests destinations"| node_storage
node_storage -.->|"uses backend"| node_s3
node_resolver -->|"transfers artifacts"| node_storage
node_resolver -->|"records metrics"| node_metrics
node_metrics -->|"stores metric data"| node_database
node_dashboard -->|"requests platform data"| node_client
node_pipeline_views -->|"loads pipeline runs"| node_client
node_reruns -->|"loads run details"| node_client
node_client -->|"calls API"| node_api
node_git -.->|"tracks run metadata"| node_database
node_dashboard -->|"opens run inspection"| node_reruns

click node_sdk "https://github.com/sematic-ai/sematic/blob/main/sematic/function.py"
click node_graph "https://github.com/sematic-ai/sematic/blob/main/sematic/graph.py"
click node_typing "https://github.com/sematic-ai/sematic/blob/main/sematic/types/serialization.py"
click node_cli "https://github.com/sematic-ai/sematic/blob/main/sematic/cli/main.py"
click node_runner "https://github.com/sematic-ai/sematic/blob/main/sematic/runner.py"
click node_local "https://github.com/sematic-ai/sematic/blob/main/sematic/runners/local_runner.py"
click node_cloud "https://github.com/sematic-ai/sematic/blob/main/sematic/runners/cloud_runner.py"
click node_resolver "https://github.com/sematic-ai/sematic/blob/main/sematic/resolver.py"
click node_scheduler "https://github.com/sematic-ai/sematic/blob/main/sematic/scheduling/job_scheduler.py"
click node_kubernetes "https://github.com/sematic-ai/sematic/blob/main/sematic/scheduling/kubernetes.py"
click node_plugins "https://github.com/sematic-ai/sematic/blob/main/sematic/plugins/abstract_external_resource.py"
click node_cache "https://github.com/sematic-ai/sematic/blob/main/sematic/caching/caching.py"
click node_api "https://github.com/sematic-ai/sematic/blob/main/sematic/api/app.py"
click node_client "https://github.com/sematic-ai/sematic/blob/main/sematic/api_client.py"
click node_database "https://github.com/sematic-ai/sematic/blob/main/sematic/db/db.py"
click node_storage "https://github.com/sematic-ai/sematic/blob/main/sematic/plugins/abstract_storage.py"
click node_s3 "https://github.com/sematic-ai/sematic/blob/main/sematic/plugins/storage/s3_storage.py"
click node_metrics "https://github.com/sematic-ai/sematic/blob/main/sematic/plugins/metrics_storage/sql/sql_metrics_storage.py"
click node_dashboard "https://github.com/sematic-ai/sematic/blob/main/sematic/ui/packages/common/src/pages/RunDetails/index.tsx"
click node_pipeline_views "https://github.com/sematic-ai/sematic/blob/main/sematic/ui/packages/common/src/pages/PipelineList/PipelineList.tsx"
click node_reruns "https://github.com/sematic-ai/sematic/blob/main/sematic/ui/packages/common/src/pages/RunDetails/RunRouter.tsx"
click node_git "https://github.com/sematic-ai/sematic/blob/main/sematic/caching/git.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_sdk,node_graph,node_typing toneBlue
class node_runner,node_local,node_cloud,node_resolver,node_scheduler,node_kubernetes,node_plugins,node_cache toneAmber
class node_database,node_storage,node_s3,node_metrics toneMint
class node_cli,node_api,node_client,node_git toneRose
class node_dashboard,node_pipeline_views,node_reruns,node_engineer,node_kube_service toneIndigo
```

## Components

### Pipeline authoring

- [Python SDK](https://github.com/sematic-ai/sematic/blob/main/sematic/function.py): `sematic/function.py`
- [Pipeline graph](https://github.com/sematic-ai/sematic/blob/main/sematic/graph.py): `sematic/graph.py`
- [Types and serialization](https://github.com/sematic-ai/sematic/blob/main/sematic/types/serialization.py): `sematic/types/serialization.py`

### Pipeline execution

- [Runner entry](https://github.com/sematic-ai/sematic/blob/main/sematic/runner.py): `sematic/runner.py`
- [Local runner](https://github.com/sematic-ai/sematic/blob/main/sematic/runners/local_runner.py): `sematic/runners/local_runner.py`
- [Cloud runner](https://github.com/sematic-ai/sematic/blob/main/sematic/runners/cloud_runner.py): `sematic/runners/cloud_runner.py`
- [Step resolver](https://github.com/sematic-ai/sematic/blob/main/sematic/resolver.py): `sematic/resolver.py`
- [Job scheduler](https://github.com/sematic-ai/sematic/blob/main/sematic/scheduling/job_scheduler.py): `sematic/scheduling/job_scheduler.py`
- [Kubernetes](https://github.com/sematic-ai/sematic/blob/main/sematic/scheduling/kubernetes.py): `sematic/scheduling/kubernetes.py`
- [Compute plugins](https://github.com/sematic-ai/sematic/blob/main/sematic/plugins/abstract_external_resource.py): `sematic/plugins/abstract_external_resource.py`
- [Step caching](https://github.com/sematic-ai/sematic/blob/main/sematic/caching/caching.py): `sematic/caching/caching.py`

### State and artifacts

- [Platform database](https://github.com/sematic-ai/sematic/blob/main/sematic/db/db.py): `sematic/db/db.py`
- [Artifact storage](https://github.com/sematic-ai/sematic/blob/main/sematic/plugins/abstract_storage.py): `sematic/plugins/abstract_storage.py`
- [S3 backend](https://github.com/sematic-ai/sematic/blob/main/sematic/plugins/storage/s3_storage.py): `sematic/plugins/storage/s3_storage.py`
- [Metrics storage](https://github.com/sematic-ai/sematic/blob/main/sematic/plugins/metrics_storage/sql/sql_metrics_storage.py): `sematic/plugins/metrics_storage/sql/sql_metrics_storage.py`

### Platform services

- [CLI](https://github.com/sematic-ai/sematic/blob/main/sematic/cli/main.py): `sematic/cli/main.py`
- [HTTP API](https://github.com/sematic-ai/sematic/blob/main/sematic/api/app.py): `sematic/api/app.py`
- [API client](https://github.com/sematic-ai/sematic/blob/main/sematic/api_client.py): `sematic/api_client.py`
- [Git metadata](https://github.com/sematic-ai/sematic/blob/main/sematic/caching/git.py): `sematic/caching/git.py`

### User experience

- [Web dashboard](https://github.com/sematic-ai/sematic/blob/main/sematic/ui/packages/common/src/pages/RunDetails/index.tsx): `sematic/ui/packages/common/src/pages/RunDetails/index.tsx`
- [Pipeline views](https://github.com/sematic-ai/sematic/blob/main/sematic/ui/packages/common/src/pages/PipelineList/PipelineList.tsx): `sematic/ui/packages/common/src/pages/PipelineList/PipelineList.tsx`
- [Run inspection](https://github.com/sematic-ai/sematic/blob/main/sematic/ui/packages/common/src/pages/RunDetails/RunRouter.tsx): `sematic/ui/packages/common/src/pages/RunDetails/RunRouter.tsx`

### Other components

- **ML engineer**
- **Kubernetes cluster**

## Connections

- ML engineer → Python SDK: defines pipeline
- Python SDK → Pipeline graph: builds graph
- Python SDK → Types and serialization: checks and serializes
- ML engineer → CLI: starts or runs
- CLI → Runner entry: invokes
- Runner entry → Local runner: selects local
- Runner entry → Cloud runner: selects cloud
- Local runner → Step resolver: resolves steps
- Cloud runner → Step resolver: resolves steps
- Cloud runner → Job scheduler: submits jobs
- Job scheduler → Kubernetes: dispatches jobs
- Kubernetes → Kubernetes cluster: runs workloads
- Step resolver → Compute plugins: manages resources
- Step resolver → Step caching: checks cached steps
- Runner entry → HTTP API: reports run state
- HTTP API → Platform database: reads and writes
- HTTP API → Artifact storage: requests destinations
- Artifact storage → S3 backend: uses backend
- Step resolver → Artifact storage: transfers artifacts
- Step resolver → Metrics storage: records metrics
- Metrics storage → Platform database: stores metric data
- Web dashboard → API client: requests platform data
- Pipeline views → API client: loads pipeline runs
- Run inspection → API client: loads run details
- API client → HTTP API: calls API
- Git metadata → Platform database: tracks run metadata
- Web dashboard → Run inspection: opens run inspection