# tiqi-group/icon architecture

> Architecture diagram of the GitHub repository tiqi-group/icon, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tiqi-group/icon
- Repository: https://github.com/tiqi-group/icon

## Overview

1. **System boundary**
   ICON is a Python-based control plane for trapped-ion experiments. It sits between a user-facing React UI, Python experiment definitions from a separate “experiment library,” connected device services, and laboratory hardware/data stores.

2. **Backend as the central coordinator**
   The `icon.server` package is the core runtime. It hosts the HTTP/WebSocket server, the scheduler, preprocessing/hardware/postprocessing workers, and all persistence access. Most meaningful state changes flow through this backend rather than directly between UI and hardware.

3. **Frontend delivery and role**
   The React/TypeScript frontend lives in `frontend/`, but the production build is served by the backend from `src/icon/server/frontend/`. The UI is for configuration, device/parameter management, experiment submission, live job monitoring, and result visualization. In architecture terms, it is a client of the backend API plus live socket updates.

4. **API layer**
   The backend exposes a set of controllers under `src/icon/server/api/` for configuration, devices, experiments, parameters, scans, scheduler, status, and experiment data. An OpenAPI spec is included in `docs/openapi.yaml`, and there is also a generated Python client under `src/icon/client/` for programmatic access.

5. **Experiment execution pipeline**
   Experiment runs are not handled as a single monolithic function. They are split into three processing stages:
   - **Pre-processing** prepares jobs and parameters.
   - **Hardware processing** drives the actual hardware-facing execution.
   - **Post-processing** collects and transforms results.
   The scheduler coordinates these stages, and each stage has its own worker/task modules under `src/icon/server/{pre_processing,hardware_processing,post_processing}/`.

6. **Device and parameter model**
   ICON manages both experiment parameters and externally provided device parameters. The frontend includes dedicated device and parameter screens, while the backend has device and parameters controllers plus repositories/models for device metadata. This boundary matters because scans can target not only experiment-local parameters but also remote device parameters exposed via services such as `pydase`.

7. **Persistence split by data type**
   ICON uses different stores for different concerns:
   - **SQLite** for job history, device metadata, experiment sources, and relational state.
   - **InfluxDB v1** for time-series parameter/state data.
   - **HDF5** for experiment result payloads.
   The repository layer in `src/icon/server/data_access/repositories/` mediates access to these stores, so most higher-level server components should be drawn as depending on repositories rather than directly on databases.

8. **SQLite schema management**
   The SQLite schema is SQLAlchemy-based and versioned with Alembic. Startup runs migrations automatically, and the migration history under `src/icon/server/data_access/db_context/sqlite/alembic/versions/` indicates that job tracking, device records, scan parameters, retries, repetitions, and related metadata are first-class durable entities.

9. **Real-time updates**
   ICON uses Socket.IO/WebSocket-style communication for live state changes. The socket setup and emit queue live under `src/icon/server/web_server/`, which suggests asynchronous push updates from backend workers/scheduler to the UI for job progress, status changes, and live plots.

10. **Configuration and control flow**
    The application is configured via a YAML file in `~/.config/icon/config.yaml` by default. Settings can be edited either in that file or from the frontend settings page. This configuration influences server binding, backend integrations, and likely database/service endpoints, so it is a key architectural input to the whole runtime.

11. **Experiment-library integration**
    A major architectural dependency is the external experiment library written for `pycrystal`. ICON parses that library to discover experiments and metadata, and it provides templates/helpers in `src/icon/server/data_access/templates/` for extracting experiment and parameter information. In diagrams, this should appear as a separate upstream codebase feeding metadata into ICON, not as code owned by ICON itself.

12. **Operational packaging**
    The repo supports two deployment modes: a packaged Linux binary and source-based execution with `uv`. Development also expects a separate InfluxDB instance via `k8s/dev.yml`. This means the runtime is a multi-process/server system with external services, not a self-contained single binary application.

Main entry points: [`src/icon/server/data_access/repositories`](https://github.com/tiqi-group/icon/tree/main/src/icon/server/data_access/repositories) (Repositories), [`src/icon/server/scheduler`](https://github.com/tiqi-group/icon/tree/main/src/icon/server/scheduler) (Scheduler), [`src/icon/server/api`](https://github.com/tiqi-group/icon/tree/main/src/icon/server/api) (API Layer), [`src/icon/server/web_server`](https://github.com/tiqi-group/icon/tree/main/src/icon/server/web_server) (Web Server)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_frontend["Frontend"]
  node_ui["Web UI<br/>React app"]
  node_socket_client(("Live Updates<br/>Socket.IO client<br/>[socket.ts]"))
end

subgraph group_backend["Backend"]
  node_api_client["Python Client<br/>API client"]
  node_http_api["API Layer<br/>HTTP controllers"]
  node_web_server["Web Server<br/>HTTP + Socket.IO"]
  node_scheduler["Scheduler<br/>run coordinator"]
  node_preproc["Pre-processing<br/>job prep"]
  node_hardware["Hardware Exec<br/>device runner"]
  node_postproc["Post-processing<br/>result pipeline"]
  node_config["Configuration<br/>YAML settings<br/>[config.py]"]
end

subgraph group_storage["Persistence"]
  node_repo_layer["Repositories<br/>data access"]
  node_sqlite[("SQLite<br/>relational store")]
  node_influxdb[("InfluxDB<br/>time-series store<br/>[influxdb_v1.py]")]
  node_hdf5[("HDF5<br/>results store<br/>[h5py.py]")]
  node_sqlite_models["SQLite Models<br/>ORM schema"]
  node_migrations["Migrations<br/>schema versioning"]
end

subgraph group_integration["External Systems"]
  node_pycrystal_lib["Experiment Library<br/>external codebase"]
  node_devicesvc["Device Services<br/>external services"]
end

node_ui -->|"uses"| node_http_api
node_socket_client -->|"subscribes"| node_web_server
node_api_client -->|"calls"| node_http_api
node_web_server -->|"hosts"| node_http_api
node_http_api -->|"controls"| node_scheduler
node_http_api -->|"reads/writes"| node_repo_layer
node_scheduler -->|"dispatches"| node_preproc
node_scheduler -->|"dispatches"| node_hardware
node_scheduler -->|"dispatches"| node_postproc
node_preproc -->|"uses"| node_repo_layer
node_hardware -->|"drives"| node_devicesvc
node_hardware -->|"records"| node_repo_layer
node_postproc -->|"stores"| node_repo_layer
node_repo_layer -->|"persists"| node_sqlite
node_repo_layer -->|"persists"| node_influxdb
node_repo_layer -->|"persists"| node_hdf5
node_sqlite_models -->|"maps to"| node_sqlite
node_migrations -->|"migrates"| node_sqlite
node_config -->|"configures"| node_web_server
node_pycrystal_lib -->|"metadata source"| node_repo_layer
node_pycrystal_lib -->|"defines jobs"| node_preproc
node_web_server -->|"emits status"| node_scheduler

click node_ui "https://github.com/tiqi-group/icon/tree/main/frontend/src"
click node_socket_client "https://github.com/tiqi-group/icon/blob/main/frontend/src/socket.ts"
click node_api_client "https://github.com/tiqi-group/icon/tree/main/src/icon/client"
click node_http_api "https://github.com/tiqi-group/icon/tree/main/src/icon/server/api"
click node_web_server "https://github.com/tiqi-group/icon/tree/main/src/icon/server/web_server"
click node_scheduler "https://github.com/tiqi-group/icon/tree/main/src/icon/server/scheduler"
click node_preproc "https://github.com/tiqi-group/icon/tree/main/src/icon/server/pre_processing"
click node_hardware "https://github.com/tiqi-group/icon/tree/main/src/icon/server/hardware_processing"
click node_postproc "https://github.com/tiqi-group/icon/tree/main/src/icon/server/post_processing"
click node_repo_layer "https://github.com/tiqi-group/icon/tree/main/src/icon/server/data_access/repositories"
click node_sqlite "https://github.com/tiqi-group/icon/tree/main/src/icon/server/data_access/db_context/sqlite"
click node_influxdb "https://github.com/tiqi-group/icon/blob/main/src/icon/server/data_access/db_context/influxdb_v1.py"
click node_hdf5 "https://github.com/tiqi-group/icon/blob/main/src/icon/server/utils/h5py.py"
click node_sqlite_models "https://github.com/tiqi-group/icon/tree/main/src/icon/server/data_access/models/sqlite"
click node_migrations "https://github.com/tiqi-group/icon/tree/main/src/icon/server/data_access/db_context/sqlite/alembic/versions"
click node_config "https://github.com/tiqi-group/icon/blob/main/src/icon/config/config.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ui,node_socket_client toneBlue
class node_api_client,node_http_api,node_web_server,node_scheduler,node_preproc,node_hardware,node_postproc,node_config toneAmber
class node_repo_layer,node_sqlite,node_influxdb,node_hdf5,node_sqlite_models,node_migrations toneMint
class node_pycrystal_lib,node_devicesvc toneRose
```

## Components

### Frontend

React/TypeScript client served in production by the backend.

- [Web UI](https://github.com/tiqi-group/icon/tree/main/frontend/src): React app · Configuration, device/parameter management, experiment submission, monitoring, and results. · `frontend/src`
- [Live Updates](https://github.com/tiqi-group/icon/blob/main/frontend/src/socket.ts): Socket.IO client · Receives real-time job, status, and plot updates from the backend. · `frontend/src/socket.ts`

### Backend

Python control plane coordinating API, scheduling, execution, and persistence.

- [Python Client](https://github.com/tiqi-group/icon/tree/main/src/icon/client): API client · Generated Python client for programmatic access to ICON APIs. · `src/icon/client`
- [API Layer](https://github.com/tiqi-group/icon/tree/main/src/icon/server/api): HTTP controllers · Configuration, devices, experiments, parameters, scans, scheduler, status, and data endpoints. · `src/icon/server/api`
- [Web Server](https://github.com/tiqi-group/icon/tree/main/src/icon/server/web_server): HTTP + Socket.IO · Hosts the server runtime and live push channel, including frontend asset serving. · `src/icon/server/web_server`
- [Scheduler](https://github.com/tiqi-group/icon/tree/main/src/icon/server/scheduler): run coordinator · Coordinates experiment lifecycle and hands work to processing stages. · `src/icon/server/scheduler`
- [Pre-processing](https://github.com/tiqi-group/icon/tree/main/src/icon/server/pre_processing): job prep · Prepares jobs, parameters, and execution inputs. · `src/icon/server/pre_processing`
- [Hardware Exec](https://github.com/tiqi-group/icon/tree/main/src/icon/server/hardware_processing): device runner · Drives hardware-facing execution and device interaction. · `src/icon/server/hardware_processing`
- [Post-processing](https://github.com/tiqi-group/icon/tree/main/src/icon/server/post_processing): result pipeline · Collects, transforms, and finalizes experiment results. · `src/icon/server/post_processing`
- [Configuration](https://github.com/tiqi-group/icon/blob/main/src/icon/config/config.py): YAML settings · Runtime settings loaded from the user configuration file and editable from the UI. · `src/icon/config/config.py`

### Persistence

Durable stores and repositories behind the server runtime.

- [Repositories](https://github.com/tiqi-group/icon/tree/main/src/icon/server/data_access/repositories): data access · Repository boundary mediating access to jobs, devices, parameters, experiment sources, and results. · `src/icon/server/data_access/repositories`
- [SQLite](https://github.com/tiqi-group/icon/tree/main/src/icon/server/data_access/db_context/sqlite): relational store · Stores job history, device metadata, experiment sources, scan parameters, retries, and related state. · `src/icon/server/data_access/db_context/sqlite`
- [InfluxDB](https://github.com/tiqi-group/icon/blob/main/src/icon/server/data_access/db_context/influxdb_v1.py): time-series store · Persists parameter and state time-series data. · `src/icon/server/data_access/db_context/influxdb_v1.py`
- [HDF5](https://github.com/tiqi-group/icon/blob/main/src/icon/server/utils/h5py.py): results store · Stores experiment result payloads. · `src/icon/server/utils/h5py.py`
- [SQLite Models](https://github.com/tiqi-group/icon/tree/main/src/icon/server/data_access/models/sqlite): ORM schema · SQLAlchemy models for jobs, runs, devices, experiment sources, and scan parameters. · `src/icon/server/data_access/models/sqlite`
- [Migrations](https://github.com/tiqi-group/icon/tree/main/src/icon/server/data_access/db_context/sqlite/alembic/versions): schema versioning · Alembic-managed SQLite schema evolution and startup migration path. · `src/icon/server/data_access/db_context/sqlite/alembic/versions`

### External Systems

Upstream experiment metadata and laboratory services outside this repository.

- **Experiment Library**: external codebase · Upstream pycrystal-based experiment definitions parsed for metadata and sequence generation.
- **Device Services**: external services · Remote device and parameter services such as pydase-backed endpoints.

## Connections

- Web UI → API Layer: uses · User actions are translated into API calls for configuration, devices, experiments, scans, and jobs.
- Live Updates → Web Server: subscribes · Receives live state, job progress, and plot updates over Socket.IO/WebSocket-style channels.
- Python Client → API Layer: calls · Programmatic access to backend endpoints via generated client bindings.
- Web Server → API Layer: hosts · The server runtime exposes the HTTP API and serves the built frontend assets.
- API Layer → Scheduler: controls · Scheduler endpoints start, inspect, and manage experiment execution.
- API Layer → Repositories: reads/writes · API controllers persist and fetch runtime state through repositories.
- Scheduler → Pre-processing: dispatches · Schedules preprocessing work before execution.
- Scheduler → Hardware Exec: dispatches · Hands prepared runs to the hardware-facing stage.
- Scheduler → Post-processing: dispatches · Triggers result collection and transformation after execution.
- Pre-processing → Repositories: uses · Reads experiment sources, parameters, and queued jobs through repositories.
- Hardware Exec → Device Services: drives · Interacts with connected device services during experiment execution.
- Hardware Exec → Repositories: records · Writes execution state and job progress through the data-access layer.
- Post-processing → Repositories: stores · Persists final job and result metadata through repositories.
- Repositories → SQLite: persists · Repository implementations backed by relational storage.
- Repositories → InfluxDB: persists · Repository implementations backed by time-series storage.
- Repositories → HDF5: persists · Repository implementations backed by result-file storage.
- SQLite Models → SQLite: maps to · ORM schema defines the durable relational entities.
- Migrations → SQLite: migrates · Alembic revisions keep the SQLite schema versioned and upgraded on startup.
- Configuration → Web Server: configures · Runtime settings affect server binding and integration endpoints.
- Experiment Library → Repositories: metadata source · Experiment discovery and parameter extraction feed ICON's repositories and templates.
- Experiment Library → Pre-processing: defines jobs · Experiment definitions become executable jobs during preprocessing.
- Web Server → Scheduler: emits status · Asynchronous emit queue pushes scheduler and worker events to connected clients.
