# daq-tools/kotori architecture

> Architecture diagram of the GitHub repository daq-tools/kotori, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/daq-tools/kotori
- Repository: https://github.com/daq-tools/kotori

## Overview

1. Kotori is a Python telemetry broker built around Twisted’s async runtime. The repo centers on acquiring sensor data from multiple protocols, normalizing it into a common channel model, storing it in time-series databases, and exposing it for export and visualization.

2. The main runtime entry points are in `kotori/core.py` and the DAQ application layer under `kotori/daq/application/`. These application modules (`basic`, `composite`, `mqttkit`) assemble a deployment from configuration: protocol listeners, decoders, storage backends, and optional graphing/export services.

3. Ingestion is split across protocol adapters in `kotori/daq/intercom/` and `kotori/io/protocol/`. Supported ingress paths include MQTT, HTTP, UDP, and WAMP. The repository also includes a Crossbar config, which indicates WAMP/Crossbar is a first-class integration point for some deployments.

4. Incoming payloads are routed through a channel/path abstraction in `kotori/io/router/path.py` and then normalized by decoders in `kotori/daq/decoder/`. The decoder layer contains vendor-specific parsers for Airrohr, Tasmota, and The Things Network uplinks, plus schema helpers for turning heterogeneous payloads into a common measurement structure.

5. The normalized data model appears to be channel-oriented and time-series-friendly: data is associated with hierarchical channel paths, tags, and timestamps. The `strategy` modules (`lan`, `wan`) suggest different addressing/routing conventions for local-network versus wide-area sensor deployments.

6. Persistence is handled primarily by `kotori/daq/storage/influx.py`, with utility code around it. InfluxDB is the core storage backend in the architecture; the docs and tests also show the project’s time-series export and graphing workflows are built around Influx-compatible data.

7. Visualization and dashboard management live in `kotori/daq/graphing/grafana/`. This subsystem talks to Grafana through its API, manages dashboards/panels/annotations, and packages Grafana resource templates. That makes Grafana an external service Kotori can provision or update automatically rather than merely query.

8. Data export is a separate subsystem under `kotori/io/export/`. It supports tabular output, HTML rendering, plots, and database-backed export. This is the main boundary for “read-only” retrieval of historical measurements, distinct from the ingestion/storage path.

9. There is a small frontend package in `kotori/frontend/` with templates, static assets, and a server/app wrapper. It looks like a lightweight admin or status UI rather than the core data path, so it sits beside the backend services instead of in-line with acquisition.

10. Firmware support is another explicit boundary: `kotori/firmware/` contains builder/service code, and the repo includes Arduino and PHP client examples under `clients/`. This indicates Kotori is not only a server but also a toolchain for generating or supporting device-side payload producers.

11. Configuration is file-driven. The `etc/` tree contains base, production, test, and example `.ini` files for different deployment profiles, plus vendor-specific presets. That configuration layer is the main way the same codebase is adapted to different telemetry scenarios.

12. Deployment and packaging are a major part of the repo architecture: Dockerfiles, `docker-compose.yml`, systemd units, Debian packaging scripts, and wheel build scripts are all present. The project is designed to run either as a packaged service or inside containerized infrastructure.

13. The `doc/` tree is extensive and effectively documents the system’s intended topology: sensor clients send MQTT/HTTP payloads into Kotori, Kotori stores and transforms them, and Grafana/InfluxDB provide the downstream analytics and visualization stack. The tests mirror this architecture with protocol, decoder, storage, and export coverage.

Main entry points: [`kotori/core.py`](https://github.com/daq-tools/kotori/blob/main/kotori/core.py) (Core), [`kotori/io/router/path.py`](https://github.com/daq-tools/kotori/blob/main/kotori/io/router/path.py) (Path router), [`etc`](https://github.com/daq-tools/kotori/tree/main/etc) (Config & delivery), [`kotori/daq/application/basic.py`](https://github.com/daq-tools/kotori/blob/main/kotori/daq/application/basic.py) (Basic app)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Runtime"]
  node_core["Core<br/>entrypoint<br/>[core.py]"]
  node_app_basic["Basic app<br/>app assembly<br/>[basic.py]"]
  node_app_composite["Composite app<br/>app assembly<br/>[composite.py]"]
  node_app_mqttkit["MQTT kit<br/>app assembly<br/>[mqttkit.py]"]
end

subgraph group_ingest["Ingest & Normalize"]
  node_mqtt_ingest["MQTT ingest<br/>protocol adapter"]
  node_http_ingest["HTTP ingest<br/>protocol adapter<br/>[http.py]"]
  node_udp_wamp["UDP/WAMP<br/>protocol adapter<br/>[udp.py]"]
  node_bus_wamp["WAMP bus<br/>bus integration<br/>[wamp.py]"]
  node_path_router["Path router<br/>[path.py]"]
  node_decoders["Decoders<br/>normalization"]
  node_strategies["Strategies<br/>routing model"]
end

subgraph group_storage["Storage & Viz"]
  node_influx_storage[("Influx storage<br/>time-series backend<br/>[influx.py]")]
  node_exporters["Export tools<br/>export subsystem"]
  node_grafana_mgr["Grafana mgr<br/>dashboard automation<br/>[manager.py]"]
  node_grafana_api["Grafana API<br/>service client<br/>[api.py]"]
end

subgraph group_interfaces["Interfaces"]
  node_frontend_app["Frontend<br/>web ui<br/>[app.py]"]
  node_firmware["Firmware<br/>device toolchain"]
end

subgraph group_delivery["Config & Delivery"]
  node_config_packaging["Config & delivery<br/>deployment assets"]
end

node_core -->|"boots"| node_app_basic
node_core -->|"boots"| node_app_composite
node_core -->|"boots"| node_app_mqttkit
node_app_basic -->|"wires"| node_mqtt_ingest
node_app_basic -->|"wires"| node_http_ingest
node_app_composite -->|"wires"| node_udp_wamp
node_app_composite -->|"integrates"| node_bus_wamp
node_mqtt_ingest -->|"routes"| node_path_router
node_http_ingest -->|"routes"| node_path_router
node_udp_wamp -->|"routes"| node_path_router
node_path_router -->|"feeds"| node_decoders
node_decoders -->|"interprets"| node_strategies
node_decoders -->|"writes"| node_influx_storage
node_influx_storage -->|"serves"| node_exporters
node_influx_storage -->|"powers"| node_grafana_mgr
node_grafana_mgr -->|"calls"| node_grafana_api
node_exporters -->|"renders"| node_frontend_app
node_config_packaging -->|"configures"| node_core
node_config_packaging -->|"packages"| node_frontend_app
node_config_packaging -->|"supports"| node_firmware

click node_core "https://github.com/daq-tools/kotori/blob/main/kotori/core.py"
click node_app_basic "https://github.com/daq-tools/kotori/blob/main/kotori/daq/application/basic.py"
click node_app_composite "https://github.com/daq-tools/kotori/blob/main/kotori/daq/application/composite.py"
click node_app_mqttkit "https://github.com/daq-tools/kotori/blob/main/kotori/daq/application/mqttkit.py"
click node_mqtt_ingest "https://github.com/daq-tools/kotori/tree/main/kotori/daq/intercom/mqtt"
click node_http_ingest "https://github.com/daq-tools/kotori/blob/main/kotori/io/protocol/http.py"
click node_udp_wamp "https://github.com/daq-tools/kotori/blob/main/kotori/daq/intercom/udp.py"
click node_bus_wamp "https://github.com/daq-tools/kotori/blob/main/kotori/bus/wamp.py"
click node_path_router "https://github.com/daq-tools/kotori/blob/main/kotori/io/router/path.py"
click node_decoders "https://github.com/daq-tools/kotori/tree/main/kotori/daq/decoder"
click node_strategies "https://github.com/daq-tools/kotori/tree/main/kotori/daq/strategy"
click node_influx_storage "https://github.com/daq-tools/kotori/blob/main/kotori/daq/storage/influx.py"
click node_exporters "https://github.com/daq-tools/kotori/tree/main/kotori/io/export"
click node_grafana_mgr "https://github.com/daq-tools/kotori/blob/main/kotori/daq/graphing/grafana/manager.py"
click node_grafana_api "https://github.com/daq-tools/kotori/blob/main/kotori/daq/graphing/grafana/api.py"
click node_frontend_app "https://github.com/daq-tools/kotori/blob/main/kotori/frontend/app.py"
click node_firmware "https://github.com/daq-tools/kotori/tree/main/kotori/firmware"
click node_config_packaging "https://github.com/daq-tools/kotori/tree/main/etc"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_core,node_app_basic,node_app_composite,node_app_mqttkit toneBlue
class node_mqtt_ingest,node_http_ingest,node_udp_wamp,node_bus_wamp,node_path_router,node_decoders,node_strategies toneAmber
class node_influx_storage,node_exporters,node_grafana_mgr,node_grafana_api toneMint
class node_frontend_app,node_firmware toneRose
class node_config_packaging toneIndigo
```

## Components

### Runtime

Core process entry points and application assembly

- [Core](https://github.com/daq-tools/kotori/blob/main/kotori/core.py): entrypoint · Main runtime bootstrap and orchestration · `kotori/core.py`
- [Basic app](https://github.com/daq-tools/kotori/blob/main/kotori/daq/application/basic.py): app assembly · Minimal telemetry deployment wiring · `kotori/daq/application/basic.py`
- [Composite app](https://github.com/daq-tools/kotori/blob/main/kotori/daq/application/composite.py): app assembly · Multi-service deployment wiring · `kotori/daq/application/composite.py`
- [MQTT kit](https://github.com/daq-tools/kotori/blob/main/kotori/daq/application/mqttkit.py): app assembly · MQTT-oriented deployment profile · `kotori/daq/application/mqttkit.py`

### Ingest & Normalize

Protocol listeners, routing, decoders, and addressing strategies

- [MQTT ingest](https://github.com/daq-tools/kotori/tree/main/kotori/daq/intercom/mqtt): protocol adapter · MQTT listeners and runtime adapters · `kotori/daq/intercom/mqtt`
- [HTTP ingest](https://github.com/daq-tools/kotori/blob/main/kotori/io/protocol/http.py): protocol adapter · HTTP acquisition and forwarding path · `kotori/io/protocol/http.py`
- [UDP/WAMP](https://github.com/daq-tools/kotori/blob/main/kotori/daq/intercom/udp.py): protocol adapter · Datagram and bus ingress paths · `kotori/daq/intercom/udp.py`
- [WAMP bus](https://github.com/daq-tools/kotori/blob/main/kotori/bus/wamp.py): bus integration · Crossbar/WAMP integration layer · `kotori/bus/wamp.py`
- [Path router](https://github.com/daq-tools/kotori/blob/main/kotori/io/router/path.py): router · Channel and path normalization · `kotori/io/router/path.py`
- [Decoders](https://github.com/daq-tools/kotori/tree/main/kotori/daq/decoder): normalization · Vendor-specific payload parsers · `kotori/daq/decoder`
- [Strategies](https://github.com/daq-tools/kotori/tree/main/kotori/daq/strategy): routing model · LAN and WAN addressing conventions · `kotori/daq/strategy`

### Storage & Viz

Persistence, exports, and Grafana integration

- [Influx storage](https://github.com/daq-tools/kotori/blob/main/kotori/daq/storage/influx.py): time-series backend · Primary persistence backend · `kotori/daq/storage/influx.py`
- [Export tools](https://github.com/daq-tools/kotori/tree/main/kotori/io/export): export subsystem · Tabular, HTML, plot, and database export · `kotori/io/export`
- [Grafana mgr](https://github.com/daq-tools/kotori/blob/main/kotori/daq/graphing/grafana/manager.py): dashboard automation · Grafana API, dashboards, panels, and resources · `kotori/daq/graphing/grafana/manager.py`
- [Grafana API](https://github.com/daq-tools/kotori/blob/main/kotori/daq/graphing/grafana/api.py): service client · External Grafana service interaction · `kotori/daq/graphing/grafana/api.py`

### Interfaces

Frontend UI and firmware support boundaries

- [Frontend](https://github.com/daq-tools/kotori/blob/main/kotori/frontend/app.py): web ui · Lightweight admin/status interface · `kotori/frontend/app.py`
- [Firmware](https://github.com/daq-tools/kotori/tree/main/kotori/firmware): device toolchain · Firmware builder and service support · `kotori/firmware`

### Config & Delivery

Deployment, packaging, and configuration assets

- [Config & delivery](https://github.com/daq-tools/kotori/tree/main/etc): deployment assets · INI configs, Docker, systemd, and packaging scripts · `etc`

## Connections

- Core → Basic app: boots · Core starts the basic deployment profile
- Core → Composite app: boots · Core starts the composite deployment profile
- Core → MQTT kit: boots · Core starts the MQTT-kit deployment profile
- Basic app → MQTT ingest: wires · Assembles MQTT listeners into the runtime
- Basic app → HTTP ingest: wires · Adds HTTP acquisition endpoints
- Composite app → UDP/WAMP: wires · Combines datagram and bus ingress
- Composite app → WAMP bus: integrates · Connects to Crossbar/WAMP deployments
- MQTT ingest → Path router: routes · Maps incoming messages to channel paths
- HTTP ingest → Path router: routes · Normalizes HTTP payload paths
- UDP/WAMP → Path router: routes · Routes UDP and WAMP payloads into channels
- Path router → Decoders: feeds · Hands normalized payloads to decoders
- Decoders → Strategies: interprets · Applies LAN/WAN naming and routing conventions
- Decoders → Influx storage: writes · Produces measurements for time-series persistence
- Influx storage → Export tools: serves · Provides read-side data for export workflows
- Influx storage → Grafana mgr: powers · Backs dashboard and annotation provisioning
- Grafana mgr → Grafana API: calls · Uses the external Grafana API
- Export tools → Frontend: renders · Feeds admin or status presentation paths
- Config & delivery → Core: configures · Deployment files and packaging shape runtime startup
- Config & delivery → Frontend: packages · Frontend assets and service wiring are deployed alongside the backend
- Config & delivery → Firmware: supports · Includes device-side support material and builders
