How isc-konstanz/lories works
1. What this repository is
- lories is a Python package for local resource integration and execution systems. Its architecture is centered on collecting data from physical/remote resources, transforming it, and optionally presenting or exporting it through a higher-level application layer.
2. Overall layering
- The codebase is organized into a reusable runtime core, a data-processing layer, a connector layer for external systems, domain-specific components, and an application/ layer for user-facing pages/views.
- This creates a clear separation between infrastructure access, data flow, and presentation.
3. Bootstrap and configuration
- Startup is driven by package entrypoints (lories/__main__.py) and configuration files under conf/ plus the core configuration modules.
- The core config subsystem handles loading settings, directories, TOML-based config, logging defaults, and application-wide constants.
4. Core runtime and registry model
- The core/ and _core/ packages provide the framework primitives: application lifecycle, activators, contexts, resources, registries/registrators, and typed abstractions for components and data.
- This is the internal backbone that other layers depend on for discovery, wiring, and lifecycle management.
5. Connectors: the external-system boundary
- lories/connectors/ is the integration layer for talking to real-world systems and protocols.
- It includes adapters for CSV, SQL, InfluxDB, MQTT, OPC UA, Modbus, serial/I2C/SDI12, cameras/OpenCV, RevPi, math/virtual sources, and an entsoe connector, making connectors the main ingress/egress boundary to hardware, files, and services.
6. Data pipeline
- lories/data/ turns raw connector output into managed streams.
- Key subsystems here are channels, converters, processors, listeners, filters, databases, retention/replication, validation, and task orchestration; this is where values are normalized, enriched, persisted, and distributed.
7. Channel-to-connector flow
- The data/channels modules bridge connectors into the processing pipeline, while converters and processors reshape raw readings into usable domain values.
- The presence of listeners and retention/replication suggests the data layer supports both live reactions and longer-lived storage/duplication of processed results.
8. Domain components
- lories/components/ contains higher-level business/domain modules rather than low-level plumbing.
- The bundled weather and tariff domains show a pattern of provider-based integration: e.g. DWD/BrightSky for weather and ENTSOE/static providers for tariff data, built on top of the connector/data foundation.
9. Application and UI layer
- lories/application/ is a separate presentation boundary with settings, an interface, login/authentication, and page/component/layout modules.
- The page registry and reusable view components indicate a structured UI composition layer that consumes the underlying components/data rather than directly talking to external systems.
10. Import/export and analysis utilities
- lories/io/ provides file/output-oriented utilities such as CSV, Excel, and plotting, which are likely used for reporting, export, and visualization.
- simulation/ provides durations, progress, and result containers, suggesting offline or synthetic execution flows alongside live operation.
11. Supporting infrastructure
- The repository includes Sphinx documentation, CI/CD publishing workflows, Debian packaging, pre-commit tooling, and release/version metadata.
- These pieces support distribution and documentation, but they do not change the runtime architecture; they mainly wrap the Python package for delivery.
12. How to draw the main diagram
- A good architecture diagram should show external resources feeding into connectors, connectors feeding the data pipeline, the data pipeline feeding components and application views, and the core/config/registry layer under all of them as shared runtime infrastructure.
- Treat core/ as the control plane, connectors/ as the integration plane, data/ as the transformation plane, components/ as the domain plane, and application/ as the presentation plane.
Main entry points: connector.py Connector core · processor.py Processors · channel.py Channels · configurator.py Config
Components 20
Runtime core
Bootstrap, configuration, lifecycle, and shared registry primitives.
- Startup · Package entrypoint and bootstrap path.
lories/__main__.py - Config · Loads settings, directories, TOML config, and logging defaults.
lories/core/configs/configurator.py - Registry · Central registration and lookup backbone for runtime objects.
lories/core/register/registry.py - Activator · Coordinates application and resource lifecycle.
lories/core/activator.py - Resources · Typed resource and context abstractions used across subsystems.
lories/core/resource.py
Connectors
External-system adapters and protocol boundaries.
- Connector core · Base connector abstractions and access boundaries.
lories/connectors/connector.py - CSV · Tabular file ingestion and export connector.
lories/connectors/csv.py - SQL · Database connector family for relational access.
lories/connectors/sql/database.py - MQTT · Message-bus connector for brokered telemetry.
lories/connectors/mqtt.py
Data pipeline
Channels, conversion, processing, persistence, and reactions.
- Channels · Bridges connector output into managed data streams.
lories/data/channels/channel.py - Converters · Transforms raw readings into domain values.
lories/data/converters/converter.py - Processors · Applies transformations and higher-level processing.
lories/data/processors/processor.py - Listeners · Consumes processed values for live reactions.
lories/data/listeners/listener.py - Retention · Controls persistence, replication, and long-lived data handling.
lories/data/retention.py
Domain components
Higher-level weather and tariff modules built on the runtime and data layers.
Application UI
Presentation layer and page composition.
- Page registry · Registers and organizes pages and view composition.
lories/application/view/pages/registry.py - Views · Reusable page, layout, and component composition.
lories/application/view/pages/view.py - App main · Application-level startup and wiring for the presentation layer.
lories/application/main.py
Export I/O
File-oriented output and reporting helpers.
- CSV export · File-oriented export and reporting helper.
lories/io/csv.py
Connections 24
- Startup → Config: loads · Bootstrap reads configuration early.
- Startup → App main: starts · Entrypoint delegates into application startup.
- Config → Registry: registers · Configuration feeds registry-backed wiring.
- Config → Activator: configures · Startup settings shape lifecycle behavior.
- Activator → Resources: manages · Lifecycle operates on shared resources and contexts.
- Registry → Connector core: wires · Registry holds connector implementations.
- Registry → Channels: wires · Registry links channel definitions and handlers.
- Connector core → CSV: implements · CSV is one concrete connector family.
- Connector core → SQL: implements · SQL is one concrete connector family.
- Connector core → MQTT: implements · MQTT is one concrete connector family.
- Connector core → Channels: feeds · Connectors provide raw inputs to channels.
- Channels → Converters: normalizes · Channels hand off values for conversion.
- Converters → Processors: reshapes · Converted data flows into processing stages.
- Processors → Listeners: publishes · Processed values can trigger listeners.
- Processors → Retention: persists · Processors can hand off to storage and replication policy.
- Channels → Retention: stores · Channel data can be retained long-term.
- Connector core → Weather: supports · Weather domain depends on connector-backed providers.
- Connector core → Tariff: supports · Tariff domain depends on connector-backed providers.
- Weather → Views: renders · Weather data is presented through views.
- Tariff → Views: renders · Tariff data is presented through views.
- Page registry → Views: composes · Page registry organizes reusable views.
- App main → Page registry: serves · Application startup exposes registered pages.
- Processors → CSV export: exports · Processed results can be written for reporting.
- Retention → CSV export: serializes · Stored data can be exported to files.