# ptarmiganlabs/butler-sos architecture

> Architecture diagram of the GitHub repository ptarmiganlabs/butler-sos, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ptarmiganlabs/butler-sos
- Repository: https://github.com/ptarmiganlabs/butler-sos

## Overview

##### Project Type and Purpose
- Butler SOS is an open source monitoring tool designed to provide DevOps insights for Qlik Sense Enterprise on Windows.
- It gathers, processes, and forwards telemetry and log data, integrating with external systems such as InfluxDB, MQTT, and New Relic.
- The README confirms this by describing its use cases and linking to extended documentation.

##### Analyzing the File Structure
- Top-level indicators include configuration files (Dockerfile, .dockerignore, eslint.config.js), CI/CD configurations (GitHub workflows, Code Climate config), and detailed docs (docker-compose files, Grafana dashboards).
- The “src” directory hosts the core application code. Key files include:
- src/butler-sos.js, which likely serves as the main entry point.
- src/bundle.js, which may bundle necessary components.
- src/config containing different log appender configurations (with subdirectories for engine, proxy, repository, scheduler) that hint at different operational modes or deployment scenarios.
- src/lib has many utility modules handling logging, telemetry, event processing (e.g., udp-event.js, post-to-influxdb.js, post-to-mqtt.js, etc.) that interact with external monitoring services.
- src/plugins shows that the tool is extensible via plugins (e.g., sensible.js, support.js).
- The “static” directory includes a configuration visualizer with an HTML page and related assets, indicating a UI component or diagnostic tool.
- Docker-compose and container resources in docs and Dockerfile indicate that the project supports containerized deployments.

- Main Components:
- Core Application: Represented by src/butler-sos.js (the main application orchestrator) and supporting libraries in src/lib.
- Configuration Subsystem: Highlight the various XML configurations and YAML templates loaded from src/config.
- Plugin System: Separate block to represent extensibility (src/plugins), showing how additional functionalities are integrated.
- Data Collection & Event Handling: Modules handling UDP events (udp-event.js and udp_handlers_* files) that ingest system or application metrics.
- Integration/Output Modules: Specific libraries built for posting data (e.g., post-to-influxdb.js, post-to-mqtt.js, post-to-new-relic.js) to external monitoring endpoints.
- Visualization/UI: The static/configvis folder, which provides a web-based configuration visualization tool.
- Deployment & Containerization: Components like Dockerfile and docker-compose setups that indicate how the application is deployed in containerized environments.

- Interactions and Data Flow:
- Input/collectors gather events (possibly via UDP) and hand them off to the core engine (butler-sos.js), where the processing occurs.
- Processed data is then routed to different output modules for integration with external services.
- The configuration subsystem feeds necessary settings to both the core engine and external integration modules.
- Plugins may intercept or augment processing and reporting.
- The UI component (in static/configvis) can be used for visualizing live configurations and status.

- Architectural Patterns and Design Principles:
- Modularity: Separation into libraries (src/lib) and plugins (src/plugins) shows a modular design for easy extension and maintenance.
- Configuration-Driven Behavior: Use of multiple configuration files (XML in src/config and YAML/JSON configs) indicates that runtime behavior is highly configurable.
- Containerization: Use of a Dockerfile and documented docker-compose files implies that the deployment is designed for both local and production environments.
- Event-Driven and Telemetry: The inclusion of UDP event handlers, heartbeat, and telemetry modules is a clear indication of event-driven data collection and monitoring patterns.

- Technologies and Frameworks:
- Node.js is the likely runtime environment (JavaScript/Node usage).
- Integration points include InfluxDB (likely a time-series database), MQTT (for lightweight messaging), and New Relic (for performance monitoring).
- Docker is used for container deployment.
- CI/CD via GitHub Actions (as seen in .github/workflows) and linting/config formatting (eslint, prettier) stresses standard development processes.

- Use descriptive labels such as “Core Engine (butler-sos.js)”, “Event Ingestion”, “Configuration Manager”, “Plugin System”, “Telemetry & Logging Modules”, and “External Integration Services”.
- Configuration flow from the config subsystem to both core processing and plugins.
- Visual Differentiation:
- Incorporate Deployment Views:

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Input Layer
    subgraph "Input Layer"
        DC_UDP["UDP Event (udp-event.js)"]:::data
        UDP_LOG["UDP Log Events (udp_handlers_log_events.js)"]:::data
        UDP_USER["UDP User Activity (udp_handlers_user_activity.js)"]:::data
        CONF["Configuration Subsystem"]:::config
    end

    %% Plugin System
    subgraph "Plugin System"
        PLUG_SENSIBLE["Sensible Plugin (sensible.js)"]:::plugin
        PLUG_SUPPORT["Support Plugin (support.js)"]:::plugin
    end

    %% Core Processing
    subgraph "Core Processing"
        CORE_BUTLER["Core Engine (butler-sos.js)"]:::core
        CORE_BUNDLE["Bundle (bundle.js)"]:::core
        TELEMETRY["Telemetry & Logging Utilities"]:::telem
    end

    %% Output Layer
    subgraph "Output Layer"
        INFLUX["InfluxDB Integration (post-to-influxdb.js)"]:::output
        MQTT["MQTT Integration (post-to-mqtt.js)"]:::output
        NEWRELIC["New Relic Integration (post-to-new-relic.js)"]:::output
    end

    %% UI and Deployment
    subgraph "UI & Deployment"
        UI["Visualization/UI Component"]:::ui
        DEPLOY["Deployment & Containerization"]:::deploy
    end

    %% Data Flow Connections
    DC_UDP --> CORE_BUTLER
    UDP_LOG  --> CORE_BUTLER
    UDP_USER --> CORE_BUTLER
    CONF --> CORE_BUTLER
    CONF --> PLUG_SENSIBLE
    CONF --> PLUG_SUPPORT
    PLUG_SENSIBLE --> CORE_BUTLER
    PLUG_SUPPORT --> CORE_BUTLER
    CORE_BUNDLE --> CORE_BUTLER
    CORE_BUTLER --> TELEMETRY
    CORE_BUTLER --> INFLUX
    CORE_BUTLER --> MQTT
    CORE_BUTLER --> NEWRELIC
    CORE_BUTLER --> UI
    DEPLOY --> CORE_BUTLER
    DEPLOY --> UI

    %% Click Events
    click CORE_BUTLER "https://github.com/ptarmiganlabs/butler-sos/blob/master/src/butler-sos.js"
    click CORE_BUNDLE "https://github.com/ptarmiganlabs/butler-sos/blob/master/src/bundle.js"
    click CONF "https://github.com/ptarmiganlabs/butler-sos/tree/master/src/config"
    click PLUG_SENSIBLE "https://github.com/ptarmiganlabs/butler-sos/blob/master/src/plugins/sensible.js"
    click PLUG_SUPPORT "https://github.com/ptarmiganlabs/butler-sos/blob/master/src/plugins/support.js"
    click DC_UDP "https://github.com/ptarmiganlabs/butler-sos/blob/master/src/lib/udp-event.js"
    click UDP_LOG "https://github.com/ptarmiganlabs/butler-sos/blob/master/src/lib/udp_handlers_log_events.js"
    click UDP_USER "https://github.com/ptarmiganlabs/butler-sos/blob/master/src/lib/udp_handlers_user_activity.js"
    click INFLUX "https://github.com/ptarmiganlabs/butler-sos/blob/master/src/lib/post-to-influxdb.js"
    click MQTT "https://github.com/ptarmiganlabs/butler-sos/blob/master/src/lib/post-to-mqtt.js"
    click NEWRELIC "https://github.com/ptarmiganlabs/butler-sos/blob/master/src/lib/post-to-new-relic.js"
    click TELEMETRY "https://github.com/ptarmiganlabs/butler-sos/blob/master/src/lib/heartbeat.js"
    click UI "https://github.com/ptarmiganlabs/butler-sos/blob/master/static/configvis/index.html"
    click DEPLOY "https://github.com/ptarmiganlabs/butler-sos/tree/master/Dockerfile"
    click DEPLOY "https://github.com/ptarmiganlabs/butler-sos/blob/master/docs/docker-compose/docker-compose.yml"

    %% Styles
    classDef data fill:#DDEEFF,stroke:#000,stroke-width:1px;
    classDef config fill:#FFDDCC,stroke:#000,stroke-width:1px;
    classDef plugin fill:#FDE2E4,stroke:#000,stroke-width:1px;
    classDef core fill:#C0F5C0,stroke:#000,stroke-width:1px;
    classDef telem fill:#FFF2CC,stroke:#000,stroke-width:1px;
    classDef output fill:#D1E8E2,stroke:#000,stroke-width:1px;
    classDef ui fill:#E5CCFF,stroke:#000,stroke-width:1px;
    classDef deploy fill:#FFE6E6,stroke:#000,stroke-width:1px;
```
