# brexhq/substation architecture

> Architecture diagram of the GitHub repository brexhq/substation, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/brexhq/substation
- Repository: https://github.com/brexhq/substation

## Overview

##### Main Components

a. Core Processing Components:
- Message Processing Engine (message/)
- Transform Engine (transform/)
- Condition Engine (condition/)
- Configuration Manager (config/)

b. Infrastructure Components:
- AWS Services Integration Layer
- HTTP Services Layer
- Storage Systems Interface
- Metrics and Monitoring

##### Key Architectural Elements

a. Input Sources:
- API Gateway
- AWS Services (Kinesis, SQS, Lambda)
- File Systems
- Standard Input

b. Processing Pipeline:
- Message Parsing
- Transformation Logic
- Condition Evaluation
- Enrichment Services

c. Output Destinations:
- AWS Services (S3, DynamoDB, etc.)
- HTTP Endpoints
- Standard Output
- File System

**[Input Layer]**

- ▼
**[Configuration Management]**

- ▼
**[Core Processing]**
- Message Processing
- Transform Engine
- Condition Engine
- Enrichment Services

- ▼
**[Output Layer]**

##### Detailed Component Relationships

- Configuration Management ↔ Core Processing
- Transform Engine ↔ External Services
- Message Processing ↔ Transform Engine

- Input Sources → Message Processing
- Transform Engine → Output Destinations

##### Implementation Details

- Serverless deployment option
- AWS service integration points
- Configuration flow using Jsonnet
- Security boundaries and encryption points

##### Deployment View

- AWS deployment architecture
- Container deployment options
- Local development setup

##### Notes

- Keep the main flow clear and centered
- Related components visually
- Version/configuration management flow
- Scale-out possibilities

- Extensible data processing
- Cloud-native routing
- Schema flexibility
- Enrichment capabilities
- Serverless deployment options

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Input
        API["API Gateway"]:::input
        AWS["AWS Services"]:::input
        FS["File Systems"]:::input
        STD["Standard Input"]:::input
    end

    subgraph Core
        MSG["Message Processing Engine"]:::core
        TRANS["Transform Engine"]:::core
        COND["Condition Engine"]:::core
        ENRICH["Enrichment Services"]:::core
    end

    subgraph Config
        CONF["Configuration Manager"]:::config
        SEC["Secret Management"]:::config
    end

    subgraph Infrastructure
        INFRA["Infrastructure Layer"]:::infra
        AWS_INT["AWS Integration"]:::infra
        HTTP["HTTP Services"]:::infra
        STORE["Storage Interface"]:::infra
        METRICS["Metrics & Monitoring"]:::infra
    end

    subgraph Output
        S3["AWS S3"]:::output
        DDB["DynamoDB"]:::output
        HTTP_OUT["HTTP Endpoints"]:::output
        STD_OUT["Standard Output"]:::output
    end

    %% Connections
    API --> MSG
    AWS --> MSG
    FS --> MSG
    STD --> MSG

    CONF --> MSG
    CONF --> TRANS
    CONF --> COND

    MSG --> TRANS
    TRANS --> COND
    COND --> ENRICH

    ENRICH --> S3
    ENRICH --> DDB
    ENRICH --> HTTP_OUT
    ENRICH --> STD_OUT

    TRANS <--> HTTP
    TRANS <--> STORE
    ENRICH <--> AWS_INT
    METRICS --> INFRA

    %% Styles
    classDef input fill:#90EE90,stroke:#333,stroke-width:2px
    classDef core fill:#87CEEB,stroke:#333,stroke-width:2px
    classDef config fill:#DDA0DD,stroke:#333,stroke-width:2px
    classDef infra fill:#D3D3D3,stroke:#333,stroke-width:2px
    classDef output fill:#FFA07A,stroke:#333,stroke-width:2px

    %% Click Events
    click MSG "https://github.com/brexhq/substation/blob/main/message/message.go"
    click TRANS "https://github.com/brexhq/substation/tree/main/transform/"
    click COND "https://github.com/brexhq/substation/tree/main/condition/"
    click CONF "https://github.com/brexhq/substation/blob/main/config/config.go"
    click AWS_INT "https://github.com/brexhq/substation/tree/main/cmd/aws/lambda/substation/"
    click HTTP "https://github.com/brexhq/substation/tree/main/internal/http/"
    click STORE "https://github.com/brexhq/substation/tree/main/internal/kv/"
    click METRICS "https://github.com/brexhq/substation/tree/main/internal/metrics/"
    click SEC "https://github.com/brexhq/substation/tree/main/internal/secrets/"
    click INFRA "https://github.com/brexhq/substation/tree/main/build/terraform/aws/"
    click API "https://github.com/brexhq/substation/tree/main/build/terraform/aws/api_gateway/"

    %% Legend
    subgraph Legend
        L1["Input Components"]:::input
        L2["Core Processing"]:::core
        L3["Configuration"]:::config
        L4["Infrastructure"]:::infra
        L5["Output Components"]:::output
    end
```
