# honeycombio/refinery architecture

> Architecture diagram of the GitHub repository honeycombio/refinery, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/honeycombio/refinery
- Repository: https://github.com/honeycombio/refinery

## Overview

##### Primary Data Path
**Client Applications → Load Balancer → Router → Collector → Sampler → Transmitter → Honeycomb API**

##### Peer Communication Path
**Router → Sharder → Peer Refinery Instances**

##### Management Paths
**Configuration Files → Config Manager → All Components**
**Redis ↔ Peer Management System**
**Metrics/Logging → External Monitoring Systems**

##### Key Design Elements to Highlight

a. Sampling Decision Flow:
- Circular buffer for trace storage
- Sampling decision points
- Stress relief mechanism

b. Clustering Architecture:
- Peer discovery mechanism
- Trace distribution logic
- Redis integration point

c. Configuration Management:
- Config file locations
- Hot reload capability
- Environment variable override points

##### Technical Details

a. Component Interfaces:
- API endpoints (/query endpoints)
- GRPC listeners
- Peer communication protocols

b. Storage Elements:
- Circular buffers
- Trace caches
- Redis connection (if used)

c. Monitoring Points:
- Metric collection points
- Logging interfaces
- Health check endpoints

##### Notes Section

Annotations for:
- Scaling considerations
- Configuration requirements
- Performance thresholds
- Common failure points

Remember to include version information and any relevant configuration dependencies, as Refinery's behavior can change significantly based on its configuration.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph "External Interface Layer"
        LB[Load Balancer]
        GRPC["gRPC Endpoint"]
        HTTP["HTTP Endpoint"]
    end

    subgraph "Core Processing"
        Router["Router"]:::router
        Collector["Collector"]:::collector
        Sampler["Sampler"]:::sampler
        Sharder["Sharder"]:::sharder
        Transmitter["Transmitter"]:::transmitter
    end

    subgraph "Supporting Systems"
        Redis[(Redis)]
        Config["Configuration Manager"]:::config
        Metrics["Metrics System"]:::metrics
        Logger["Logging System"]:::logger
        Health["Health Monitor"]:::health
        PeerMgmt["Peer Management"]:::peer
    end

    %% External Traffic Flow
    LB --> GRPC & HTTP
    GRPC & HTTP --> Router

    %% Main Processing Flow
    Router --> Collector
    Collector --> Sampler
    Sampler --> Transmitter
    Transmitter -->|"Send to"| HoneycombAPI[("Honeycomb API")]

    %% Peer Communication
    Router --> Sharder
    Sharder --> PeerMgmt
    PeerMgmt <--> Redis
    PeerMgmt -->|"Communicate with"| PeerInstances["Peer Refinery Instances"]

    %% Supporting Systems Integration
    Config -.->|"Configure"| Router & Collector & Sampler & Sharder & Transmitter
    Metrics -.->|"Monitor"| Router & Collector & Sampler & Transmitter
    Logger -.->|"Log"| Router & Collector & Sampler & Transmitter
    Health -.->|"Check"| Router & Collector & Sampler & Transmitter

    %% Click Events
    click Router "https://github.com/honeycombio/refinery/blob/main/route/route.go"
    click Collector "https://github.com/honeycombio/refinery/blob/main/collect/collect.go"
    click Sampler "https://github.com/honeycombio/refinery/blob/main/sample/sample.go"
    click Sharder "https://github.com/honeycombio/refinery/blob/main/sharder/sharder.go"
    click Transmitter "https://github.com/honeycombio/refinery/blob/main/transmit/transmit.go"
    click Config "https://github.com/honeycombio/refinery/blob/main/config/config.go"
    click Metrics "https://github.com/honeycombio/refinery/blob/main/metrics/metrics.go"
    click Logger "https://github.com/honeycombio/refinery/blob/main/logger/logger.go"
    click PeerMgmt "https://github.com/honeycombio/refinery/blob/main/internal/peer/peers.go"
    click Health "https://github.com/honeycombio/refinery/blob/main/internal/health/health.go"

    %% Styling
    classDef router fill:#f96
    classDef collector fill:#9cf
    classDef sampler fill:#f9f
    classDef sharder fill:#ff9
    classDef transmitter fill:#9f9
    classDef config fill:#ccc
    classDef metrics fill:#fc9
    classDef logger fill:#c9c
    classDef peer fill:#99f
    classDef health fill:#f99

    %% Legend
    subgraph Legend
        L1[Router]:::router
        L2[Collector]:::collector
        L3[Sampler]:::sampler
        L4[Sharder]:::sharder
        L5[Transmitter]:::transmitter
        L6[Supporting Systems]:::config
        L7[(Database)]
    end
```
