# shopify/semian architecture

> Architecture diagram of the GitHub repository shopify/semian, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shopify/semian
- Repository: https://github.com/shopify/semian

## Overview

##### Main Components Overview

a) Core Components:
- Circuit Breaker System
- Bulkhead System
- Resource Management Layer
- Adapter Layer
- Instrumentation System

b) External Integration Points:
- Database Adapters (MySQL2, Redis, etc.)
- HTTP Clients
- Active Record Integration
- SysV Semaphores (System Level)

##### Detailed Component Breakdown

Circuit Breaker Component:
**[Circuit Breaker]**
- State Management
- Closed State (Normal Operation)
- Open State (Failing Fast)
- Half-Open State (Testing Recovery)
- Error Tracking
- Success Tracking

Bulkhead Component:
**[Bulkhead System]**
- Ticket Management
- Ticket Allocation
- Ticket Release
- SysV Semaphore Integration
- Worker Registration

##### Component Relationships and Data Flow

**Client Request → Resource Access Layer**
- ↓
- Circuit Breaker Check
- ↓
- Bulkhead Check
- ↓
- Adapter-Specific Logic
- ↓
- External Resource

##### Implementation Details to Highlight

- SysV semaphore usage for cross-process coordination
- Local state management in circuit breakers
- Resource quota management
- Error threshold tracking
- Success threshold tracking

**[Client Applications]**
- ↓
**[Semian Resource Layer]**
- Circuit Breaker
- State Management
- Bulkhead
- Ticket Management
- Adapter Layer
- ↓
**[External Resources]**

##### Additional Notes

- Timing indicators for critical paths
- Configuration points
- Monitoring/metric collection points

This system design should be created using a tool that supports:
- Clear hierarchical relationships
- Bidirectional flows
- State transitions
- Component grouping
- Detailed annotations

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph Client["Client Applications"]
        App[Client Request]
    end

    subgraph Core["Core Protection Layer"]
        Resource["Resource Management"]:::resource
        Circuit["Circuit Breaker"]:::circuit
        Bulkhead["Bulkhead System"]:::bulkhead
        State["State Management"]:::state
    end

    subgraph Adapters["Adapter Layer"]
        Protected["Protected Resource"]:::adapter
        MySQL["MySQL Adapter"]:::adapter
        Redis["Redis Adapter"]:::adapter
        HTTP["HTTP Adapter"]:::adapter
        AR["ActiveRecord Adapter"]:::adapter
    end

    subgraph Implementation["System Implementation"]
        Semaphores["SysV Semaphores"]:::impl
        Tickets["Ticket Management"]:::impl
        Window["Sliding Window"]:::impl
    end

    subgraph External["External Services"]
        DB[(Databases)]
        Cache[(Redis Cache)]
        API[("HTTP APIs")]
    end

    %% Relationships
    App --> Resource
    Resource --> Circuit
    Resource --> Bulkhead
    Circuit --> State
    State --> Window
    Bulkhead --> Tickets
    Tickets --> Semaphores

    Resource --> Protected
    Protected --> MySQL & Redis & HTTP & AR
    MySQL --> DB
    Redis --> Cache
    HTTP --> API
    AR --> DB

    %% Click Events
    click Circuit "https://github.com/shopify/semian/blob/main/lib/semian/circuit_breaker.rb"
    click Resource "https://github.com/shopify/semian/blob/main/lib/semian/resource.rb"
    click Bulkhead "https://github.com/shopify/semian/blob/main/ext/semian/tickets.c"
    click Protected "https://github.com/shopify/semian/blob/main/lib/semian/protected_resource.rb"
    click State "https://github.com/shopify/semian/blob/main/lib/semian/simple_state.rb"
    click Window "https://github.com/shopify/semian/blob/main/lib/semian/simple_sliding_window.rb"
    click Semaphores "https://github.com/shopify/semian/blob/main/ext/semian/sysv_semaphores.c"
    click MySQL "https://github.com/shopify/semian/blob/main/lib/semian/mysql2.rb"
    click Redis "https://github.com/shopify/semian/tree/main/lib/semian/redis"
    click HTTP "https://github.com/shopify/semian/blob/main/lib/semian/net_http.rb"
    click AR "https://github.com/shopify/semian/blob/main/lib/semian/activerecord_trilogy_adapter.rb"

    %% Styling
    classDef resource fill:#2374ab
    classDef circuit fill:#2a9d8f
    classDef bulkhead fill:#e9c46a
    classDef state fill:#f4a261
    classDef adapter fill:#8ecae6
    classDef impl fill:#ddd
    classDef external fill:#grey

    %% Legend
    subgraph Legend
        L1["Resource Management"]:::resource
        L2["Circuit Breaker"]:::circuit
        L3["Bulkhead"]:::bulkhead
        L4["State Management"]:::state
        L5["Adapters"]:::adapter
        L6["Implementation"]:::impl
    end
```
