# siemens/link-checker-service architecture

> Architecture diagram of the GitHub repository siemens/link-checker-service, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/siemens/link-checker-service
- Repository: https://github.com/siemens/link-checker-service

## Overview

##### Main Components

a) Core Service Components:
- HTTP Server Layer (using Gin framework)
- URL Checker Engine
- Cache System
- Authentication Module (JWT)
- Stats Collector
- Rate Limiters (IP-based, Domain-based)

b) External Interfaces:
- Client Applications
- Target URLs/Websites
- Proxy Servers (optional)
- PAC Script Server (optional)

##### Specific Design Elements

**┌─ Client Layer ─────────────────────────────────┐**
- Web Clients | jQuery Example | Large List Check │
- ┬───────────────────────────┘
- ↓
**┌─ API Layer ─────────────────────────────────────────┐**
- /checkUrls  | /checkUrls/stream | /stats | /version │
- ┬───────────────────────────────┘
- ↓
**┌─ Service Layer ──────────────────────────────────┐**
- ┌─────────────┐ ┌──────────┐ ┌───────────────┐  │
- URL Validator│ │Dedup Logic│ │Request Handler│  │
- ┘ └──────────┘ └───────────────┘  │
- ┬───────────────────────────┘
- ↓
**┌─ Infrastructure Layer ───────────────────────────┐**
- ┌─────────┐ ┌────────────┐ ┌────────────────┐   │
- Cache  │ │Rate Limiter│ │URL Checker Pool│   │
- ┘ └────────────┘ └────────────────┘   │
- ┬───────────────────────────┘
- ↓
**┌─ External Systems Layer ────────────────────────┐**
- Target URLs | Proxy Servers | PAC Script Server │
- ┘

##### Important Flows to Highlight

a) Primary Flow:
- Client Request → API Layer → URL Validation → Deduplication → Checker Pool → External URLs
- Response path back through caching layer

b) Optimization Chain (as shown in docs/img/optimization-chain.svg):
- Rate Limiting → Deduplication → Caching → Domain Rate Limiting → Concurrent Checking

##### Notes for Implementation

- The modular plugin-based architecture for URL checking
- The multiple optimization layers
- The separation between synchronous and streaming endpoints
- The configuration flexibility (environment variables, config file)
- The health check and monitoring capabilities

Remember to maintain a balance between showing enough detail to understand the system's complexity while keeping it clear and readable at a glance.

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph Client
        WC[Web Clients]
        JQ["jQuery Example"]:::example
        LC["Large List Check"]:::example
    end

    subgraph API
        CU["/checkUrls"]:::api
        CS["/checkUrls/stream"]:::api
        ST["/stats"]:::api
        VE["/version"]:::api
    end

    subgraph Service
        SV["Server"]:::core
        UV["URL Validator"]:::core
        DL["Dedup Logic"]:::core
        RH["Request Handler"]:::core
    end

    subgraph Infrastructure
        CA["Cache System"]:::cache
        RL["Rate Limiter"]:::limiter
        CP["URL Checker Pool"]:::checker
        SC["Stats Collector"]:::stats
        PS["Plugin System"]:::plugin
    end

    subgraph External
        TU[Target URLs]
        PR[Proxy Servers]
        PA[PAC Script Server]
    end

    %% Client to API connections
    WC --> CU
    JQ --> CU
    LC --> CS
    
    %% API to Service connections
    CU --> SV
    CS --> SV
    ST --> SV
    VE --> SV
    
    %% Service Layer internal connections
    SV --> UV
    SV --> DL
    SV --> RH
    
    %% Service to Infrastructure connections
    RH --> CA
    RH --> RL
    RH --> CP
    CP --> SC
    CP --> PS
    
    %% Infrastructure to External connections
    CP --> TU
    CP --> PR
    CP --> PA

    %% Click events for component mapping
    click SV "https://github.com/siemens/link-checker-service/blob/master/server/server.go"
    click CP "https://github.com/siemens/link-checker-service/blob/master/infrastructure/url_checker.go"
    click CA "https://github.com/siemens/link-checker-service/blob/master/infrastructure/cache.go"
    click SC "https://github.com/siemens/link-checker-service/blob/master/infrastructure/stats_collector.go"
    click RL "https://github.com/siemens/link-checker-service/blob/master/infrastructure/cc_limited_url_checker.go"
    click UV "https://github.com/siemens/link-checker-service/blob/master/server/url_normalizer.go"
    click DL "https://github.com/siemens/link-checker-service/blob/master/server/deduplicator.go"
    click PS "https://github.com/siemens/link-checker-service/blob/master/infrastructure/url_checker_plugin.go"
    click JQ "https://github.com/siemens/link-checker-service/tree/master/test/jquery_example"
    click LC "https://github.com/siemens/link-checker-service/tree/master/test/large_list_check"

    %% Styling
    classDef core fill:#2374f7,stroke:#fff,stroke-width:2px,color:#fff
    classDef api fill:#87ceeb,stroke:#333,stroke-width:1px
    classDef cache fill:#ffeb3b,stroke:#333,stroke-width:1px
    classDef limiter fill:#ff9800,stroke:#333,stroke-width:1px
    classDef checker fill:#4caf50,stroke:#333,stroke-width:1px
    classDef stats fill:#9c27b0,stroke:#fff,stroke-width:1px,color:#fff
    classDef plugin fill:#795548,stroke:#fff,stroke-width:1px,color:#fff
    classDef example fill:#ddd,stroke:#333,stroke-width:1px

    %% Legend
    subgraph Legend
        L1[Core Service]:::core
        L2[API Endpoint]:::api
        L3[Cache Component]:::cache
        L4[Rate Limiter]:::limiter
        L5[Checker Component]:::checker
        L6[Stats Component]:::stats
        L7[Plugin System]:::plugin
        L8[Example Client]:::example
    end
```
