# devslash-paul/fetchdsl architecture

> Architecture diagram of the GitHub repository devslash-paul/fetchdsl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/devslash-paul/fetchdsl
- Repository: https://github.com/devslash-paul/fetchdsl

## Overview

##### Main Components

a) Core Components:
- API Layer (from 'api' directory)
- Service Layer (from 'service' directory)
- HTTP Client Adapters
- Session Management
- Data Management

b) Supporting Components:
- Request Pipeline
- Response Pipeline
- Data Providers
- Output Handlers

##### Architecture Pattern

Top Level:
**User DSL Interface**
**↓**
**Core API Layer**
**↓**
**Service Implementation**
**↓**
**HTTP Client Adapters (Ktor)**

##### Key Relationships to Highlight

a) Request Flow:
**DSL Builder → Request Producer → HTTP Driver → Client Adapter → External HTTP Endpoints**

b) Data Flow:
**Data Suppliers (File/List) → Request Producer → Pipeline Processors → Output Handlers**

##### Detailed Component Breakdown

a) Core API Components:
- CookieJar
- SessionManager
- HttpRequest
- FetchDsl

b) Service Implementation:
- HttpDriver
- RequestProducer
- Runner

c) Pipeline Components:
- Pre-processors (pre/)
- Post-processors (post/)
- Pipes (pipes/)
- Outputs (outputs/)

d) Data Management:
- Data Suppliers
- Body Providers
- URL Providers

**[User Code/DSL Layer]**
- ↓
**[API Definition Layer]**
- ↓
**[Service Implementation Layer]**
- ↓
**[HTTP Client Adaptation Layer]**
- ↓
**[External Systems]**

##### Additional Notes

- Notes about parallel execution capabilities
- Extensibility points (adapters, processors)
- Relationship with external dependencies (Ktor)

- The DSL-first nature of the project
- The pipeline processing approach
- The flexibility in data sources and outputs
- The separation between API definition and implementation

This architecture should be represented as a combination of:

Remember to maintain a balance between showing the high-level architecture while including enough detail to understand the unique aspects of this HTTP automation tool, particularly its DSL nature and pipeline processing capabilities.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph "User Interface Layer"
        DSL["DSL Builder"]:::dsl
    end

    subgraph "API Layer"
        CookieJar["Cookie Jar"]:::api
        SessionMgr["Session Manager"]:::api
        HttpReq["HTTP Request"]:::api
        FetchDsl["Fetch DSL"]:::api
    end

    subgraph "Service Layer"
        HttpDriver["HTTP Driver"]:::service
        ReqProducer["Request Producer"]:::service
        Runner["Runner"]:::service
        HttpAdapter["HTTP Client Adapter"]:::service
        KtorAdapter["Ktor Adapter"]:::service
    end

    subgraph "Pipeline Processing"
        PreProc["Pre-processors"]:::pipeline
        PostProc["Post-processors"]:::pipeline
        Pipes["Pipes"]:::pipeline
    end

    subgraph "Data Management"
        DataSuppliers["Data Suppliers"]:::data
        BodyProviders["Body Providers"]:::data
        UrlProviders["URL Providers"]:::data
    end

    subgraph "Special Features"
        RateLimit["Rate Limiter"]:::feature
        Polling["Polling Mechanism"]:::feature
        ErrorHandler["Error Handler"]:::feature
    end

    subgraph "External Systems"
        HTTP["HTTP Endpoints"]:::external
    end

    %% Relationships
    DSL --> FetchDsl
    FetchDsl --> HttpReq
    HttpReq --> HttpDriver
    HttpDriver --> HttpAdapter
    HttpAdapter --> KtorAdapter
    KtorAdapter --> HTTP

    ReqProducer --> Runner
    Runner --> HttpDriver

    DataSuppliers --> ReqProducer
    BodyProviders --> ReqProducer
    UrlProviders --> ReqProducer

    HttpDriver --> PreProc
    PreProc --> Pipes
    Pipes --> PostProc

    SessionMgr --> CookieJar
    HttpDriver --> RateLimit
    HttpDriver --> Polling
    HttpDriver --> ErrorHandler

    %% Styles
    classDef dsl fill:#f9f,stroke:#333
    classDef api fill:#9cf,stroke:#333
    classDef service fill:#9f9,stroke:#333
    classDef pipeline fill:#fc9,stroke:#333
    classDef data fill:#ff9,stroke:#333
    classDef feature fill:#c9f,stroke:#333
    classDef external fill:#ddd,stroke:#333

    %% Click Events
    click CookieJar "https://github.com/devslash-paul/fetchdsl/blob/master/api/src/main/kotlin/net/devslash/CookieJar.kt"
    click SessionMgr "https://github.com/devslash-paul/fetchdsl/blob/master/api/src/main/kotlin/net/devslash/SessionManager.kt"
    click HttpReq "https://github.com/devslash-paul/fetchdsl/blob/master/api/src/main/kotlin/net/devslash/HttpRequest.kt"
    click FetchDsl "https://github.com/devslash-paul/fetchdsl/blob/master/api/src/main/kotlin/net/devslash/FetchDsl.kt"
    click HttpDriver "https://github.com/devslash-paul/fetchdsl/blob/master/service/src/main/kotlin/net/devslash/HttpDriver.kt"
    click ReqProducer "https://github.com/devslash-paul/fetchdsl/blob/master/service/src/main/kotlin/net/devslash/RequestProducer.kt"
    click Runner "https://github.com/devslash-paul/fetchdsl/blob/master/service/src/main/kotlin/net/devslash/Runner.kt"
    click HttpAdapter "https://github.com/devslash-paul/fetchdsl/blob/master/service/src/main/kotlin/net/devslash/HttpClientAdapter.kt"
    click KtorAdapter "https://github.com/devslash-paul/fetchdsl/blob/master/service/src/main/kotlin/net/devslash/KtorClientAdapter.kt"
    click PreProc "https://github.com/devslash-paul/fetchdsl/tree/master/service/src/main/kotlin/net/devslash/pre"
    click PostProc "https://github.com/devslash-paul/fetchdsl/tree/master/service/src/main/kotlin/net/devslash/post"
    click Pipes "https://github.com/devslash-paul/fetchdsl/tree/master/service/src/main/kotlin/net/devslash/pipes"
    click DataSuppliers "https://github.com/devslash-paul/fetchdsl/tree/master/service/src/main/kotlin/net/devslash/data"
    click BodyProviders "https://github.com/devslash-paul/fetchdsl/blob/master/service/src/main/kotlin/net/devslash/BodyProviders.kt"
    click UrlProviders "https://github.com/devslash-paul/fetchdsl/blob/master/service/src/main/kotlin/net/devslash/UrlProviders.kt"
    click RateLimit "https://github.com/devslash-paul/fetchdsl/blob/master/service/src/main/kotlin/net/devslash/RateLimiter.kt"
    click Polling "https://github.com/devslash-paul/fetchdsl/blob/master/service/src/main/kotlin/net/devslash/PollUntil.kt"
    click ErrorHandler "https://github.com/devslash-paul/fetchdsl/blob/master/api/src/main/kotlin/net/devslash/err/RetryOnTransitiveError.kt"
```
