# crawlbase/crawlbase-ruby architecture

> Architecture diagram of the GitHub repository crawlbase/crawlbase-ruby, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/crawlbase/crawlbase-ruby
- Repository: https://github.com/crawlbase/crawlbase-ruby

## Overview

##### Main Components Identification

a. Core Components:
- Main API Client
- Specialized API Services:
- Crawling API
- Scraper API
- Leads API
- Screenshots API
- Storage API

**[Ruby Applications] --> [Crawlbase Gem]**

**[Base API Client] <-- [Service-Specific APIs]**

**[Crawlbase Cloud Services]**

##### Specific Elements

a. Client Interface Layer:
- Main entry points (API classes)
- Initialization flow with token configuration

b. Core Components Layer:
- Base API class (lib/crawlbase/api.rb)
- Individual API service classes
- Shared utilities and configurations

c. External Integration Layer:
- Crawlbase API endpoints
- Different service endpoints (Crawling, Screenshots, Storage, etc.)

##### Connection and Flow Representation

Use the following conventions:

- Double lines (═) for inheritance relationships
- Blue: Core components
- Green: External services
- Orange: Client applications
- Gray: Utility classes

##### Detailed Component Breakdown

**[Base API Client]**
- Token configuration
- HTTP request handling
- Response processing

**[Service-Specific APIs]**
- Crawling API
- Scraper API
- Leads API
- Screenshots API
- Storage API

**[External Services]**
- Crawlbase REST APIs
- Storage Services
- Screenshot Services

##### Important Notes

- Dependency-free nature of the gem
- Error handling flow
- Response object structure
- Authentication token flow
- Different request types (GET/POST)

## Diagram (Mermaid)

```mermaid
graph TB
    %% Client Layer
    subgraph ClientLayer["Client Layer"]
        RubyApp["Ruby Applications"]
        MainEntry["Main Entry Point"]
    end

    %% Core Layer
    subgraph CoreLayer["Core Layer"]
        BaseAPI["Base API Client"]:::core
        Version["Version Management"]:::utility
        
        subgraph Services["Service-Specific APIs"]
            ScraperAPI["Scraper API"]:::service
            LeadsAPI["Leads API"]:::service
            ScreenshotsAPI["Screenshots API"]:::service
            StorageAPI["Storage API"]:::service
        end
    end

    %% External Layer
    subgraph ExternalLayer["External Services"]
        CrawlbaseAPI["Crawlbase REST APIs"]:::external
        StorageService["Storage Services"]:::external
        ScreenshotService["Screenshot Services"]:::external
    end

    %% Testing Layer
    subgraph TestingLayer["Testing Suite"]
        APITests["API Tests"]:::test
        ScreenshotsTests["Screenshots Tests"]:::test
        StorageTests["Storage Tests"]:::test
    end

    %% Relationships
    RubyApp --> MainEntry
    MainEntry --> BaseAPI
    BaseAPI --> Services
    ScraperAPI --> CrawlbaseAPI
    LeadsAPI --> CrawlbaseAPI
    ScreenshotsAPI --> ScreenshotService
    StorageAPI --> StorageService
    
    %% Testing Relationships
    APITests -.-> BaseAPI
    ScreenshotsTests -.-> ScreenshotsAPI
    StorageTests -.-> StorageAPI

    %% Click Events
    click MainEntry "https://github.com/crawlbase/crawlbase-ruby/blob/main/lib/crawlbase.rb"
    click BaseAPI "https://github.com/crawlbase/crawlbase-ruby/blob/main/lib/crawlbase/api.rb"
    click ScraperAPI "https://github.com/crawlbase/crawlbase-ruby/blob/main/lib/crawlbase/scraper_api.rb"
    click LeadsAPI "https://github.com/crawlbase/crawlbase-ruby/blob/main/lib/crawlbase/leads_api.rb"
    click ScreenshotsAPI "https://github.com/crawlbase/crawlbase-ruby/blob/main/lib/crawlbase/screenshots_api.rb"
    click StorageAPI "https://github.com/crawlbase/crawlbase-ruby/blob/main/lib/crawlbase/storage_api.rb"
    click Version "https://github.com/crawlbase/crawlbase-ruby/blob/main/lib/crawlbase/version.rb"
    click APITests "https://github.com/crawlbase/crawlbase-ruby/blob/main/spec/api_spec.rb"
    click ScreenshotsTests "https://github.com/crawlbase/crawlbase-ruby/blob/main/spec/screenshots_api_spec.rb"
    click StorageTests "https://github.com/crawlbase/crawlbase-ruby/blob/main/spec/storage_api_spec.rb"

    %% Styles
    classDef core fill:#2374ab
    classDef service fill:#48a9a6
    classDef external fill:#4caf50
    classDef test fill:#ff9800
    classDef utility fill:#9e9e9e

    %% Legend
    subgraph Legend
        CoreComponent["Core Component"]:::core
        ServiceComponent["Service Component"]:::service
        ExternalComponent["External Service"]:::external
        TestComponent["Test Component"]:::test
        UtilityComponent["Utility Component"]:::utility
    end
```
