# omkarcloud/botasaurus architecture

> Architecture diagram of the GitHub repository omkarcloud/botasaurus, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/omkarcloud/botasaurus
- Repository: https://github.com/omkarcloud/botasaurus

## Overview

##### Main Components Overview

a) Core Components:
- Browser Module (botasaurus/browser.py)
- Request Module (botasaurus/request.py)
- Task Module (botasaurus/task.py)
- Cache System (botasaurus/cache.py)
- Storage System (botasaurus/botasaurus_storage.py)

b) UI Components:
- Server (botasaurus_server/)
- Controls (botasaurus-controls/)
- API Layer (botasaurus_api/)

c) Utility Components:
- Sitemap Parser (botasaurus/sitemap.py)
- IP Utils (botasaurus/ip_utils.py)
- Local Storage (botasaurus/local_storage.py)

##### Component Relationships

a) Core Flow:
- User Code → Decorators (@browser, @request, @task)
- Decorators → Core Components
- Core Components → Storage/Cache Systems

b) UI Flow:
- Client → Server
- Server → Controls
- Server → API Layer
- Server → Core Components

##### Architectural Patterns

Key architectural patterns:

- Decorator Pattern (for @browser, @request, @task)
- MVC Pattern (in server component)
- Queue System (for parallel processing)
- Caching Layer (for optimization)

##### Implementation Details

Technical aspects:

- Browser automation capabilities
- Request handling system
- Parallel processing mechanism
- Caching strategy
- API endpoints
- Storage interfaces

##### Notes

Important notes about:
- Scalability features
- Security measures
- Integration points
- Configuration options

Remember to emphasize the framework's unique selling points:
- Automated UI generation
- Multiple scraping methods (browser/request/task)
- Built-in caching and parallel processing
- Comprehensive utility system

This will create a comprehensive and accurate representation of the Botasaurus framework's architecture while maintaining clarity and usefulness for technical stakeholders.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Core["Core Components"]
        Browser["Browser Module"]:::core
        Request["Request Module"]:::core
        Task["Task Module"]:::core
        Cache["Cache System"]:::core
        Storage["Storage System"]:::core
    end

    subgraph Decorators["Decorator Layer"]
        BrowserDec["Browser Decorator"]:::decorator
        RequestDec["Request Decorator"]:::decorator
        TaskDec["Task Decorator"]:::decorator
    end

    subgraph UI["UI Components"]
        Server["Server"]:::ui
        Controls["Controls"]:::ui
        API["API Layer"]:::ui
        UIComp["UI Component"]:::ui
    end

    subgraph Executors["Execution Layer"]
        TaskEx["Task Executor"]:::executor
        WorkerEx["Worker Executor"]:::executor
        MasterEx["Master Executor"]:::executor
    end

    subgraph Utils["Utility Components"]
        Sitemap["Sitemap Parser"]:::util
        IPUtils["IP Utils"]:::util
        LocalStore["Local Storage"]:::util
    end

    %% Core Relationships
    BrowserDec --> Browser
    RequestDec --> Request
    TaskDec --> Task
    Browser --> Cache
    Request --> Cache
    Task --> Cache
    Cache --> Storage

    %% UI Flow
    Controls --> Server
    Server --> API
    Server --> UIComp
    API --> Core
    
    %% Executor Relationships
    Server --> TaskEx
    TaskEx --> WorkerEx
    TaskEx --> MasterEx
    WorkerEx --> Core
    MasterEx --> Core

    %% Utility Usage
    Core --> Utils

    %% Styling
    classDef core fill:#2196F3,stroke:#1565C0,color:white
    classDef decorator fill:#FFA726,stroke:#F57C00,color:white
    classDef ui fill:#4CAF50,stroke:#2E7D32,color:white
    classDef executor fill:#9C27B0,stroke:#6A1B9A,color:white
    classDef util fill:#78909C,stroke:#546E7A,color:white

    %% Click Events
    click Browser "https://github.com/omkarcloud/botasaurus/blob/master/botasaurus/browser.py"
    click Request "https://github.com/omkarcloud/botasaurus/blob/master/botasaurus/request.py"
    click Task "https://github.com/omkarcloud/botasaurus/blob/master/botasaurus/task.py"
    click Cache "https://github.com/omkarcloud/botasaurus/blob/master/botasaurus/cache.py"
    click Storage "https://github.com/omkarcloud/botasaurus/blob/master/botasaurus/botasaurus_storage.py"
    click Server "https://github.com/omkarcloud/botasaurus/blob/master/botasaurus_server/botasaurus_server/server.py"
    click Controls "https://github.com/omkarcloud/botasaurus/blob/master/botasaurus-controls/src/index.ts"
    click API "https://github.com/omkarcloud/botasaurus/blob/master/botasaurus_api/botasaurus_api/__init__.py"
    click Sitemap "https://github.com/omkarcloud/botasaurus/blob/master/botasaurus/sitemap.py"
    click IPUtils "https://github.com/omkarcloud/botasaurus/blob/master/botasaurus/ip_utils.py"
    click LocalStore "https://github.com/omkarcloud/botasaurus/blob/master/botasaurus/local_storage.py"
    click BrowserDec "https://github.com/omkarcloud/botasaurus/blob/master/botasaurus/browser_decorator.py"
    click RequestDec "https://github.com/omkarcloud/botasaurus/blob/master/botasaurus/request_decorator.py"
    click TaskDec "https://github.com/omkarcloud/botasaurus/blob/master/botasaurus/task_decorator.py"
    click UIComp "https://github.com/omkarcloud/botasaurus/blob/master/botasaurus_server/botasaurus_server/ui.py"
    click TaskEx "https://github.com/omkarcloud/botasaurus/blob/master/botasaurus_server/botasaurus_server/task_executor.py"
    click WorkerEx "https://github.com/omkarcloud/botasaurus/blob/master/botasaurus_server/botasaurus_server/worker_executor.py"
    click MasterEx "https://github.com/omkarcloud/botasaurus/blob/master/botasaurus_server/botasaurus_server/master_executor.py"
```
