# oxylabs/oxylabs-mcp architecture

> Architecture diagram of the GitHub repository oxylabs/oxylabs-mcp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/oxylabs/oxylabs-mcp
- Repository: https://github.com/oxylabs/oxylabs-mcp

## Overview

Based on both the file tree and the README, we’re dealing with a Python-based MCP server that acts as a bridge between AI assistants (like Claude) and Oxylabs’ web scraping services. This server receives scraping requests, processes them (rendering JavaScript when needed, cleaning HTML, parsing data, etc.) and then returns AI-ready data. It leverages two key internal “tools” – one for standard scraping (oxylabs_scraper) and one for the “web unblocker” functionality targeted at hard-to-access sites.

##### Main Components
- MCP Server: The heart of the project (as seen in src/oxylabs_mcp/server.py) which handles incoming requests. This should be represented as a core service.
- Utility Functions: Include a component representing the utility or helper functions (from src/oxylabs_mcp/utils.py) that are used for processing the scraped data.
- Scraping Modules: Distinguish two sub-components within the server:
- oxylabs_scraper – handling typical website scraping through the Oxylabs Web Scraper API.
- oxylabs_web_unblocker – dedicated to bypassing blocks and geo-restrictions using the Web Unblocker.
- Dev & Deployment Elements: Indicate Docker (through Dockerfile) for containerization, and optionally show “MCP Inspector” (accessed at http://localhost:5173) for debugging.
- Configuration & Env Setup: Note the use of environment variables (like OXYLABS_USERNAME, OXYLABS_PASSWORD) and the uv package manager which helps with dependency and environment management.

##### Architectural Patterns / Design Principles
- Modularity: The separation between server endpoints (server.py) and helper functions (utils.py) and, at a further granularity, the separation of concerns between scraping and unblocking modules.
- API Gateway/Facade Pattern: The MCP server acts as a standardized gateway for various kinds of scraping operations.
- Containerization: With the Dockerfile included, the project is designed for containerized deployment.

##### Relevant Technologies and Tools
- Python 3.12+ is a key runtime.
- uv package manager for environment and dependency management.
- Docker (via Dockerfile) for deployment.
- External Web Service endpoints from Oxylabs (Web Scraper API and Web Unblocker).

- While it is important to capture all of the key elements and interactions, avoid overcomplicating. Make sure that each module’s inputs and outputs are clearly annotated (for example, showing how parameters like url, parse, and render are used to route the request).

## Diagram (Mermaid)

```mermaid
flowchart TD
    A["Claude Desktop / AI Client"]:::externalStyle

    subgraph "MCP Server"
        direction TB
        M["Server (server.py)"]:::core
        U["Utility Functions (utils.py)"]:::utility
        subgraph "Scraping Modules"
            direction TB
            SCR["Scraping Module (oxylabs_scraper)"]:::module
            UNB["Unblocking Module (oxylabs_web_unblocker)"]:::module
        end
    end

    API_S["Oxylabs Web Scraper API"]:::api
    API_U["Oxylabs Web Unblocker API"]:::api
    I["MCP Inspector"]:::inspector
    D["Docker Container / Deployment Environment"]:::deployment
    E["Environment & Dependency Management (uv.lock)"]:::deployment

    A -->|"sends_request"| M
    M -->|"calls_util"| U
    M -->|"routes_to(scrape)"| SCR
    M -->|"routes_to(unblock)"| UNB
    SCR -->|"calls"| API_S
    UNB -->|"calls"| API_U
    API_S -->|"returns_data"| M
    API_U -->|"returns_data"| M
    M -->|"responds_to"| A
    I ---|"debugs"| M
    D -->|"hosts"| M
    E -->|"manages"| M

    click M "https://github.com/oxylabs/oxylabs-mcp/blob/main/src/oxylabs_mcp/server.py"
    click U "https://github.com/oxylabs/oxylabs-mcp/blob/main/src/oxylabs_mcp/utils.py"
    click D "https://github.com/oxylabs/oxylabs-mcp/tree/main/Dockerfile"
    click E "https://github.com/oxylabs/oxylabs-mcp/blob/main/uv.lock"

    classDef externalStyle fill:#9fceff,stroke:#1f78b4,stroke-width:2px;
    classDef core fill:#c2f0c2,stroke:#32A852,stroke-width:2px;
    classDef utility fill:#f9dcc4,stroke:#D99058,stroke-width:2px;
    classDef module fill:#f9d1d1,stroke:#D94040,stroke-width:2px;
    classDef api fill:#ffd1dc,stroke:#ff69b4,stroke-width:2px;
    classDef inspector fill:#d1e0ff,stroke:#6fa3ef,stroke-width:2px;
    classDef deployment fill:#e0e0e0,stroke:#808080,stroke-width:2px;
```
