# jordyvandijken/vlrggapi architecture

> Architecture diagram of the GitHub repository jordyvandijken/vlrggapi, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jordyvandijken/vlrggapi
- Repository: https://github.com/jordyvandijken/vlrggapi

## Overview

##### Project type and purpose
- Project type: A thin RESTful microservice built with FastAPI, designed to scrape and re-expose content from vlr.gg.
- Purpose: Provide cached JSON endpoints for Valorant esports news, match results, rankings, stats, and streams by scraping vlr.gg.

##### File structure
- Top-level files:
- main.py – entrypoint launching the FastAPI app via uvicorn
- Dockerfile, docker-compose.yml, Procfile, app.json, runtime.txt, vercel.json – containerization & deployment configs
- requirements.txt, pip/ – Python dependencies
- Directories:
- api/ – contains scraping logic
– base_scraper.py: abstract scraper class handles HTTP requests, parsing
– scrape.py: endpoint-specific scraper subclasses
- models/ – Pydantic response schemas (models/responses.py)
- utils/ – helper functions and constants used across scrapers and endpoints
- tests/ – pytest suite for API endpoints
- CI/CD workflows:
- .github/workflows/docker*.yml – automated build & publish pipelines

##### README for architecture clues
- Technology stack: FastAPI, uvicorn, Requests, Selectolax (HTML parser)
- No persistent datastore: data is fetched & cached in memory (FastAPI’s caching or reverse proxy).
- Endpoints follow a uniform pattern: GET, 5-minute cache, JSON ‘data’ wrapper.

##### Main components
a. Client Layer
– Web or mobile clients calling the REST API
b. API Layer
– FastAPI application served by uvicorn/Gunicorn
– Endpoint routers (defined implicitly in main.py)
– Pydantic models for request/response validation
c. Scraper Layer
– BaseScraper (HTTP client + HTML parser)
– Scraper subclasses in scrape.py for each endpoint
– Utility functions/constants supporting scraping logic
d. External System
– vlr.gg website (source of truth)
e. Caching Layer
– In-memory cache (FastAPI cache middleware or similar) for 5-minute TTL
f. Infrastructure & Deployment
– Docker container
– docker-compose for local dev
– Vercel/GitHub Actions for CI/CD & hosting

##### Relationships and data flows
1) Client issues HTTP GET → API gateway (uvicorn)
2) FastAPI routes request → validate via Pydantic model → check cache
3a) Cache hit: return cached JSON
3b) Cache miss: invoke Scraper Layer
- BaseScraper fetches HTML from vlr.gg
- Selectolax parses DOM; scrape.py extracts fields
- utils/helpers apply transformations
- models/responses wrap segments into JSON
4) API returns JSON response → Client
5) CI/CD pipelines build Docker image → deploy to Vercel/AWS/GCP

##### Architectural patterns & design principles
- Layered architecture: clear separation between HTTP interface (FastAPI), business logic (scrapers), data modeling (Pydantic), and utilities.
- Single responsibility: each scraper class handles exactly one endpoint.
- Caching strategy: 5-minute TTL to reduce load on external site.
- Containerization: uniform runtime across dev/staging/prod.

##### Capturing essential details
- Parser library (Selectolax) next to the scraping component.
- Pydantic models under “Data Modeling.”
- Test suite as an auxiliary component (optional) to illustrate quality gate in CI.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Layer
    subgraph "Client Layer"
        direction TB
        Client["Web/Mobile Client"]:::client
    end

    %% API Layer
    subgraph "API Layer"
        direction TB
        Uvicorn["uvicorn/Gunicorn"]:::api
        FastAPIApp["FastAPI App"]:::api
        Router["Endpoint Routers"]:::api
        Cache["In-memory Cache (TTL=300s)"]:::cache
        PydanticModels["Pydantic Models"]:::model
    end

    %% Scraper Layer
    subgraph "Scraper Layer"
        direction TB
        BaseScraper["BaseScraper"]:::logic
        ScraperSubs["Scraper Subclasses"]:::logic
        Parser["Selectolax Parser"]:::logic
        UtilsConstants["Constants"]:::util
        UtilsHelpers["Helpers"]:::util
        UtilsMisc["Misc Utils"]:::util
    end

    %% External System
    subgraph "External System"
        direction TB
        VLR["vlr.gg Website"]:::external
    end

    %% Infrastructure & Deployment
    subgraph "Infrastructure & Deployment"
        direction TB
        DockerfileNode["Docker Container"]:::infra
        DockerCompose["docker-compose"]:::infra
        ProcfileNode["Procfile"]:::infra
        AppJson["app.json"]:::infra
        VercelJson["vercel.json"]:::infra
        RuntimeTxt["runtime.txt"]:::infra
        CI_CD1[".github/workflows/docker.yml"]:::infra
        CI_CD2[".github/workflows/docker_dev.yml"]:::infra
        Dependencies["requirements.txt & pip/"]:::infra
        TestSuite["Test Suite"]:::infra
    end

    %% Connections
    Client -->|"HTTP GET"| Uvicorn
    Uvicorn -->|forwards| FastAPIApp
    FastAPIApp -->|routes| Router
    Router -->|checks| Cache
    Cache -->|hit returns data| Router
    Cache -->|miss| BaseScraper
    BaseScraper -->|fetch HTML| VLR
    BaseScraper -->|parses| Parser
    Parser -->|extracts| ScraperSubs
    ScraperSubs -->|uses| UtilsConstants
    ScraperSubs -->|uses| UtilsHelpers
    ScraperSubs -->|uses| UtilsMisc
    ScraperSubs -->|populates| PydanticModels
    PydanticModels -->|returns JSON| Router
    Router -->|response| Client

    %% Deployment Flow
    FastAPIApp -->|containerized by| DockerfileNode
    DockerfileNode -->|orchestrated by| DockerCompose
    DockerfileNode -->|deployed via| ProcfileNode
    DockerfileNode -->|deployed via| AppJson
    DockerfileNode -->|deployed via| VercelJson
    DockerfileNode -->|runtime| RuntimeTxt
    DockerfileNode -->|built by| CI_CD1
    DockerfileNode -->|built by| CI_CD2
    TestSuite -->|quality check| CI_CD1

    %% Click Events
    click FastAPIApp "https://github.com/jordyvandijken/vlrggapi/blob/master/main.py"
    click DockerfileNode "https://github.com/jordyvandijken/vlrggapi/tree/master/Dockerfile"
    click DockerCompose "https://github.com/jordyvandijken/vlrggapi/blob/master/docker-compose.yml"
    click ProcfileNode "https://github.com/jordyvandijken/vlrggapi/tree/master/Procfile"
    click AppJson "https://github.com/jordyvandijken/vlrggapi/blob/master/app.json"
    click VercelJson "https://github.com/jordyvandijken/vlrggapi/blob/master/vercel.json"
    click RuntimeTxt "https://github.com/jordyvandijken/vlrggapi/blob/master/runtime.txt"
    click CI_CD1 "https://github.com/jordyvandijken/vlrggapi/blob/master/.github/workflows/docker.yml"
    click CI_CD2 "https://github.com/jordyvandijken/vlrggapi/blob/master/.github/workflows/docker_dev.yml"
    click BaseScraper "https://github.com/jordyvandijken/vlrggapi/blob/master/api/base_scraper.py"
    click ScraperSubs "https://github.com/jordyvandijken/vlrggapi/blob/master/api/scrape.py"
    click PydanticModels "https://github.com/jordyvandijken/vlrggapi/blob/master/models/responses.py"
    click UtilsConstants "https://github.com/jordyvandijken/vlrggapi/blob/master/utils/constants.py"
    click UtilsHelpers "https://github.com/jordyvandijken/vlrggapi/blob/master/utils/helpers.py"
    click UtilsMisc "https://github.com/jordyvandijken/vlrggapi/blob/master/utils/utils.py"
    click Dependencies "https://github.com/jordyvandijken/vlrggapi/blob/master/requirements.txt"
    click TestSuite "https://github.com/jordyvandijken/vlrggapi/blob/master/tests/test_api.py"

    %% Styles
    classDef client fill:#B3E5FC,stroke:#0288D1,color:#000
    classDef api fill:#BBDEFB,stroke:#1976D2,color:#000
    classDef cache fill:#C8E6C9,stroke:#388E3C,color:#000
    classDef logic fill:#C8E6C9,stroke:#2E7D32,color:#000
    classDef model fill:#E1BEE7,stroke:#8E24AA,color:#000
    classDef util fill:#FFF9C4,stroke:#FBC02D,color:#000
    classDef external fill:#FFE0B2,stroke:#F57C00,color:#000
    classDef infra fill:#CFD8DC,stroke:#455A64,color:#000
```
