# teyim/upwork-job-scraper architecture

> Architecture diagram of the GitHub repository teyim/upwork-job-scraper, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/teyim/upwork-job-scraper
- Repository: https://github.com/teyim/upwork-job-scraper

## Overview

##### Main Components Identification
The project is a specialized web scraping and notification system with these key components:

- Scraping Service
- Database Layer
- Telegram Notification Service
- Cron Job Scheduler
- HTTP Client Layer

##### Detailed Component Breakdown

Layer 1 - Scheduling:
**[Cron Job Scheduler] (cron-job.ts)**

Layer 2 - Data Collection:
**[Scraper Service] (lib/scraper.ts)**
**[Axios HTTP Client] (lib/axios.ts)**

Layer 3 - Data Storage:
**[MongoDB Database] (dbHelpers.ts)**

Layer 4 - Notification:
**[Telegram Service] (lib/Telegram.ts)**

##### Technical Details

Annotations for:
- TypeScript as the primary language
- Docker containerization
- Express.js framework
- Puppeteer for web scraping
- MongoDB for data persistence

##### Additional Elements

Include:
- Version information
- Docker container boundaries
- Configuration dependencies (constants.ts)

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph Infrastructure
        CRON[Cron Job Scheduler]
        DOCKER[Docker Container]
    end

    subgraph "Core Services"
        SCRAPER[Scraper Service]
        HTTP[HTTP Client]
        TELEGRAM[Telegram Service]
    end

    subgraph "Data Layer"
        DB[(MongoDB)]
        CONFIG[Configuration]
        TYPES[Type Definitions]
    end

    subgraph "External Services"
        UPWORK[Upwork API]
        TAPI[Telegram API]
    end

    %% Core Application Flow
    CRON -->|Triggers| SCRAPER
    SCRAPER -->|Fetches| UPWORK
    SCRAPER -->|Stores| DB
    SCRAPER -->|Notifies| TELEGRAM
    TELEGRAM -->|Sends| TAPI
    HTTP -->|Supports| SCRAPER

    %% Configuration Dependencies
    CONFIG -.->|Configures| SCRAPER
    CONFIG -.->|Configures| TELEGRAM
    TYPES -.->|Types| SCRAPER
    TYPES -.->|Types| TELEGRAM

    %% Infrastructure
    DOCKER -.->|Contains| SCRAPER
    DOCKER -.->|Contains| TELEGRAM
    DOCKER -.->|Contains| DB

    %% Styling
    classDef external fill:#90EE90
    classDef core fill:#ADD8E6
    classDef data fill:#FFE4B5
    classDef infra fill:#D3D3D3

    class UPWORK,TAPI external
    class SCRAPER,HTTP,TELEGRAM core
    class DB,CONFIG,TYPES data
    class CRON,DOCKER infra

    %% Click Events
    click SCRAPER "https://github.com/teyim/Upwork-job-scraper/blob/main/lib/scraper.ts"
    click DB "https://github.com/teyim/Upwork-job-scraper/blob/main/dbHelpers.ts"
    click TELEGRAM "https://github.com/teyim/Upwork-job-scraper/blob/main/lib/Telegram.ts"
    click CRON "https://github.com/teyim/Upwork-job-scraper/blob/main/cron-job.ts"
    click HTTP "https://github.com/teyim/Upwork-job-scraper/blob/main/lib/axios.ts"
    click CONFIG "https://github.com/teyim/Upwork-job-scraper/blob/main/constants.ts"
    click TYPES "https://github.com/teyim/Upwork-job-scraper/blob/main/types.ts"
    click DOCKER "https://github.com/teyim/Upwork-job-scraper/tree/main/Dockerfile"

    %% Legend
    subgraph Legend
        E[External Services]:::external
        C[Core Services]:::core
        D[Data Layer]:::data
        I[Infrastructure]:::infra
    end
```
