# developer-dreamer/dorker architecture

> Architecture diagram of the GitHub repository developer-dreamer/dorker, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/developer-dreamer/dorker
- Repository: https://github.com/developer-dreamer/dorker

## Overview

Dorker collects public job listings from ATS platforms, normalizes and stores them, then analyzes listings for candidate fit and supports application-material generation and review through Telegram. The principal documented run path starts with the scraper orchestration and proceeds through configuration, concurrent ATS execution, and persistence. Matching and application generation form a separate workflow. The graph preserves the listed scraper integrations and analytics capabilities, while the available excerpts do not establish every runtime connection or the normalizer’s role in the scraper path.

Main entry points: [`src/analytics/engine.py`](https://github.com/developer-dreamer/dorker/blob/main/src/analytics/engine.py) (Matching engine), [`orchestration/scraper.py`](https://github.com/developer-dreamer/dorker/blob/main/orchestration/scraper.py) (Scraper orchestration), [`src/scraping/pipeline_engine/scraper_runner.py`](https://github.com/developer-dreamer/dorker/blob/main/src/scraping/pipeline_engine/scraper_runner.py) (ATS runner), [`src/scraping/scrapers/base.py`](https://github.com/developer-dreamer/dorker/blob/main/src/scraping/scrapers/base.py) (Scraper base)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_scraping["Job collection"]
  node_scraper_entry["Scraper orchestration<br/>[scraper.py]"]
  node_config["ATS configuration"]
  node_run_engine["Scrape run engine<br/>[engine.py]"]
  node_runner["ATS runner<br/>[scraper_runner.py]"]
  node_scraper_adapters["ATS scrapers"]
  node_scraper_base["Scraper base<br/>[base.py]"]
  node_fetch["Fetch utilities<br/>[fetch.py]"]
  node_scrape_models["Scrape job schema<br/>[models.py]"]
  node_normalizer["Job normalizer<br/>[normalizer.py]"]
end

subgraph group_analytics["Matching and applications"]
  node_matching_entry["Analytics orchestration<br/>[analyst.py]"]
  node_matching_engine["Matching engine<br/>[engine.py]"]
  node_classifiers["Job classifiers"]
  node_slm_classifier["SLM extraction"]
  node_description_filter["Description filter"]
  node_generators["Application generators"]
  node_telegram["Telegram client<br/>[telegram.py]"]
end

subgraph group_persistence["Data persistence"]
  node_scrape_store[("Scrape storage")]
  node_analytics_store[("Analytics repositories")]
end

subgraph group_shared["Shared domain"]
  node_scrape_domain["Job and company models<br/>[job.py]"]
  node_match_domain["Match and application models<br/>[match.py]"]
end

node_operator(("Operator"))

node_operator -->|"starts scraping"| node_scraper_entry
node_scraper_entry -->|"loads configuration"| node_config
node_scraper_entry -->|"starts run"| node_run_engine
node_scraper_entry -->|"configures storage"| node_scrape_store
node_scraper_entry -->|"loads companies"| node_scrape_domain
node_config -->|"resolves scraper"| node_scraper_base
node_run_engine -->|"runs ATS"| node_runner
node_run_engine -->|"uses configuration"| node_config
node_run_engine -->|"writes jobs"| node_scrape_store
node_runner -->|"uses scraper base"| node_scraper_base
node_runner -->|"uses configuration"| node_config
node_runner -->|"uses repository"| node_scrape_store
node_runner -->|"processes jobs"| node_scrape_models
node_scraper_adapters -->|"extend base"| node_scraper_base
node_scraper_base -->|"uses fetch"| node_fetch
node_scraper_base -->|"uses models"| node_scrape_models
node_matching_entry -->|"runs analytics"| node_matching_engine
node_matching_entry -->|"uses repositories"| node_analytics_store
node_matching_entry -->|"starts client"| node_telegram
node_matching_engine -->|"reads and writes"| node_analytics_store
node_matching_engine -->|"uses classifiers"| node_classifiers
node_matching_engine -->|"uses filter"| node_description_filter
node_matching_engine -->|"generates entities"| node_slm_classifier
node_matching_engine -.->|"generates materials"| node_generators
node_matching_engine -->|"saves matches"| node_match_domain
node_telegram -.->|"requests matching"| node_matching_engine
node_normalizer -.->|"normalizes jobs"| node_scrape_domain

click node_scraper_entry "https://github.com/developer-dreamer/dorker/blob/main/orchestration/scraper.py"
click node_config "https://github.com/developer-dreamer/dorker/blob/main/src/scraping/configuration_manager.py"
click node_run_engine "https://github.com/developer-dreamer/dorker/blob/main/src/scraping/pipeline_engine/engine.py"
click node_runner "https://github.com/developer-dreamer/dorker/blob/main/src/scraping/pipeline_engine/scraper_runner.py"
click node_scraper_adapters "https://github.com/developer-dreamer/dorker/tree/main/src/scraping/scrapers"
click node_scraper_base "https://github.com/developer-dreamer/dorker/blob/main/src/scraping/scrapers/base.py"
click node_fetch "https://github.com/developer-dreamer/dorker/blob/main/src/scraping/fetch.py"
click node_scrape_models "https://github.com/developer-dreamer/dorker/blob/main/src/scraping/models.py"
click node_normalizer "https://github.com/developer-dreamer/dorker/blob/main/orchestration/normalizer.py"
click node_scrape_store "https://github.com/developer-dreamer/dorker/tree/main/src/scraping/database"
click node_matching_entry "https://github.com/developer-dreamer/dorker/blob/main/orchestration/analyst.py"
click node_matching_engine "https://github.com/developer-dreamer/dorker/blob/main/src/analytics/engine.py"
click node_classifiers "https://github.com/developer-dreamer/dorker/tree/main/src/analytics/classification"
click node_slm_classifier "https://github.com/developer-dreamer/dorker/blob/main/src/analytics/classification/classify_job_tier_slm.py"
click node_description_filter "https://github.com/developer-dreamer/dorker/blob/main/src/analytics/ml/filter_description.py"
click node_generators "https://github.com/developer-dreamer/dorker/tree/main/src/analytics/generation"
click node_telegram "https://github.com/developer-dreamer/dorker/blob/main/src/client/telegram.py"
click node_analytics_store "https://github.com/developer-dreamer/dorker/tree/main/src/database/postgres"
click node_scrape_domain "https://github.com/developer-dreamer/dorker/blob/main/src/shared/models/job.py"
click node_match_domain "https://github.com/developer-dreamer/dorker/blob/main/src/shared/models/match.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_scraper_entry,node_config,node_run_engine,node_runner,node_scraper_adapters,node_scraper_base,node_fetch,node_scrape_models,node_normalizer toneBlue
class node_matching_entry,node_matching_engine,node_classifiers,node_slm_classifier,node_description_filter,node_generators,node_telegram toneAmber
class node_scrape_store,node_analytics_store toneMint
class node_scrape_domain,node_match_domain toneRose
class node_operator toneIndigo
```

## Components

### Job collection

- [Scraper orchestration](https://github.com/developer-dreamer/dorker/blob/main/orchestration/scraper.py): `orchestration/scraper.py`
- [ATS configuration](https://github.com/developer-dreamer/dorker/blob/main/src/scraping/configuration_manager.py): `src/scraping/configuration_manager.py`
- [Scrape run engine](https://github.com/developer-dreamer/dorker/blob/main/src/scraping/pipeline_engine/engine.py): `src/scraping/pipeline_engine/engine.py`
- [ATS runner](https://github.com/developer-dreamer/dorker/blob/main/src/scraping/pipeline_engine/scraper_runner.py): `src/scraping/pipeline_engine/scraper_runner.py`
- [ATS scrapers](https://github.com/developer-dreamer/dorker/tree/main/src/scraping/scrapers): `src/scraping/scrapers`
- [Scraper base](https://github.com/developer-dreamer/dorker/blob/main/src/scraping/scrapers/base.py): `src/scraping/scrapers/base.py`
- [Fetch utilities](https://github.com/developer-dreamer/dorker/blob/main/src/scraping/fetch.py): `src/scraping/fetch.py`
- [Scrape job schema](https://github.com/developer-dreamer/dorker/blob/main/src/scraping/models.py): `src/scraping/models.py`
- [Job normalizer](https://github.com/developer-dreamer/dorker/blob/main/orchestration/normalizer.py): `orchestration/normalizer.py`

### Matching and applications

- [Analytics orchestration](https://github.com/developer-dreamer/dorker/blob/main/orchestration/analyst.py): `orchestration/analyst.py`
- [Matching engine](https://github.com/developer-dreamer/dorker/blob/main/src/analytics/engine.py): `src/analytics/engine.py`
- [Job classifiers](https://github.com/developer-dreamer/dorker/tree/main/src/analytics/classification): `src/analytics/classification`
- [SLM extraction](https://github.com/developer-dreamer/dorker/blob/main/src/analytics/classification/classify_job_tier_slm.py): `src/analytics/classification/classify_job_tier_slm.py`
- [Description filter](https://github.com/developer-dreamer/dorker/blob/main/src/analytics/ml/filter_description.py): `src/analytics/ml/filter_description.py`
- [Application generators](https://github.com/developer-dreamer/dorker/tree/main/src/analytics/generation): `src/analytics/generation`
- [Telegram client](https://github.com/developer-dreamer/dorker/blob/main/src/client/telegram.py): `src/client/telegram.py`

### Data persistence

- [Scrape storage](https://github.com/developer-dreamer/dorker/tree/main/src/scraping/database): `src/scraping/database`
- [Analytics repositories](https://github.com/developer-dreamer/dorker/tree/main/src/database/postgres): `src/database/postgres`

### Shared domain

- [Job and company models](https://github.com/developer-dreamer/dorker/blob/main/src/shared/models/job.py): `src/shared/models/job.py`
- [Match and application models](https://github.com/developer-dreamer/dorker/blob/main/src/shared/models/match.py): `src/shared/models/match.py`

### Other components

- **Operator**

## Connections

- Operator → Scraper orchestration: starts scraping
- Scraper orchestration → ATS configuration: loads configuration (evidence: `orchestration/scraper.py`)
- Scraper orchestration → Scrape run engine: starts run (evidence: `orchestration/scraper.py`)
- Scraper orchestration → Scrape storage: configures storage (evidence: `orchestration/scraper.py`)
- Scraper orchestration → Job and company models: loads companies (evidence: `orchestration/scraper.py`)
- ATS configuration → Scraper base: resolves scraper (evidence: `src/scraping/configuration_manager.py`)
- Scrape run engine → ATS runner: runs ATS (evidence: `src/scraping/pipeline_engine/engine.py`)
- Scrape run engine → ATS configuration: uses configuration (evidence: `src/scraping/pipeline_engine/engine.py`)
- Scrape run engine → Scrape storage: writes jobs (evidence: `src/scraping/pipeline_engine/engine.py`)
- ATS runner → Scraper base: uses scraper base (evidence: `src/scraping/pipeline_engine/scraper_runner.py`)
- ATS runner → ATS configuration: uses configuration (evidence: `src/scraping/pipeline_engine/scraper_runner.py`)
- ATS runner → Scrape storage: uses repository (evidence: `src/scraping/pipeline_engine/scraper_runner.py`)
- ATS runner → Scrape job schema: processes jobs (evidence: `src/scraping/pipeline_engine/scraper_runner.py`)
- ATS scrapers → Scraper base: extend base (evidence: `src/scraping/scrapers/avature.py`)
- Scraper base → Fetch utilities: uses fetch (evidence: `src/scraping/scrapers/base.py`)
- Scraper base → Scrape job schema: uses models (evidence: `src/scraping/scrapers/base.py`)
- Analytics orchestration → Matching engine: runs analytics (evidence: `orchestration/analyst.py`)
- Analytics orchestration → Analytics repositories: uses repositories (evidence: `orchestration/analyst.py`)
- Analytics orchestration → Telegram client: starts client (evidence: `orchestration/analyst.py`)
- Matching engine → Analytics repositories: reads and writes (evidence: `src/analytics/engine.py`)
- Matching engine → Job classifiers: uses classifiers (evidence: `src/analytics/engine.py`)
- Matching engine → Description filter: uses filter (evidence: `src/analytics/engine.py`)
- Matching engine → SLM extraction: generates entities (evidence: `src/analytics/engine.py`)
- Matching engine → Application generators: generates materials
- Matching engine → Match and application models: saves matches (evidence: `src/analytics/engine.py`)
- Telegram client → Matching engine: requests matching
- Job normalizer → Job and company models: normalizes jobs
