# pranayb16/jobly architecture

> Architecture diagram of the GitHub repository pranayb16/jobly, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pranayb16/jobly
- Repository: https://github.com/pranayb16/jobly

## Overview

Jobly turns employer career-site listings into a historical, enriched hiring-intelligence dataset, then serves company, role, skill, and trend views alongside a legacy job board. The principal daily workflow is explicitly documented and orchestrated by the pipeline command: reconcile sources, crawl and retain job history, enrich a bounded queue, then snapshot companies. API and frontend details are represented at the documented capability level; several implementations were not sampled, so those links cite the README rather than implying source inspection.

Main entry points: [`backend/jobly/commands/pipeline.py`](https://github.com/pranayb16/jobly/blob/main/backend/jobly/commands/pipeline.py) (Daily Pipeline), [`backend/jobly/enrichment/worker.py`](https://github.com/pranayb16/jobly/blob/main/backend/jobly/enrichment/worker.py) (Enrichment Queue Worker), [`backend/jobly/sources/manager.py`](https://github.com/pranayb16/jobly/blob/main/backend/jobly/sources/manager.py) (Source Reconciliation), [`backend/jobly/crawling/worker.py`](https://github.com/pranayb16/jobly/blob/main/backend/jobly/crawling/worker.py) (Crawl Worker)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingestion["Source Ingestion"]
  node_pipeline["Daily Pipeline<br/>[pipeline.py]"]
  node_candidate_store[("Source Candidates")]
  node_source_manager["Source Reconciliation<br/>[manager.py]"]
  node_crawler["ATS Crawling<br/>[runner.py]"]
  node_crawl_worker["Crawl Worker<br/>[worker.py]"]
  node_job_store[("Jobs and History<br/>[repository.py]")]
  node_companies["Company Identity<br/>[repository.py]"]
end

subgraph group_enrichment["Job Enrichment"]
  node_enrichment_worker["Enrichment Queue Worker<br/>[worker.py]"]
  node_enrichment_logic["Classification and Inputs<br/>[input_builder.py]"]
  node_enrichment_store[("Job Enrichments")]
end

subgraph group_intelligence["Market Intelligence"]
  node_snapshot_builder["Daily Snapshots<br/>[snapshots.py]"]
  node_snapshot_store[("Company Snapshots")]
  node_intelligence_queries["Intelligence Queries<br/>[repository.py]"]
  node_trend_queries["Trend Queries<br/>[trends.py]"]
end

subgraph group_serving["API Serving"]
  node_api["FastAPI Application<br/>[main.py]"]
  node_intelligence_routes["Intelligence Routes<br/>[companies.py]"]
  node_legacy_jobs_api["Legacy Jobs API<br/>[routes.py]"]
end

subgraph group_frontend["Web Experience"]
  node_intelligence_web["Intelligence Pages<br/>[page.tsx]"]
  node_job_board["Job Board<br/>[JobsPage.tsx]"]
  node_jobs_api_proxy["Jobs API Routes<br/>[route.ts]"]
end

node_scheduler(("Daily Scheduler"))
node_ats["Employer ATS Sites"]
node_gemini["Gemini"]
node_web_user(("Job Seeker"))

node_scheduler -->|"triggers"| node_pipeline
node_pipeline -->|"reconciles"| node_source_manager
node_source_manager -->|"reads candidates"| node_candidate_store
node_source_manager -->|"links identity"| node_companies
node_pipeline -->|"runs crawl"| node_crawl_worker
node_crawl_worker -->|"crawls sources"| node_crawler
node_crawler -->|"fetches listings"| node_ats
node_crawl_worker -->|"saves job state"| node_job_store
node_job_store -->|"resolves company"| node_companies
node_job_store -->|"queues changes"| node_enrichment_store
node_pipeline -->|"runs enrichment"| node_enrichment_worker
node_enrichment_worker -->|"builds inputs"| node_enrichment_logic
node_enrichment_worker -->|"classifies jobs"| node_gemini
node_enrichment_worker -->|"writes results"| node_enrichment_store
node_pipeline -->|"builds snapshots"| node_snapshot_builder
node_snapshot_builder -->|"writes snapshots"| node_snapshot_store
node_intelligence_routes -->|"queries intelligence"| node_intelligence_queries
node_intelligence_routes -->|"queries trends"| node_trend_queries
node_api -->|"registers routes"| node_intelligence_routes
node_api -->|"registers routes"| node_legacy_jobs_api
node_web_user -->|"browses intelligence"| node_intelligence_web
node_web_user -->|"searches jobs"| node_job_board
node_intelligence_web -->|"requests data"| node_api
node_job_board -->|"loads listings"| node_jobs_api_proxy
node_jobs_api_proxy -->|"forwards requests"| node_legacy_jobs_api

click node_pipeline "https://github.com/pranayb16/jobly/blob/main/backend/jobly/commands/pipeline.py"
click node_source_manager "https://github.com/pranayb16/jobly/blob/main/backend/jobly/sources/manager.py"
click node_crawler "https://github.com/pranayb16/jobly/blob/main/backend/jobly/crawling/runner.py"
click node_crawl_worker "https://github.com/pranayb16/jobly/blob/main/backend/jobly/crawling/worker.py"
click node_job_store "https://github.com/pranayb16/jobly/blob/main/backend/jobly/jobs/repository.py"
click node_companies "https://github.com/pranayb16/jobly/blob/main/backend/jobly/companies/repository.py"
click node_enrichment_worker "https://github.com/pranayb16/jobly/blob/main/backend/jobly/enrichment/worker.py"
click node_enrichment_logic "https://github.com/pranayb16/jobly/blob/main/backend/jobly/enrichment/input_builder.py"
click node_snapshot_builder "https://github.com/pranayb16/jobly/blob/main/backend/jobly/intelligence/snapshots.py"
click node_intelligence_queries "https://github.com/pranayb16/jobly/blob/main/backend/jobly/intelligence/repository.py"
click node_trend_queries "https://github.com/pranayb16/jobly/blob/main/backend/jobly/intelligence/trends.py"
click node_api "https://github.com/pranayb16/jobly/blob/main/backend/jobly/api/main.py"
click node_intelligence_routes "https://github.com/pranayb16/jobly/blob/main/backend/jobly/api/routes/companies.py"
click node_legacy_jobs_api "https://github.com/pranayb16/jobly/blob/main/backend/jobly/products/jobs/routes.py"
click node_intelligence_web "https://github.com/pranayb16/jobly/blob/main/frontend/app/companies/page.tsx"
click node_job_board "https://github.com/pranayb16/jobly/blob/main/frontend/components/JobsPage.tsx"
click node_jobs_api_proxy "https://github.com/pranayb16/jobly/blob/main/frontend/app/api/jobs/route.ts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_pipeline,node_candidate_store,node_source_manager,node_crawler,node_crawl_worker,node_job_store,node_companies toneBlue
class node_enrichment_worker,node_enrichment_logic,node_enrichment_store toneAmber
class node_snapshot_builder,node_snapshot_store,node_intelligence_queries,node_trend_queries toneMint
class node_api,node_intelligence_routes,node_legacy_jobs_api,node_scheduler toneRose
class node_intelligence_web,node_job_board,node_jobs_api_proxy,node_ats,node_gemini,node_web_user toneIndigo
```

## Components

### Source Ingestion

- [Daily Pipeline](https://github.com/pranayb16/jobly/blob/main/backend/jobly/commands/pipeline.py): `backend/jobly/commands/pipeline.py`
- **Source Candidates**
- [Source Reconciliation](https://github.com/pranayb16/jobly/blob/main/backend/jobly/sources/manager.py): `backend/jobly/sources/manager.py`
- [ATS Crawling](https://github.com/pranayb16/jobly/blob/main/backend/jobly/crawling/runner.py): `backend/jobly/crawling/runner.py`
- [Crawl Worker](https://github.com/pranayb16/jobly/blob/main/backend/jobly/crawling/worker.py): `backend/jobly/crawling/worker.py`
- [Jobs and History](https://github.com/pranayb16/jobly/blob/main/backend/jobly/jobs/repository.py): `backend/jobly/jobs/repository.py`
- [Company Identity](https://github.com/pranayb16/jobly/blob/main/backend/jobly/companies/repository.py): `backend/jobly/companies/repository.py`

### Job Enrichment

- [Enrichment Queue Worker](https://github.com/pranayb16/jobly/blob/main/backend/jobly/enrichment/worker.py): `backend/jobly/enrichment/worker.py`
- [Classification and Inputs](https://github.com/pranayb16/jobly/blob/main/backend/jobly/enrichment/input_builder.py): `backend/jobly/enrichment/input_builder.py`
- **Job Enrichments**

### Market Intelligence

- [Daily Snapshots](https://github.com/pranayb16/jobly/blob/main/backend/jobly/intelligence/snapshots.py): `backend/jobly/intelligence/snapshots.py`
- **Company Snapshots**
- [Intelligence Queries](https://github.com/pranayb16/jobly/blob/main/backend/jobly/intelligence/repository.py): `backend/jobly/intelligence/repository.py`
- [Trend Queries](https://github.com/pranayb16/jobly/blob/main/backend/jobly/intelligence/trends.py): `backend/jobly/intelligence/trends.py`

### API Serving

- [FastAPI Application](https://github.com/pranayb16/jobly/blob/main/backend/jobly/api/main.py): `backend/jobly/api/main.py`
- [Intelligence Routes](https://github.com/pranayb16/jobly/blob/main/backend/jobly/api/routes/companies.py): `backend/jobly/api/routes/companies.py`
- [Legacy Jobs API](https://github.com/pranayb16/jobly/blob/main/backend/jobly/products/jobs/routes.py): `backend/jobly/products/jobs/routes.py`

### Web Experience

- [Intelligence Pages](https://github.com/pranayb16/jobly/blob/main/frontend/app/companies/page.tsx): `frontend/app/companies/page.tsx`
- [Job Board](https://github.com/pranayb16/jobly/blob/main/frontend/components/JobsPage.tsx): `frontend/components/JobsPage.tsx`
- [Jobs API Routes](https://github.com/pranayb16/jobly/blob/main/frontend/app/api/jobs/route.ts): `frontend/app/api/jobs/route.ts`

### Other components

- **Daily Scheduler**
- **Employer ATS Sites**
- **Gemini**
- **Job Seeker**

## Connections

- Daily Scheduler → Daily Pipeline: triggers (evidence: `README.md`)
- Daily Pipeline → Source Reconciliation: reconciles (evidence: `backend/jobly/commands/pipeline.py`)
- Source Reconciliation → Source Candidates: reads candidates (evidence: `README.md`)
- Source Reconciliation → Company Identity: links identity (evidence: `backend/jobly/sources/manager.py`)
- Daily Pipeline → Crawl Worker: runs crawl (evidence: `backend/jobly/commands/pipeline.py`)
- Crawl Worker → ATS Crawling: crawls sources (evidence: `backend/jobly/crawling/worker.py`)
- ATS Crawling → Employer ATS Sites: fetches listings (evidence: `README.md`)
- Crawl Worker → Jobs and History: saves job state (evidence: `backend/jobly/crawling/worker.py`)
- Jobs and History → Company Identity: resolves company (evidence: `backend/jobly/jobs/repository.py`)
- Jobs and History → Job Enrichments: queues changes (evidence: `backend/jobly/jobs/repository.py`)
- Daily Pipeline → Enrichment Queue Worker: runs enrichment (evidence: `backend/jobly/commands/pipeline.py`)
- Enrichment Queue Worker → Classification and Inputs: builds inputs (evidence: `backend/jobly/enrichment/worker.py`)
- Enrichment Queue Worker → Gemini: classifies jobs (evidence: `README.md`)
- Enrichment Queue Worker → Job Enrichments: writes results (evidence: `README.md`)
- Daily Pipeline → Daily Snapshots: builds snapshots (evidence: `backend/jobly/commands/pipeline.py`)
- Daily Snapshots → Company Snapshots: writes snapshots (evidence: `README.md`)
- Intelligence Routes → Intelligence Queries: queries intelligence (evidence: `backend/jobly/api/routes/companies.py`)
- Intelligence Routes → Trend Queries: queries trends (evidence: `backend/jobly/api/routes/companies.py`)
- FastAPI Application → Intelligence Routes: registers routes (evidence: `backend/jobly/api/main.py`)
- FastAPI Application → Legacy Jobs API: registers routes (evidence: `backend/jobly/api/main.py`)
- Job Seeker → Intelligence Pages: browses intelligence (evidence: `README.md`)
- Job Seeker → Job Board: searches jobs (evidence: `README.md`)
- Intelligence Pages → FastAPI Application: requests data (evidence: `README.md`)
- Job Board → Jobs API Routes: loads listings (evidence: `frontend/components/JobsPage.tsx`)
- Jobs API Routes → Legacy Jobs API: forwards requests (evidence: `README.md`)
