# dhanush-10-1/jobpulse architecture

> Architecture diagram of the GitHub repository dhanush-10-1/jobpulse, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dhanush-10-1/jobpulse
- Repository: https://github.com/dhanush-10-1/jobpulse

## Overview

JobPulse ingests Python.org listings, validates and parses them, then upserts canonical-URL records and retains stale listings as inactive history. Its principal user flow is the React job explorer fetching through the FastAPI service and presenting searchable, paginated listings and analytics. Separate AI capabilities cover enrichment, embeddings, semantic search, matching, and resume-based recommendations; Kafka event handling and scheduled Airflow ingestion are additional paths. The graph follows shown calls and SOURCE INDEX references; several AI and messaging internals were not excerpted, so uncertain links are omitted.

Main entry points: [`jobpulse/database/repository.py`](https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/database/repository.py) (PostgreSQL job store), [`jobpulse/ingestion/pipeline.py`](https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ingestion/pipeline.py) (Ingestion pipeline), [`frontend/src/App.jsx`](https://github.com/dhanush-10-1/jobpulse/blob/main/frontend/src/App.jsx) (Explorer app), [`api.py`](https://github.com/dhanush-10-1/jobpulse/blob/main/api.py) (FastAPI service)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Ingestion"]
  node_pipeline["Ingestion pipeline<br/>[pipeline.py]"]
  node_fetch["HTML fetcher<br/>[fetch.py]"]
  node_parser["Listing parser<br/>[parser.py]"]
  node_jobmodel["Job model<br/>[models.py]"]
  node_export["JSON exporter<br/>[exporter.py]"]
end

subgraph group_data["Data services"]
  node_db[("PostgreSQL job store<br/>[repository.py]")]
  node_connection["DB connection<br/>[connections.py]"]
  node_config["Configuration<br/>[config.py]"]
end

subgraph group_ai["AI capabilities"]
  node_enrichment["Job enrichment<br/>[enrichment.py]"]
  node_embedding["Job embeddings<br/>[embeddings.py]"]
  node_search["Semantic search<br/>[search.py]"]
  node_matching["Job matching<br/>[matching.py]"]
  node_resume["Resume analysis<br/>[resume.py]"]
  node_recommend["Recommendations<br/>[recommendations.py]"]
  node_aiclient["Gemini client<br/>[client.py]"]
end

subgraph group_web["Job explorer"]
  node_api["FastAPI service<br/>[api.py]"]
  node_app["Explorer app<br/>[App.jsx]"]
  node_filters["Search filters<br/>[SearchFilters.jsx]"]
  node_jobcard["Job cards<br/>[JobCard.jsx]"]
  node_details["Job details<br/>[JobDetails.jsx]"]
  node_pagination["Pagination<br/>[Pagination.jsx]"]
  node_analytics["Analytics dashboard"]
end

subgraph group_ops["Event and scheduling"]
  node_dag["Airflow DAG"]
  node_kafka["Kafka producer<br/>[producer.py]"]
  node_consumer["Kafka consumer<br/>[consumer.py]"]
end

node_operator(("CLI operator"))
node_pythonjobs(("Python.org jobs"))
node_gemini(("Gemini API"))

node_operator -->|"runs"| node_pipeline
node_pipeline -->|"fetches HTML"| node_fetch
node_fetch -->|"requests listings"| node_pythonjobs
node_pipeline -->|"parses listings"| node_parser
node_parser -->|"creates jobs"| node_jobmodel
node_pipeline -->|"upserts jobs"| node_db
node_pipeline -->|"exports jobs"| node_export
node_dag -->|"orchestrates"| node_pipeline
node_consumer -->|"upserts events"| node_db
node_app -->|"renders"| node_filters
node_app -->|"renders"| node_jobcard
node_app -->|"renders"| node_details
node_app -->|"renders"| node_pagination
node_app -->|"renders"| node_analytics
node_app -.->|"requests jobs"| node_api
node_api -->|"dispatches"| node_matching
node_api -->|"dispatches"| node_search
node_api -->|"queries jobs"| node_db
node_db -->|"connects through"| node_connection
node_pipeline -->|"reads settings"| node_config
node_embedding -->|"requests vectors"| node_aiclient
node_embedding -->|"saves vectors"| node_db
node_aiclient -->|"calls provider"| node_gemini
node_enrichment -->|"reads and saves"| node_db
node_enrichment -->|"uses client"| node_aiclient
node_matching -->|"reads jobs"| node_db
node_search -->|"searches vectors"| node_db
node_resume -->|"uses client"| node_aiclient
node_resume -.->|"requests analysis"| node_gemini
node_recommend -.->|"uses job data"| node_db

click node_pipeline "https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ingestion/pipeline.py"
click node_fetch "https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ingestion/fetch.py"
click node_parser "https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ingestion/parser.py"
click node_jobmodel "https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ingestion/models.py"
click node_export "https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/storage/exporter.py"
click node_dag "https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/dags/jobpulse_ingestion.py"
click node_kafka "https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/messaging/producer.py"
click node_consumer "https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/messaging/consumer.py"
click node_api "https://github.com/dhanush-10-1/jobpulse/blob/main/api.py"
click node_app "https://github.com/dhanush-10-1/jobpulse/blob/main/frontend/src/App.jsx"
click node_filters "https://github.com/dhanush-10-1/jobpulse/blob/main/frontend/src/components/SearchFilters.jsx"
click node_jobcard "https://github.com/dhanush-10-1/jobpulse/blob/main/frontend/src/components/JobCard.jsx"
click node_details "https://github.com/dhanush-10-1/jobpulse/blob/main/frontend/src/components/JobDetails.jsx"
click node_pagination "https://github.com/dhanush-10-1/jobpulse/blob/main/frontend/src/components/Pagination.jsx"
click node_analytics "https://github.com/dhanush-10-1/jobpulse/blob/main/frontend/src/components/AnalyticsDashboard.jsx"
click node_db "https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/database/repository.py"
click node_connection "https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/database/connections.py"
click node_enrichment "https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ai/enrichment.py"
click node_embedding "https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ai/embeddings.py"
click node_search "https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ai/search.py"
click node_matching "https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ai/matching.py"
click node_resume "https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ai/resume.py"
click node_recommend "https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ai/recommendations.py"
click node_aiclient "https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ai/client.py"
click node_config "https://github.com/dhanush-10-1/jobpulse/blob/main/config.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_pipeline,node_fetch,node_parser,node_jobmodel,node_export toneBlue
class node_db,node_connection,node_config toneAmber
class node_enrichment,node_embedding,node_search,node_matching,node_resume,node_recommend,node_aiclient,node_gemini toneMint
class node_api,node_app,node_filters,node_jobcard,node_details,node_pagination,node_analytics toneRose
class node_dag,node_kafka,node_consumer,node_operator,node_pythonjobs toneIndigo
```

## Components

### Ingestion

- [Ingestion pipeline](https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ingestion/pipeline.py): `jobpulse/ingestion/pipeline.py`
- [HTML fetcher](https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ingestion/fetch.py): `jobpulse/ingestion/fetch.py`
- [Listing parser](https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ingestion/parser.py): `jobpulse/ingestion/parser.py`
- [Job model](https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ingestion/models.py): `jobpulse/ingestion/models.py`
- [JSON exporter](https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/storage/exporter.py): `jobpulse/storage/exporter.py`

### Data services

- [PostgreSQL job store](https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/database/repository.py): `jobpulse/database/repository.py`
- [DB connection](https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/database/connections.py): `jobpulse/database/connections.py`
- [Configuration](https://github.com/dhanush-10-1/jobpulse/blob/main/config.py): `config.py`

### AI capabilities

- [Job enrichment](https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ai/enrichment.py): `jobpulse/ai/enrichment.py`
- [Job embeddings](https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ai/embeddings.py): `jobpulse/ai/embeddings.py`
- [Semantic search](https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ai/search.py): `jobpulse/ai/search.py`
- [Job matching](https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ai/matching.py): `jobpulse/ai/matching.py`
- [Resume analysis](https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ai/resume.py): `jobpulse/ai/resume.py`
- [Recommendations](https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ai/recommendations.py): `jobpulse/ai/recommendations.py`
- [Gemini client](https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/ai/client.py): `jobpulse/ai/client.py`

### Job explorer

- [FastAPI service](https://github.com/dhanush-10-1/jobpulse/blob/main/api.py): `api.py`
- [Explorer app](https://github.com/dhanush-10-1/jobpulse/blob/main/frontend/src/App.jsx): `frontend/src/App.jsx`
- [Search filters](https://github.com/dhanush-10-1/jobpulse/blob/main/frontend/src/components/SearchFilters.jsx): `frontend/src/components/SearchFilters.jsx`
- [Job cards](https://github.com/dhanush-10-1/jobpulse/blob/main/frontend/src/components/JobCard.jsx): `frontend/src/components/JobCard.jsx`
- [Job details](https://github.com/dhanush-10-1/jobpulse/blob/main/frontend/src/components/JobDetails.jsx): `frontend/src/components/JobDetails.jsx`
- [Pagination](https://github.com/dhanush-10-1/jobpulse/blob/main/frontend/src/components/Pagination.jsx): `frontend/src/components/Pagination.jsx`
- [Analytics dashboard](https://github.com/dhanush-10-1/jobpulse/blob/main/frontend/src/components/AnalyticsDashboard.jsx): `frontend/src/components/AnalyticsDashboard.jsx`

### Event and scheduling

- [Airflow DAG](https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/dags/jobpulse_ingestion.py): `jobpulse/dags/jobpulse_ingestion.py`
- [Kafka producer](https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/messaging/producer.py): `jobpulse/messaging/producer.py`
- [Kafka consumer](https://github.com/dhanush-10-1/jobpulse/blob/main/jobpulse/messaging/consumer.py): `jobpulse/messaging/consumer.py`

### Other components

- **CLI operator**
- **Python.org jobs**
- **Gemini API**

## Connections

- CLI operator → Ingestion pipeline: runs
- Ingestion pipeline → HTML fetcher: fetches HTML (evidence: `jobpulse/ingestion/pipeline.py`)
- HTML fetcher → Python.org jobs: requests listings
- Ingestion pipeline → Listing parser: parses listings (evidence: `jobpulse/ingestion/pipeline.py`)
- Listing parser → Job model: creates jobs (evidence: `jobpulse/ingestion/parser.py`)
- Ingestion pipeline → PostgreSQL job store: upserts jobs (evidence: `jobpulse/ingestion/pipeline.py`)
- Ingestion pipeline → JSON exporter: exports jobs (evidence: `jobpulse/ingestion/pipeline.py`)
- Airflow DAG → Ingestion pipeline: orchestrates (evidence: `jobpulse/dags/jobpulse_ingestion.py`)
- Kafka consumer → PostgreSQL job store: upserts events (evidence: `jobpulse/messaging/consumer.py`)
- Explorer app → Search filters: renders (evidence: `frontend/src/App.jsx`)
- Explorer app → Job cards: renders (evidence: `frontend/src/App.jsx`)
- Explorer app → Job details: renders (evidence: `frontend/src/App.jsx`)
- Explorer app → Pagination: renders (evidence: `frontend/src/App.jsx`)
- Explorer app → Analytics dashboard: renders (evidence: `frontend/src/App.jsx`)
- Explorer app → FastAPI service: requests jobs
- FastAPI service → Job matching: dispatches (evidence: `api.py`)
- FastAPI service → Semantic search: dispatches (evidence: `api.py`)
- FastAPI service → PostgreSQL job store: queries jobs (evidence: `api.py`)
- PostgreSQL job store → DB connection: connects through (evidence: `jobpulse/database/repository.py`)
- Ingestion pipeline → Configuration: reads settings (evidence: `jobpulse/ingestion/pipeline.py`)
- Job embeddings → Gemini client: requests vectors (evidence: `jobpulse/ai/embeddings.py`)
- Job embeddings → PostgreSQL job store: saves vectors (evidence: `jobpulse/ai/embeddings.py`)
- Gemini client → Gemini API: calls provider (evidence: `jobpulse/ai/client.py`)
- Job enrichment → PostgreSQL job store: reads and saves (evidence: `jobpulse/ai/enrichment.py`)
- Job enrichment → Gemini client: uses client (evidence: `jobpulse/ai/enrichment.py`)
- Job matching → PostgreSQL job store: reads jobs (evidence: `jobpulse/ai/matching.py`)
- Semantic search → PostgreSQL job store: searches vectors (evidence: `jobpulse/ai/search.py`)
- Resume analysis → Gemini client: uses client (evidence: `jobpulse/ai/resume.py`)
- Resume analysis → Gemini API: requests analysis
- Recommendations → PostgreSQL job store: uses job data