# karpathy/jobs architecture

> Architecture diagram of the GitHub repository karpathy/jobs, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/karpathy/jobs
- Repository: https://github.com/karpathy/jobs

## Overview

This repository builds a research tool for exploring BLS Occupational Outlook Handbook data. Its main workflow gathers occupation pages, turns them into structured statistics, adds LLM-estimated AI exposure, and publishes an interactive treemap with four selectable metrics. A second output packages the data for LLM-assisted analysis. The graph follows documented and excerpted data flows; the website’s internal implementation is unsampled, so its rendering details are not expanded.

Main entry points: [`occupations.json`](https://github.com/karpathy/jobs/blob/master/occupations.json) (Occupation List), [`score.py`](https://github.com/karpathy/jobs/blob/master/score.py) (Exposure Scorer), [`process.py`](https://github.com/karpathy/jobs/blob/master/process.py) (Markdown Processor), [`make_prompt.py`](https://github.com/karpathy/jobs/blob/master/make_prompt.py) (Prompt Builder)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_catalog["Occupation Catalog"]
  node_index_parser["Index Parser"]
  node_occupation_list["Occupation List<br/>[occupations.json]"]
end

subgraph group_collection["Data Collection"]
  node_scraper["Page Scraper<br/>[scrape.py]"]
  node_raw_pages["Raw Pages"]
  node_detail_parser["Detail Parser<br/>[parse_detail.py]"]
  node_markdown_processor["Markdown Processor<br/>[process.py]"]
  node_occupation_pages["Occupation Markdown"]
end

subgraph group_analysis["Scoring and Analysis"]
  node_stats_extractor["Stats Extractor<br/>[make_csv.py]"]
  node_occupation_stats["Occupation Statistics<br/>[occupations.csv]"]
  node_exposure_scorer["Exposure Scorer<br/>[score.py]"]
  node_exposure_scores["Exposure Scores<br/>[scores.json]"]
end

subgraph group_products["Data Products"]
  node_site_data_builder["Site Data Builder<br/>[build_site_data.py]"]
  node_site_data["Treemap Data<br/>[data.json]"]
  node_prompt_builder["Prompt Builder<br/>[make_prompt.py]"]
  node_analysis_prompt["Analysis Prompt<br/>[prompt.md]"]
end

subgraph group_exploration["Web Exploration"]
  node_treemap["Interactive Treemap<br/>[index.html]"]
end

node_researcher(("Researcher"))
node_bls["BLS Handbook"]
node_openrouter["OpenRouter LLM"]

node_researcher -->|"explores"| node_treemap
node_index_parser -->|"writes"| node_occupation_list
node_occupation_list -->|"lists targets"| node_scraper
node_bls -->|"serves pages"| node_scraper
node_scraper -->|"saves HTML"| node_raw_pages
node_raw_pages -->|"supplies HTML"| node_markdown_processor
node_markdown_processor -->|"calls parser"| node_detail_parser
node_detail_parser -->|"returns Markdown"| node_markdown_processor
node_markdown_processor -->|"writes Markdown"| node_occupation_pages
node_raw_pages -->|"supplies HTML"| node_stats_extractor
node_occupation_list -->|"provides metadata"| node_stats_extractor
node_stats_extractor -->|"writes statistics"| node_occupation_stats
node_occupation_list -->|"lists occupations"| node_exposure_scorer
node_occupation_pages -->|"supplies descriptions"| node_exposure_scorer
node_exposure_scorer -->|"requests scores"| node_openrouter
node_openrouter -->|"returns ratings"| node_exposure_scorer
node_exposure_scorer -->|"saves results"| node_exposure_scores
node_occupation_stats -->|"supplies statistics"| node_site_data_builder
node_exposure_scores -->|"supplies scores"| node_site_data_builder
node_site_data_builder -->|"writes merged data"| node_site_data
node_site_data -->|"provides visualization data"| node_treemap
node_occupation_list -->|"provides occupations"| node_prompt_builder
node_occupation_stats -->|"provides statistics"| node_prompt_builder
node_exposure_scores -->|"provides rationales"| node_prompt_builder
node_prompt_builder -->|"writes prompt"| node_analysis_prompt

click node_index_parser "https://github.com/karpathy/jobs/blob/master/parse_occupations.py"
click node_occupation_list "https://github.com/karpathy/jobs/blob/master/occupations.json"
click node_scraper "https://github.com/karpathy/jobs/blob/master/scrape.py"
click node_raw_pages "https://github.com/karpathy/jobs/tree/master/html"
click node_detail_parser "https://github.com/karpathy/jobs/blob/master/parse_detail.py"
click node_markdown_processor "https://github.com/karpathy/jobs/blob/master/process.py"
click node_stats_extractor "https://github.com/karpathy/jobs/blob/master/make_csv.py"
click node_occupation_stats "https://github.com/karpathy/jobs/blob/master/occupations.csv"
click node_exposure_scorer "https://github.com/karpathy/jobs/blob/master/score.py"
click node_exposure_scores "https://github.com/karpathy/jobs/blob/master/scores.json"
click node_site_data_builder "https://github.com/karpathy/jobs/blob/master/build_site_data.py"
click node_site_data "https://github.com/karpathy/jobs/blob/master/site/data.json"
click node_prompt_builder "https://github.com/karpathy/jobs/blob/master/make_prompt.py"
click node_analysis_prompt "https://github.com/karpathy/jobs/blob/master/prompt.md"
click node_treemap "https://github.com/karpathy/jobs/blob/master/site/index.html"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_index_parser,node_occupation_list toneBlue
class node_scraper,node_raw_pages,node_detail_parser,node_markdown_processor,node_occupation_pages toneAmber
class node_stats_extractor,node_occupation_stats,node_exposure_scorer,node_exposure_scores,node_openrouter toneMint
class node_site_data_builder,node_site_data,node_prompt_builder,node_analysis_prompt toneRose
class node_treemap,node_researcher,node_bls toneIndigo
```

## Components

### Occupation Catalog

- [Index Parser](https://github.com/karpathy/jobs/blob/master/parse_occupations.py): `parse_occupations.py`
- [Occupation List](https://github.com/karpathy/jobs/blob/master/occupations.json): `occupations.json`

### Data Collection

- [Page Scraper](https://github.com/karpathy/jobs/blob/master/scrape.py): `scrape.py`
- [Raw Pages](https://github.com/karpathy/jobs/tree/master/html): `html`
- [Detail Parser](https://github.com/karpathy/jobs/blob/master/parse_detail.py): `parse_detail.py`
- [Markdown Processor](https://github.com/karpathy/jobs/blob/master/process.py): `process.py`
- **Occupation Markdown**

### Scoring and Analysis

- [Stats Extractor](https://github.com/karpathy/jobs/blob/master/make_csv.py): `make_csv.py`
- [Occupation Statistics](https://github.com/karpathy/jobs/blob/master/occupations.csv): `occupations.csv`
- [Exposure Scorer](https://github.com/karpathy/jobs/blob/master/score.py): `score.py`
- [Exposure Scores](https://github.com/karpathy/jobs/blob/master/scores.json): `scores.json`

### Data Products

- [Site Data Builder](https://github.com/karpathy/jobs/blob/master/build_site_data.py): `build_site_data.py`
- [Treemap Data](https://github.com/karpathy/jobs/blob/master/site/data.json): `site/data.json`
- [Prompt Builder](https://github.com/karpathy/jobs/blob/master/make_prompt.py): `make_prompt.py`
- [Analysis Prompt](https://github.com/karpathy/jobs/blob/master/prompt.md): `prompt.md`

### Web Exploration

- [Interactive Treemap](https://github.com/karpathy/jobs/blob/master/site/index.html): `site/index.html`

### Other components

- **Researcher**
- **BLS Handbook**
- **OpenRouter LLM**

## Connections

- Researcher → Interactive Treemap: explores (evidence: `README.md`)
- Index Parser → Occupation List: writes (evidence: `parse_occupations.py`)
- Occupation List → Page Scraper: lists targets (evidence: `scrape.py`)
- BLS Handbook → Page Scraper: serves pages (evidence: `README.md`)
- Page Scraper → Raw Pages: saves HTML (evidence: `scrape.py`)
- Raw Pages → Markdown Processor: supplies HTML (evidence: `process.py`)
- Markdown Processor → Detail Parser: calls parser (evidence: `process.py`)
- Detail Parser → Markdown Processor: returns Markdown (evidence: `process.py`)
- Markdown Processor → Occupation Markdown: writes Markdown (evidence: `process.py`)
- Raw Pages → Stats Extractor: supplies HTML (evidence: `make_csv.py`)
- Occupation List → Stats Extractor: provides metadata (evidence: `make_csv.py`)
- Stats Extractor → Occupation Statistics: writes statistics (evidence: `make_csv.py`)
- Occupation List → Exposure Scorer: lists occupations (evidence: `score.py`)
- Occupation Markdown → Exposure Scorer: supplies descriptions (evidence: `score.py`)
- Exposure Scorer → OpenRouter LLM: requests scores (evidence: `score.py`)
- OpenRouter LLM → Exposure Scorer: returns ratings (evidence: `score.py`)
- Exposure Scorer → Exposure Scores: saves results (evidence: `score.py`)
- Occupation Statistics → Site Data Builder: supplies statistics (evidence: `build_site_data.py`)
- Exposure Scores → Site Data Builder: supplies scores (evidence: `build_site_data.py`)
- Site Data Builder → Treemap Data: writes merged data (evidence: `build_site_data.py`)
- Treemap Data → Interactive Treemap: provides visualization data (evidence: `README.md`)
- Occupation List → Prompt Builder: provides occupations (evidence: `make_prompt.py`)
- Occupation Statistics → Prompt Builder: provides statistics (evidence: `make_prompt.py`)
- Exposure Scores → Prompt Builder: provides rationales (evidence: `make_prompt.py`)
- Prompt Builder → Analysis Prompt: writes prompt (evidence: `make_prompt.py`)
