How karpathy/jobs works
This repository builds a research tool for exploring BLS Occupational Outlook Handbook data. Its main workflow gathers occupation pages, turns them into structured statistics, adds LLM-estimated AI exposure, and publishes an interactive treemap with four selectable metrics. A second output packages the data for LLM-assisted analysis. The graph follows documented and excerpted data flows; the website’s internal implementation is unsampled, so its rendering details are not expanded.
Main entry points: occupations.json Occupation List · score.py Exposure Scorer · process.py Markdown Processor · make_prompt.py Prompt Builder
Components 19
Occupation Catalog
- Index Parser
parse_occupations.py - Occupation List
occupations.json
Data Collection
- Page Scraper
scrape.py - Raw Pages
html - Detail Parser
parse_detail.py - Markdown Processor
process.py - Occupation Markdown
Scoring and Analysis
- Stats Extractor
make_csv.py - Occupation Statistics
occupations.csv - Exposure Scorer
score.py - Exposure Scores
scores.json
Data Products
- Site Data Builder
build_site_data.py - Treemap Data
site/data.json - Prompt Builder
make_prompt.py - Analysis Prompt
prompt.md
Web Exploration
- Interactive Treemap
site/index.html
Other components
- Researcher
- BLS Handbook
- OpenRouter LLM
Connections 25
- Researcher → Interactive Treemap: explores
README.md - Index Parser → Occupation List: writes
parse_occupations.py - Occupation List → Page Scraper: lists targets
scrape.py - BLS Handbook → Page Scraper: serves pages
README.md - Page Scraper → Raw Pages: saves HTML
scrape.py - Raw Pages → Markdown Processor: supplies HTML
process.py - Markdown Processor → Detail Parser: calls parser
process.py - Detail Parser → Markdown Processor: returns Markdown
process.py - Markdown Processor → Occupation Markdown: writes Markdown
process.py - Raw Pages → Stats Extractor: supplies HTML
make_csv.py - Occupation List → Stats Extractor: provides metadata
make_csv.py - Stats Extractor → Occupation Statistics: writes statistics
make_csv.py - Occupation List → Exposure Scorer: lists occupations
score.py - Occupation Markdown → Exposure Scorer: supplies descriptions
score.py - Exposure Scorer → OpenRouter LLM: requests scores
score.py - OpenRouter LLM → Exposure Scorer: returns ratings
score.py - Exposure Scorer → Exposure Scores: saves results
score.py - Occupation Statistics → Site Data Builder: supplies statistics
build_site_data.py - Exposure Scores → Site Data Builder: supplies scores
build_site_data.py - Site Data Builder → Treemap Data: writes merged data
build_site_data.py - Treemap Data → Interactive Treemap: provides visualization data
README.md - Occupation List → Prompt Builder: provides occupations
make_prompt.py - Occupation Statistics → Prompt Builder: provides statistics
make_prompt.py - Exposure Scores → Prompt Builder: provides rationales
make_prompt.py - Prompt Builder → Analysis Prompt: writes prompt
make_prompt.py