How divyamunderconfident/unmet works
Unmet turns public forum posts into a daily ranked index of unmet needs, then presents that index as a static website. The documented workflow covers source retrieval, model classification, embedding-based grouping, ranking and naming, and publication; the site also provides problem details, methodology and attribution, and RSS. The pipeline entry point and sampled calls establish the main control flow. The detailed problem-page implementation and some site data access code were not excerpted, so their internal links are limited to documented or indexed evidence.
Main entry points: __main__.py Pipeline orchestration · data.ts Site data access · layout.tsx Shared site layout · llm.py Model gateway
Components 21
Post collection
- Post fetching
unmet/fetch.py
Need analysis
- Need classification
unmet/classify.py - Model gateway
unmet/llm.py - Pipeline settings
unmet/config.py - Need clustering
unmet/cluster.py - Ranking and naming
unmet/rank.py
Index publication
- Classification progress
data/classified.json - Accepted needs
data/needs.json - Pipeline orchestration
unmet/__main__.py - Published problems index
site/public/data/problems.json
Public site
- Site data access
site/lib/data.ts - Ranked home page
site/app/page.tsx - Problem detail pages
site/app/problem/[id]/page.tsx - Method and attribution
site/app/method/page.tsx - RSS feed
site/app/feed.xml/route.ts - Shared site layout
site/app/layout.tsx
Other components
- Pipeline operator
- Hacker News
- Stack Exchange
- Chat model providers
- Local embeddings
Connections 22
- Pipeline operator → Pipeline orchestration: runs
README.md - Pipeline orchestration → Post fetching: fetches posts
unmet/__main__.py - Hacker News → Post fetching: supplies posts
unmet/fetch.py - Stack Exchange → Post fetching: supplies posts
unmet/fetch.py - Pipeline orchestration → Need classification: classifies posts
unmet/__main__.py - Need classification → Model gateway: requests completions
unmet/classify.py - Model gateway → Chat model providers: calls provider
unmet/llm.py - Pipeline orchestration → Classification progress: saves progress
unmet/__main__.py - Pipeline orchestration → Accepted needs: saves needs
unmet/__main__.py - Pipeline orchestration → Need clustering: groups needs
unmet/__main__.py - Need clustering → Local embeddings: embeds problems
README.md - Pipeline orchestration → Ranking and naming: ranks groups
unmet/__main__.py - Ranking and naming → Model gateway: names top groups
unmet/rank.py - Pipeline orchestration → Published problems index: writes index
unmet/__main__.py - Published problems index → Site data access: reads index
README.md - Ranked home page → Site data access: loads data
site/app/page.tsx - Problem detail pages → Site data access: reads problem
- Method and attribution → Site data access: loads edition data
site/app/method/page.tsx - RSS feed → Site data access: loads feed data
site/app/feed.xml/route.ts - Shared site layout → Ranked home page: wraps page
site/app/layout.tsx - Shared site layout → Method and attribution: wraps page
- Shared site layout → Problem detail pages: wraps page