# divyamunderconfident/unmet architecture

> Architecture diagram of the GitHub repository divyamunderconfident/unmet, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/divyamunderconfident/unmet
- Repository: https://github.com/divyamunderconfident/unmet

## Overview

Unmet turns public forum posts into a daily ranked index of unmet needs, then presents that index as a static website. The documented workflow covers source retrieval, model classification, embedding-based grouping, ranking and naming, and publication; the site also provides problem details, methodology and attribution, and RSS. The pipeline entry point and sampled calls establish the main control flow. The detailed problem-page implementation and some site data access code were not excerpted, so their internal links are limited to documented or indexed evidence.

Main entry points: [`unmet/__main__.py`](https://github.com/divyamunderconfident/unmet/blob/main/unmet/__main__.py) (Pipeline orchestration), [`site/lib/data.ts`](https://github.com/divyamunderconfident/unmet/blob/main/site/lib/data.ts) (Site data access), [`site/app/layout.tsx`](https://github.com/divyamunderconfident/unmet/blob/main/site/app/layout.tsx) (Shared site layout), [`unmet/llm.py`](https://github.com/divyamunderconfident/unmet/blob/main/unmet/llm.py) (Model gateway)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_collection["Post collection"]
  node_fetch["Post fetching<br/>[fetch.py]"]
end

subgraph group_analysis["Need analysis"]
  node_classify["Need classification<br/>[classify.py]"]
  node_llm["Model gateway<br/>[llm.py]"]
  node_config["Pipeline settings<br/>[config.py]"]
  node_cluster["Need clustering<br/>[cluster.py]"]
  node_rank["Ranking and naming<br/>[rank.py]"]
end

subgraph group_publication["Index publication"]
  node_classified[("Classification progress<br/>[classified.json]")]
  node_needs[("Accepted needs<br/>[needs.json]")]
  node_runner["Pipeline orchestration<br/>[__main__.py]"]
  node_index[("Published problems index<br/>[problems.json]")]
end

subgraph group_web["Public site"]
  node_dataaccess["Site data access<br/>[data.ts]"]
  node_home["Ranked home page<br/>[page.tsx]"]
  node_details["Problem detail pages<br/>[page.tsx]"]
  node_method["Method and attribution<br/>[page.tsx]"]
  node_feed["RSS feed<br/>[route.ts]"]
  node_layout["Shared site layout<br/>[layout.tsx]"]
end

node_operator(("Pipeline operator"))
node_hn(("Hacker News"))
node_se(("Stack Exchange"))
node_model["Chat model providers"]
node_embeddings["Local embeddings"]

node_operator -->|"runs"| node_runner
node_runner -->|"fetches posts"| node_fetch
node_hn -->|"supplies posts"| node_fetch
node_se -->|"supplies posts"| node_fetch
node_runner -->|"classifies posts"| node_classify
node_classify -->|"requests completions"| node_llm
node_llm -->|"calls provider"| node_model
node_runner -->|"saves progress"| node_classified
node_runner -->|"saves needs"| node_needs
node_runner -->|"groups needs"| node_cluster
node_cluster -->|"embeds problems"| node_embeddings
node_runner -->|"ranks groups"| node_rank
node_rank -->|"names top groups"| node_llm
node_runner -->|"writes index"| node_index
node_index -->|"reads index"| node_dataaccess
node_home -->|"loads data"| node_dataaccess
node_details -.->|"reads problem"| node_dataaccess
node_method -->|"loads edition data"| node_dataaccess
node_feed -->|"loads feed data"| node_dataaccess
node_layout -->|"wraps page"| node_home
node_layout -.->|"wraps page"| node_method
node_layout -.->|"wraps page"| node_details

click node_fetch "https://github.com/divyamunderconfident/unmet/blob/main/unmet/fetch.py"
click node_classify "https://github.com/divyamunderconfident/unmet/blob/main/unmet/classify.py"
click node_llm "https://github.com/divyamunderconfident/unmet/blob/main/unmet/llm.py"
click node_config "https://github.com/divyamunderconfident/unmet/blob/main/unmet/config.py"
click node_classified "https://github.com/divyamunderconfident/unmet/blob/main/data/classified.json"
click node_needs "https://github.com/divyamunderconfident/unmet/blob/main/data/needs.json"
click node_cluster "https://github.com/divyamunderconfident/unmet/blob/main/unmet/cluster.py"
click node_rank "https://github.com/divyamunderconfident/unmet/blob/main/unmet/rank.py"
click node_runner "https://github.com/divyamunderconfident/unmet/blob/main/unmet/__main__.py"
click node_index "https://github.com/divyamunderconfident/unmet/blob/main/site/public/data/problems.json"
click node_dataaccess "https://github.com/divyamunderconfident/unmet/blob/main/site/lib/data.ts"
click node_home "https://github.com/divyamunderconfident/unmet/blob/main/site/app/page.tsx"
click node_details "https://github.com/divyamunderconfident/unmet/blob/main/site/app/problem/%5Bid%5D/page.tsx"
click node_method "https://github.com/divyamunderconfident/unmet/blob/main/site/app/method/page.tsx"
click node_feed "https://github.com/divyamunderconfident/unmet/blob/main/site/app/feed.xml/route.ts"
click node_layout "https://github.com/divyamunderconfident/unmet/blob/main/site/app/layout.tsx"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_fetch toneBlue
class node_classify,node_llm,node_config,node_cluster,node_rank toneAmber
class node_classified,node_needs,node_runner,node_index toneMint
class node_dataaccess,node_home,node_details,node_method,node_feed,node_layout toneRose
class node_operator,node_hn,node_se,node_model,node_embeddings toneIndigo
```

## Components

### Post collection

- [Post fetching](https://github.com/divyamunderconfident/unmet/blob/main/unmet/fetch.py): `unmet/fetch.py`

### Need analysis

- [Need classification](https://github.com/divyamunderconfident/unmet/blob/main/unmet/classify.py): `unmet/classify.py`
- [Model gateway](https://github.com/divyamunderconfident/unmet/blob/main/unmet/llm.py): `unmet/llm.py`
- [Pipeline settings](https://github.com/divyamunderconfident/unmet/blob/main/unmet/config.py): `unmet/config.py`
- [Need clustering](https://github.com/divyamunderconfident/unmet/blob/main/unmet/cluster.py): `unmet/cluster.py`
- [Ranking and naming](https://github.com/divyamunderconfident/unmet/blob/main/unmet/rank.py): `unmet/rank.py`

### Index publication

- [Classification progress](https://github.com/divyamunderconfident/unmet/blob/main/data/classified.json): `data/classified.json`
- [Accepted needs](https://github.com/divyamunderconfident/unmet/blob/main/data/needs.json): `data/needs.json`
- [Pipeline orchestration](https://github.com/divyamunderconfident/unmet/blob/main/unmet/__main__.py): `unmet/__main__.py`
- [Published problems index](https://github.com/divyamunderconfident/unmet/blob/main/site/public/data/problems.json): `site/public/data/problems.json`

### Public site

- [Site data access](https://github.com/divyamunderconfident/unmet/blob/main/site/lib/data.ts): `site/lib/data.ts`
- [Ranked home page](https://github.com/divyamunderconfident/unmet/blob/main/site/app/page.tsx): `site/app/page.tsx`
- [Problem detail pages](https://github.com/divyamunderconfident/unmet/blob/main/site/app/problem/%5Bid%5D/page.tsx): `site/app/problem/[id]/page.tsx`
- [Method and attribution](https://github.com/divyamunderconfident/unmet/blob/main/site/app/method/page.tsx): `site/app/method/page.tsx`
- [RSS feed](https://github.com/divyamunderconfident/unmet/blob/main/site/app/feed.xml/route.ts): `site/app/feed.xml/route.ts`
- [Shared site layout](https://github.com/divyamunderconfident/unmet/blob/main/site/app/layout.tsx): `site/app/layout.tsx`

### Other components

- **Pipeline operator**
- **Hacker News**
- **Stack Exchange**
- **Chat model providers**
- **Local embeddings**

## Connections

- Pipeline operator → Pipeline orchestration: runs (evidence: `README.md`)
- Pipeline orchestration → Post fetching: fetches posts (evidence: `unmet/__main__.py`)
- Hacker News → Post fetching: supplies posts (evidence: `unmet/fetch.py`)
- Stack Exchange → Post fetching: supplies posts (evidence: `unmet/fetch.py`)
- Pipeline orchestration → Need classification: classifies posts (evidence: `unmet/__main__.py`)
- Need classification → Model gateway: requests completions (evidence: `unmet/classify.py`)
- Model gateway → Chat model providers: calls provider (evidence: `unmet/llm.py`)
- Pipeline orchestration → Classification progress: saves progress (evidence: `unmet/__main__.py`)
- Pipeline orchestration → Accepted needs: saves needs (evidence: `unmet/__main__.py`)
- Pipeline orchestration → Need clustering: groups needs (evidence: `unmet/__main__.py`)
- Need clustering → Local embeddings: embeds problems (evidence: `README.md`)
- Pipeline orchestration → Ranking and naming: ranks groups (evidence: `unmet/__main__.py`)
- Ranking and naming → Model gateway: names top groups (evidence: `unmet/rank.py`)
- Pipeline orchestration → Published problems index: writes index (evidence: `unmet/__main__.py`)
- Published problems index → Site data access: reads index (evidence: `README.md`)
- Ranked home page → Site data access: loads data (evidence: `site/app/page.tsx`)
- Problem detail pages → Site data access: reads problem
- Method and attribution → Site data access: loads edition data (evidence: `site/app/method/page.tsx`)
- RSS feed → Site data access: loads feed data (evidence: `site/app/feed.xml/route.ts`)
- Shared site layout → Ranked home page: wraps page (evidence: `site/app/layout.tsx`)
- Shared site layout → Method and attribution: wraps page
- Shared site layout → Problem detail pages: wraps page
