# sudo-ali-dev/rankistan architecture

> Architecture diagram of the GitHub repository sudo-ali-dev/rankistan, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sudo-ali-dev/rankistan
- Repository: https://github.com/sudo-ali-dev/rankistan

## Overview

1. **High-level shape**  
   Rankistan is a three-part system: a static React/Vite frontend, a Node.js data-processing pipeline, and a Cloudflare Worker for private/server-side API calls. The frontend reads prebuilt leaderboard data from `public/data.json`; it does not talk directly to GitHub or Groq.

2. **Frontend app boundary**  
   `src/App.jsx` routes the UI into six main surfaces: Leaderboard, Map, Badge Generator, Register, Evolution, and About. The app is a client-side SPA built with React and styled with Tailwind/PostCSS; `src/main.jsx` mounts it, and the pages/components under `src/pages` and `src/components` provide the visible features.

3. **Leaderboard as a static data product**  
   The main user-facing dataset is `public/data.json`, which is generated by the pipeline and served as a static asset. The Leaderboard page consumes this file for ranking, filtering, search, pagination, CSV export, and tag display. This makes the UI fast and decoupled from live GitHub API usage.

4. **Data pipeline and batch refresh model**  
   The backend pipeline lives in `scripts/` and is orchestrated by `scripts/run-all.js`. Its core stages are discovery/fetch (`scripts/fetch-devs.js`), scoring (`scripts/score.js`), and output shaping (`scripts/write-leaderboard.js`). The pipeline is designed for incremental refresh, not full rebuilds.

5. **GitHub API as the source of truth**  
   `scripts/fetch-devs.js` discovers and refreshes developer records from GitHub using search and profile/activity data. The discovery step is split into 24 account-creation-date batches to stay under GitHub search limits and to spread API load over time. Each hourly run updates only one batch.

6. **Incremental leaderboard replacement**  
   Each record carries a `batch_index`. When a batch reruns, the pipeline removes only the old rows for that batch, inserts the newly fetched/scored rows, deduplicates by username, and re-sorts the list. This is why the leaderboard stays live throughout the day instead of being wiped and rebuilt once per day.

7. **Shared scoring logic between pipeline and Register**  
   Scoring is centralized in `score-config.json`, `src/utils/activity-score.js`, and `src/utils/score-breakdown.js`. The pipeline and the Register page use the same scoring math so the on-site estimate matches the batch ranking rules. Score inputs include public stars, recent meaningful activity, followers, and public repo count, with caps and age-based discounting.

8. **Activity filtering and trust boundaries**  
   Not every GitHub user qualifies. The pipeline applies contribution thresholds and inactivity limits before publishing a developer to the leaderboard. That filtering boundary matters architecturally: the public site only ever sees the curated output, not raw search results or unvetted accounts.

9. **Location normalization and the map subsystem**  
   The Developer Map uses deterministic location normalization from `src/utils/location.js` to group profiles into canonical Pakistani place keys. It combines token-aware country detection, city alias matching, and fallback handling so the map, list, and search all share one location model. Map rendering is purely frontend, but its buckets come from the normalized data.

10. **Tagging and classification flow**  
   During fetch, developers are assigned category tags using `src/utils/tag-matcher.js` and `src/utils/tag-keywords.json`. These tags flow into the final leaderboard JSON and are reused by the UI. The architecture treats tags as derived metadata, not primary source data.

11. **Cloudflare Worker for private API functionality**  
   `cloudflare/worker.js` exposes two routes: `/api/dev-summary` for on-demand AI summaries and `/api/badge/{username}` for Shields.io badge payloads. This keeps the Groq key off the frontend bundle and gives the static site dynamic capabilities without turning the main app into a server.

12. **Summary generation flow**  
   Developer summaries are generated only when a card is expanded. The frontend sends sanitized metadata to the Worker, the Worker enforces CORS and per-IP limits, calls Groq with secrets stored server-side, validates the response, and returns the summary. The frontend then caches the result in memory for repeat views.

13. **Badge generation flow**  
   The Badge Generator page produces snippets that point Shields.io at the Worker badge endpoint. Shields fetches the JSON badge payload from the Worker, which reads the current leaderboard and returns rank text. That means badge state is always tied to the latest published `data.json`.

14. **Deployment and scheduling boundaries**  
   The frontend is deployed as a static site, likely via GitHub Pages, while the Worker is deployed separately with Wrangler (`cloudflare/wrangler.toml`). Hourly updates are triggered by GitHub Actions workflows, with cron-job.org dispatching the update workflow. The scheduler maps UTC hour to the 24 batch indices.

15. **Auxiliary CI jobs**  
   There is also a digest-generation path in `scripts/generate-digest.js` and `.github/workflows/update-digest.yml`, indicating a separate automated content pipeline for periodic summaries. It sits beside the leaderboard pipeline rather than inside the main ranking flow.

Main entry points: [`src/App.jsx`](https://github.com/sudo-ali-dev/rankistan/blob/main/src/App.jsx) (App), [`scripts/fetch-devs.js`](https://github.com/sudo-ali-dev/rankistan/blob/main/scripts/fetch-devs.js) (Fetch Devs), [`scripts/score.js`](https://github.com/sudo-ali-dev/rankistan/blob/main/scripts/score.js) (Score), [`scripts/write-leaderboard.js`](https://github.com/sudo-ali-dev/rankistan/blob/main/scripts/write-leaderboard.js) (Write Leaderboard)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_frontend["Frontend SPA"]
  node_src_main(("Entry<br/>React mount<br/>[main.jsx]"))
  node_src_app["App<br/>SPA router<br/>[App.jsx]"]
  node_leaderboard_page["Leaderboard<br/>data UI<br/>[Leaderboard.jsx]"]
  node_map_page["Map<br/>location UI<br/>[DevMap.jsx]"]
  node_register_page["Register<br/>scoring UI<br/>[Register.jsx]"]
  node_badge_page["Badge Generator<br/>badge UI<br/>[BadgeGenerator.jsx]"]
  node_evolution_page["Evolution<br/>history UI<br/>[Evolution.jsx]"]
  node_about_page["About<br/>info UI<br/>[About.jsx]"]
  node_dev_card["Dev Card<br/>profile card<br/>[DevCard.jsx]"]
  node_score_panel["Score Panel<br/>breakdown UI"]
end

subgraph group_pipeline["Data Pipeline"]
  node_run_all["Run All<br/>orchestrator<br/>[run-all.js]"]
  node_fetch_devs["Fetch Devs<br/>GitHub ingest<br/>[fetch-devs.js]"]
  node_score_script["Score<br/>ranking engine<br/>[score.js]"]
  node_write_leaderboard["Write Leaderboard<br/>publisher"]
  node_generate_digest["Generate Digest<br/>digest job<br/>[generate-digest.js]"]
end

subgraph group_worker["Cloudflare Worker"]
  node_worker_api["Worker API<br/>Cloudflare Worker<br/>[worker.js]"]
end

subgraph group_shared["Shared Logic"]
  node_data_json[("Leaderboard Data<br/>static dataset<br/>[data.json]")]
  node_registered_devs[("Registered Devs<br/>static dataset")]
  node_activity_score["Activity Score<br/>shared scoring<br/>[activity-score.js]"]
  node_location_utils["Location Rules<br/>location normalization<br/>[location.js]"]
  node_tag_matcher["Tag Matcher<br/>classification<br/>[tag-matcher.js]"]
end

node_src_main -->|"mounts"| node_src_app
node_src_app -->|"routes"| node_leaderboard_page
node_src_app -->|"routes"| node_map_page
node_src_app -->|"routes"| node_register_page
node_src_app -->|"routes"| node_badge_page
node_src_app -->|"routes"| node_evolution_page
node_src_app -->|"routes"| node_about_page
node_leaderboard_page -->|"reads"| node_data_json
node_dev_card -->|"summaries"| node_worker_api
node_badge_page -->|"badge endpoint"| node_worker_api
node_register_page -->|"uses"| node_activity_score
node_score_panel -->|"explains"| node_activity_score
node_map_page -->|"groups by"| node_location_utils
node_run_all -->|"orchestrates"| node_fetch_devs
node_fetch_devs -->|"passes records"| node_score_script
node_score_script -->|"classifies"| node_tag_matcher
node_score_script -->|"scores"| node_activity_score
node_fetch_devs -->|"batch output"| node_write_leaderboard
node_write_leaderboard -->|"publishes"| node_data_json
node_write_leaderboard -->|"updates"| node_registered_devs
node_generate_digest -.->|"adjacent flow"| node_worker_api

click node_src_main "https://github.com/sudo-ali-dev/rankistan/blob/main/src/main.jsx"
click node_src_app "https://github.com/sudo-ali-dev/rankistan/blob/main/src/App.jsx"
click node_leaderboard_page "https://github.com/sudo-ali-dev/rankistan/blob/main/src/pages/Leaderboard.jsx"
click node_map_page "https://github.com/sudo-ali-dev/rankistan/blob/main/src/pages/DevMap.jsx"
click node_register_page "https://github.com/sudo-ali-dev/rankistan/blob/main/src/pages/Register.jsx"
click node_badge_page "https://github.com/sudo-ali-dev/rankistan/blob/main/src/pages/BadgeGenerator.jsx"
click node_evolution_page "https://github.com/sudo-ali-dev/rankistan/blob/main/src/pages/Evolution.jsx"
click node_about_page "https://github.com/sudo-ali-dev/rankistan/blob/main/src/pages/About.jsx"
click node_dev_card "https://github.com/sudo-ali-dev/rankistan/blob/main/src/components/DevCard.jsx"
click node_score_panel "https://github.com/sudo-ali-dev/rankistan/blob/main/src/components/ScoreBreakdownPanel.jsx"
click node_data_json "https://github.com/sudo-ali-dev/rankistan/blob/main/public/data.json"
click node_registered_devs "https://github.com/sudo-ali-dev/rankistan/blob/main/public/registered_devs.json"
click node_run_all "https://github.com/sudo-ali-dev/rankistan/blob/main/scripts/run-all.js"
click node_fetch_devs "https://github.com/sudo-ali-dev/rankistan/blob/main/scripts/fetch-devs.js"
click node_score_script "https://github.com/sudo-ali-dev/rankistan/blob/main/scripts/score.js"
click node_write_leaderboard "https://github.com/sudo-ali-dev/rankistan/blob/main/scripts/write-leaderboard.js"
click node_generate_digest "https://github.com/sudo-ali-dev/rankistan/blob/main/scripts/generate-digest.js"
click node_worker_api "https://github.com/sudo-ali-dev/rankistan/blob/main/cloudflare/worker.js"
click node_activity_score "https://github.com/sudo-ali-dev/rankistan/blob/main/src/utils/activity-score.js"
click node_location_utils "https://github.com/sudo-ali-dev/rankistan/blob/main/src/utils/location.js"
click node_tag_matcher "https://github.com/sudo-ali-dev/rankistan/blob/main/src/utils/tag-matcher.js"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_src_main,node_src_app,node_leaderboard_page,node_map_page,node_register_page,node_badge_page,node_evolution_page,node_about_page,node_dev_card,node_score_panel toneBlue
class node_run_all,node_fetch_devs,node_score_script,node_write_leaderboard,node_generate_digest toneAmber
class node_worker_api toneMint
class node_data_json,node_registered_devs,node_activity_score,node_location_utils,node_tag_matcher toneRose
```

## Components

### Frontend SPA

Static React/Vite client and its pages/components.

- [Entry](https://github.com/sudo-ali-dev/rankistan/blob/main/src/main.jsx): React mount · Bootstraps the SPA into the browser. · `src/main.jsx`
- [App](https://github.com/sudo-ali-dev/rankistan/blob/main/src/App.jsx): SPA router · Routes the client into the main product surfaces. · `src/App.jsx`
- [Leaderboard](https://github.com/sudo-ali-dev/rankistan/blob/main/src/pages/Leaderboard.jsx): data UI · Renders ranking, search, filtering, pagination, and export from static leaderboard data. · `src/pages/Leaderboard.jsx`
- [Map](https://github.com/sudo-ali-dev/rankistan/blob/main/src/pages/DevMap.jsx): location UI · Visualizes developers using normalized location buckets. · `src/pages/DevMap.jsx`
- [Register](https://github.com/sudo-ali-dev/rankistan/blob/main/src/pages/Register.jsx): scoring UI · Lets users estimate ranking using the shared scoring model. · `src/pages/Register.jsx`
- [Badge Generator](https://github.com/sudo-ali-dev/rankistan/blob/main/src/pages/BadgeGenerator.jsx): badge UI · Builds badge snippets that point to the worker badge endpoint. · `src/pages/BadgeGenerator.jsx`
- [Evolution](https://github.com/sudo-ali-dev/rankistan/blob/main/src/pages/Evolution.jsx): history UI · Shows leaderboard movement and change over time. · `src/pages/Evolution.jsx`
- [About](https://github.com/sudo-ali-dev/rankistan/blob/main/src/pages/About.jsx): info UI · Explains the project and its public-facing behavior. · `src/pages/About.jsx`
- [Dev Card](https://github.com/sudo-ali-dev/rankistan/blob/main/src/components/DevCard.jsx): profile card · Expands developer detail, score breakdown, and summary display. · `src/components/DevCard.jsx`
- [Score Panel](https://github.com/sudo-ali-dev/rankistan/blob/main/src/components/ScoreBreakdownPanel.jsx): breakdown UI · Displays score components and matching logic for the register and cards. · `src/components/ScoreBreakdownPanel.jsx`

### Data Pipeline

Batch GitHub discovery, scoring, and leaderboard generation.

- [Run All](https://github.com/sudo-ali-dev/rankistan/blob/main/scripts/run-all.js): orchestrator · Coordinates the batch pipeline end to end. · `scripts/run-all.js`
- [Fetch Devs](https://github.com/sudo-ali-dev/rankistan/blob/main/scripts/fetch-devs.js): GitHub ingest · Discovers and refreshes developer records from GitHub in incremental batches. · `scripts/fetch-devs.js`
- [Score](https://github.com/sudo-ali-dev/rankistan/blob/main/scripts/score.js): ranking engine · Computes ranking values from activity and profile signals. · `scripts/score.js`
- [Write Leaderboard](https://github.com/sudo-ali-dev/rankistan/blob/main/scripts/write-leaderboard.js): publisher · Merges batch results, deduplicates, sorts, and emits the static leaderboard JSON. · `scripts/write-leaderboard.js`
- [Generate Digest](https://github.com/sudo-ali-dev/rankistan/blob/main/scripts/generate-digest.js): digest job · Separate content-generation pipeline for periodic summaries. · `scripts/generate-digest.js`

### Cloudflare Worker

Private API layer for AI summaries and badge payloads.

- [Worker API](https://github.com/sudo-ali-dev/rankistan/blob/main/cloudflare/worker.js): Cloudflare Worker · Serves private summary and badge endpoints without exposing secrets to the frontend. · `cloudflare/worker.js`

### Shared Logic

Scoring, tagging, and location rules reused across systems.

- [Leaderboard Data](https://github.com/sudo-ali-dev/rankistan/blob/main/public/data.json): static dataset · Published developer ranking consumed by the frontend and badge flow. · `public/data.json`
- [Registered Devs](https://github.com/sudo-ali-dev/rankistan/blob/main/public/registered_devs.json): static dataset · Secondary published dataset used by the site. · `public/registered_devs.json`
- [Activity Score](https://github.com/sudo-ali-dev/rankistan/blob/main/src/utils/activity-score.js): shared scoring · Central scoring math reused by the pipeline and register page. · `src/utils/activity-score.js`
- [Location Rules](https://github.com/sudo-ali-dev/rankistan/blob/main/src/utils/location.js): location normalization · Canonicalizes profile locations into map buckets. · `src/utils/location.js`
- [Tag Matcher](https://github.com/sudo-ali-dev/rankistan/blob/main/src/utils/tag-matcher.js): classification · Derives category tags from developer metadata. · `src/utils/tag-matcher.js`

## Connections

- Entry → App: mounts · Browser entry mounts the application shell.
- App → Leaderboard: routes · Main UI surface for rankings and filters.
- App → Map: routes · Routes to the developer map surface.
- App → Register: routes · Routes to score estimation.
- App → Badge Generator: routes · Routes to badge snippet generation.
- App → Evolution: routes · Routes to evolution/history view.
- App → About: routes · Routes to project information.
- Leaderboard → Leaderboard Data: reads · Consumes the static ranking dataset.
- Dev Card → Worker API: summaries · Requests on-demand AI summaries when expanded.
- Badge Generator → Worker API: badge endpoint · Generates snippets targeting the worker badge payload.
- Register → Activity Score: uses · Shares the same scoring math as the backend.
- Score Panel → Activity Score: explains · Breaks down the shared ranking formula.
- Map → Location Rules: groups by · Uses normalized place keys for map buckets.
- Run All → Fetch Devs: orchestrates · Runs discovery as the first batch stage.
- Fetch Devs → Score: passes records · Fetched developer rows are scored before publishing.
- Score → Tag Matcher: classifies · Assigns derived tags during ranking preparation.
- Score → Activity Score: scores · Uses the shared scoring rules.
- Fetch Devs → Write Leaderboard: batch output · Updated batch records flow into publication.
- Write Leaderboard → Leaderboard Data: publishes · Emits the static leaderboard consumed by the site.
- Write Leaderboard → Registered Devs: updates · Keeps the auxiliary published dataset in sync.
- Generate Digest → Worker API: adjacent flow · Separate automation path beside the leaderboard pipeline.
