How sudo-ali-dev/rankistan works
1. High-level shape
Rankistan is a three-part system: a static React/Vite frontend, a Node.js data-processing pipeline, and a Cloudflare Worker for private/server-side API calls. The frontend reads prebuilt leaderboard data from public/data.json; it does not talk directly to GitHub or Groq.
2. Frontend app boundary
src/App.jsx routes the UI into six main surfaces: Leaderboard, Map, Badge Generator, Register, Evolution, and About. The app is a client-side SPA built with React and styled with Tailwind/PostCSS; src/main.jsx mounts it, and the pages/components under src/pages and src/components provide the visible features.
3. Leaderboard as a static data product
The main user-facing dataset is public/data.json, which is generated by the pipeline and served as a static asset. The Leaderboard page consumes this file for ranking, filtering, search, pagination, CSV export, and tag display. This makes the UI fast and decoupled from live GitHub API usage.
4. Data pipeline and batch refresh model
The backend pipeline lives in scripts/ and is orchestrated by scripts/run-all.js. Its core stages are discovery/fetch (scripts/fetch-devs.js), scoring (scripts/score.js), and output shaping (scripts/write-leaderboard.js). The pipeline is designed for incremental refresh, not full rebuilds.
5. GitHub API as the source of truth
scripts/fetch-devs.js discovers and refreshes developer records from GitHub using search and profile/activity data. The discovery step is split into 24 account-creation-date batches to stay under GitHub search limits and to spread API load over time. Each hourly run updates only one batch.
6. Incremental leaderboard replacement
Each record carries a batch_index. When a batch reruns, the pipeline removes only the old rows for that batch, inserts the newly fetched/scored rows, deduplicates by username, and re-sorts the list. This is why the leaderboard stays live throughout the day instead of being wiped and rebuilt once per day.
7. Shared scoring logic between pipeline and Register
Scoring is centralized in score-config.json, src/utils/activity-score.js, and src/utils/score-breakdown.js. The pipeline and the Register page use the same scoring math so the on-site estimate matches the batch ranking rules. Score inputs include public stars, recent meaningful activity, followers, and public repo count, with caps and age-based discounting.
8. Activity filtering and trust boundaries
Not every GitHub user qualifies. The pipeline applies contribution thresholds and inactivity limits before publishing a developer to the leaderboard. That filtering boundary matters architecturally: the public site only ever sees the curated output, not raw search results or unvetted accounts.
9. Location normalization and the map subsystem
The Developer Map uses deterministic location normalization from src/utils/location.js to group profiles into canonical Pakistani place keys. It combines token-aware country detection, city alias matching, and fallback handling so the map, list, and search all share one location model. Map rendering is purely frontend, but its buckets come from the normalized data.
10. Tagging and classification flow
During fetch, developers are assigned category tags using src/utils/tag-matcher.js and src/utils/tag-keywords.json. These tags flow into the final leaderboard JSON and are reused by the UI. The architecture treats tags as derived metadata, not primary source data.
11. Cloudflare Worker for private API functionality
cloudflare/worker.js exposes two routes: /api/dev-summary for on-demand AI summaries and /api/badge/{username} for Shields.io badge payloads. This keeps the Groq key off the frontend bundle and gives the static site dynamic capabilities without turning the main app into a server.
12. Summary generation flow
Developer summaries are generated only when a card is expanded. The frontend sends sanitized metadata to the Worker, the Worker enforces CORS and per-IP limits, calls Groq with secrets stored server-side, validates the response, and returns the summary. The frontend then caches the result in memory for repeat views.
13. Badge generation flow
The Badge Generator page produces snippets that point Shields.io at the Worker badge endpoint. Shields fetches the JSON badge payload from the Worker, which reads the current leaderboard and returns rank text. That means badge state is always tied to the latest published data.json.
14. Deployment and scheduling boundaries
The frontend is deployed as a static site, likely via GitHub Pages, while the Worker is deployed separately with Wrangler (cloudflare/wrangler.toml). Hourly updates are triggered by GitHub Actions workflows, with cron-job.org dispatching the update workflow. The scheduler maps UTC hour to the 24 batch indices.
15. Auxiliary CI jobs
There is also a digest-generation path in scripts/generate-digest.js and .github/workflows/update-digest.yml, indicating a separate automated content pipeline for periodic summaries. It sits beside the leaderboard pipeline rather than inside the main ranking flow.
Main entry points: App.jsx App · fetch-devs.js Fetch Devs · score.js Score · write-leaderboard.js Write Leaderboard
Components 21
Frontend SPA
Static React/Vite client and its pages/components.
- Entry · Bootstraps the SPA into the browser.
src/main.jsx - App · Routes the client into the main product surfaces.
src/App.jsx - Leaderboard · Renders ranking, search, filtering, pagination, and export from static leaderboard data.
src/pages/Leaderboard.jsx - Map · Visualizes developers using normalized location buckets.
src/pages/DevMap.jsx - Register · Lets users estimate ranking using the shared scoring model.
src/pages/Register.jsx - Badge Generator · Builds badge snippets that point to the worker badge endpoint.
src/pages/BadgeGenerator.jsx - Evolution · Shows leaderboard movement and change over time.
src/pages/Evolution.jsx - About · Explains the project and its public-facing behavior.
src/pages/About.jsx - Dev Card · Expands developer detail, score breakdown, and summary display.
src/components/DevCard.jsx - Score Panel · Displays score components and matching logic for the register and cards.
src/components/ScoreBreakdownPanel.jsx
Data Pipeline
Batch GitHub discovery, scoring, and leaderboard generation.
- Run All · Coordinates the batch pipeline end to end.
scripts/run-all.js - Fetch Devs · Discovers and refreshes developer records from GitHub in incremental batches.
scripts/fetch-devs.js - Score · Computes ranking values from activity and profile signals.
scripts/score.js - Write Leaderboard · Merges batch results, deduplicates, sorts, and emits the static leaderboard JSON.
scripts/write-leaderboard.js - Generate Digest · Separate content-generation pipeline for periodic summaries.
scripts/generate-digest.js
Cloudflare Worker
Private API layer for AI summaries and badge payloads.
- Worker API · Serves private summary and badge endpoints without exposing secrets to the frontend.
cloudflare/worker.js
Shared Logic
Scoring, tagging, and location rules reused across systems.
- Leaderboard Data · Published developer ranking consumed by the frontend and badge flow.
public/data.json - Registered Devs · Secondary published dataset used by the site.
public/registered_devs.json - Activity Score · Central scoring math reused by the pipeline and register page.
src/utils/activity-score.js - Location Rules · Canonicalizes profile locations into map buckets.
src/utils/location.js - Tag Matcher · Derives category tags from developer metadata.
src/utils/tag-matcher.js
Connections 21
- Entry → App: mounts · Browser entry mounts the application shell.
- App → Leaderboard: routes · Main UI surface for rankings and filters.
- App → Map: routes · Routes to the developer map surface.
- App → Register: routes · Routes to score estimation.
- App → Badge Generator: routes · Routes to badge snippet generation.
- App → Evolution: routes · Routes to evolution/history view.
- App → About: routes · Routes to project information.
- Leaderboard → Leaderboard Data: reads · Consumes the static ranking dataset.
- Dev Card → Worker API: summaries · Requests on-demand AI summaries when expanded.
- Badge Generator → Worker API: badge endpoint · Generates snippets targeting the worker badge payload.
- Register → Activity Score: uses · Shares the same scoring math as the backend.
- Score Panel → Activity Score: explains · Breaks down the shared ranking formula.
- Map → Location Rules: groups by · Uses normalized place keys for map buckets.
- Run All → Fetch Devs: orchestrates · Runs discovery as the first batch stage.
- Fetch Devs → Score: passes records · Fetched developer rows are scored before publishing.
- Score → Tag Matcher: classifies · Assigns derived tags during ranking preparation.
- Score → Activity Score: scores · Uses the shared scoring rules.
- Fetch Devs → Write Leaderboard: batch output · Updated batch records flow into publication.
- Write Leaderboard → Leaderboard Data: publishes · Emits the static leaderboard consumed by the site.
- Write Leaderboard → Registered Devs: updates · Keeps the auxiliary published dataset in sync.
- Generate Digest → Worker API: adjacent flow · Separate automation path beside the leaderboard pipeline.