How ourongxing/newsnow works
Project type and purpose
• By examining the file tree and README, we see that NewsNow is a full-stack, SSR/PWA news‐aggregator web application.
• The “src” folder holds a React/Vite frontend (TSX, atoms/hooks/components), the “server” folder holds serverless API endpoints (Nitro config, endpoints under server/api, scrapers under server/sources), and “shared” holds common types/constants used by both.
• Key features: real-time scraping of multiple news sources, GitHub OAuth login, user data sync, caching (default 30m), PWA support, adaptive scraping intervals.
File structure
• Top-level:
• Dockerfile, docker-compose.yml – for containerized deployment.
• nitro.config.ts, vite.config.ts, pwa.config.ts – builder configs (Nitro for SSR, Vite for bundling, PWA setup).
• server/api – entry points for HTTP APIs (login, sync, news queries).
• server/sources – individual scraper modules for each external source.
• server/database – caching and user DB logic.
• shared – shared types, constants, metadata used by both front and back.
• src – React app: components, hooks, routes, styling.
• Patterns:
• Monorepo with code-sharing
• Serverless function pattern (each file in server/api becomes a route)
• PWA offline capability (public/sw.js)
• Middleware for auth (server/middleware/auth.ts)
README for additional insights
• Describes SSR + static build for Cloudflare Pages or Vercel, with output dist/output/public
• Guides on GitHub OAuth flow (API entrypoints, callback URL, JWT)
• Advises Cloudflare D1 for persistent storage
a. Main Components
1) Client (Browser/PWA)
3) SSR/Serverless Layer (Nitro engine)
4) API Routes (server/api/*)
5) Scraper Modules (server/sources/*)
6) Database Layer (Cloudflare D1 or other connectors via server/database/*.ts)
7) External Services:
• GitHub OAuth
• External news websites (36kr, HackerNews, Zhihu, etc.)
b. Relationships & Data Flow
• Client requests page → CDN/static server → index.html + bundled JS
• Client (or service worker) → API endpoints (/api/latest, /api/s, /api/me/sync)
• API endpoint → Middleware auth (JWT verification) → Database (user cache)
• API endpoint → Scraper module → External news site → returns raw data → server/utils (normalize → RSS/JSON)
• Normalized data cached in Database → returned to client
• On login: Client → /api/oauth/github → GitHub OAuth → callback → issue JWT → store in cookie/localStorage
c. Architectural Patterns & Design Principles
• Serverless functions (one route per file)
• Monorepo with shared code (avoid duplication)
• Atomic React state via Jotai atoms (src/atoms)
• PWA strategy: service worker caching + overlay-scrollbar + offline reading
• Adaptive scraping: central “source” util orchestrates intervals to avoid IP bans
d. Key Technologies & Frameworks
• Frontend: React, Vite, Typescript, Jotai, Tailwind/UnoCSS (uno.config.ts)
• Server: Nitro, Node.js, Typescript, Cloudflare Workers/D1 adapter
• Deployment: Docker Compose, Cloudflare Pages/Workers, Vercel
• CI/CD: GitHub Actions (auto-refresh, docker build, release)
Elements
• Data flows: “Cache lookup,” “Fetch & normalize,” “Issue JWT.”
Final Advice
• PWA aspects explicitly (service worker, offline cache).