How mckaywrigley/clarity-ai works
Project type and purpose
• This is a full‐stack Next.js application (React + serverless API routes) that mimics Perplexity.ai by scraping the web, building a prompt, and calling OpenAI to generate answers.
• The README makes clear it’s a “Perplexity.ai clone” that:
1. Accepts a user query
2. Scrapes Google results
3. Parses webpage text
4. Builds an LLM prompt
5. Calls OpenAI’s API
6. Streams the answer back to the user
File structure
• Top-level directories and files:
• pages/ – Next.js page components and API routes
– pages/index.tsx (UI)
– pages/api/answer.ts & pages/api/sources.ts (serverless endpoints)
• components/ – React UI pieces (Search, Answer)
• utils/ – business logic modules (scraping, prompt building)
• types/ – TypeScript interfaces
• public/, styles/, tailwind.config.js – static assets & styling
• next.config.js, tsconfig.json, PostCSS, ESLint – framework & build configs
• There is no persistent database; all data flows through in-memory scraping and OpenAI calls.
README for additional insights
• Describes the end-to-end flow (scrape → parse → prompt → OpenAI).
a. Main components:
• Client (Browser UI)
• Next.js Frontend (React + Tailwind in pages/index.tsx & components/)
• Next.js API Server (Serverless functions in pages/api/)
• Scraper Module (utils/sources.ts)
• Parser Module (utils/answer.ts)
• Prompt Builder & LLM Gateway (utils/answer.ts calling OpenAI)
• External Services: Google Search (scraping), OpenAI API
• Hosting/Infra: Vercel (Next.js runtime)
b. Relationships & interactions:
1. User enters query in the Search component →
2. Frontend calls /api/sources to retrieve links (Scraper Module calls Google via HTTP, parses HTML) →
3. Frontend displays sources (optional) →
4. User requests an answer → Frontend calls /api/answer →
5. API handler orchestrates: calls Scraper → Parser → Prompt Builder → OpenAI API → streams tokens back →
6. Frontend renders incremental Answer component updates
c. Architectural patterns and design principles:
• Serverless functions for backend logic (no monolithic server)
• Separation of concerns: UI (components) vs. data-fetching (pages/api) vs. business logic (utils/)
• Streaming response pattern for LLM output
• Utility modules abstracting scraping/parsing to stay framework-agnostic
d. Relevant technologies/frameworks:
• Next.js (SSR/SSG + API routes)
• React + Tailwind CSS (UI styling)
• Node.js + Cheerio or similar for web scraping
• OpenAI API (text-davinci-003 or streaming models)
• TypeScript for type safety