# mckaywrigley/naval-gpt architecture

> Architecture diagram of the GitHub repository mckaywrigley/naval-gpt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mckaywrigley/naval-gpt
- Repository: https://github.com/mckaywrigley/naval-gpt

## Overview

##### Project type and purpose
- This is a full-stack Next.js application (“Naval GPT”) that provides AI-powered search, chat, and audio playback around Naval Ravikant’s “How To Get Rich” content.
- It includes both a React front-end (pages, components, styles) and a Node.js/TypeScript back-end via Next.js API routes. It also bundles an ETL pipeline (scraping, embedding text/audio) implemented in Node.js and Python.

##### File structure
- Top-level directories:
- components (UI widgets)
- pages (Next.js front-end + API routes under pages/api)
- public (static assets, including podcast.mp3)
- scripts (data-processing: scraping, embedding, Python Whisper pipeline)
- utils, types (shared helper functions & type definitions)
- Key configuration: next.config.js, tsconfig.json, tailwind.config.js, schema.sql (DB schema).
- Deployment hints: .env.local for OpenAI & Supabase, Supabase hosting, SQL editor for schema.

##### README for additional insights
- Architecture: ETL → vector DB (Postgres + pgvector on Supabase) → Next.js API → React UI.
- External dependencies: Naval’s blog HTML, podcast MP3, OpenAI Embeddings & ChatCompletion APIs, OpenAI Whisper for audio.

a. Main components
- Data Ingestion & ETL pipeline
– Scrape script (scripts/scrape.ts → JSON)
– Text embed script (scripts/embed-text.ts → OpenAI embeddings → Supabase)
– Audio processing pipeline (scripts/main.py using Whisper → JSON)
– Audio embed script (scripts/embed-audio.ts → OpenAI embeddings → Supabase)
- Database & Storage
– Supabase Postgres with pgvector extension (stores “passages” and “clips” tables)
– schema.sql defines tables, vectors, RLS policies
– public/static storage for podcast.mp3
- Next.js Application
– Front-end React (pages/index.tsx, components/*)
– API routes (pages/api/search.ts, pages/api/answer.ts)
– Utility modules (utils/index.ts, types/index.ts)
- External Services
– OpenAI API (text-embedding-ada-002, GPT-3.5-turbo, Whisper)
– Naval’s website & podcast host

b. Relationships & data flow
1. ETL: scrape → embed → save to DB
2. User interacts with React UI → calls /api/search or /api/answer
3. API route invokes:
- Generate OpenAI embedding for query → compare cosine similarity in pgvector → return top passages/clips
- For chat: compile prompt with passages → call GPT-3.5-turbo → return response
4. React UI displays search results, chat thread, and controls audio player (seeks to clip timestamp).

c. Architectural patterns & principles
- Clear separation of concerns: ETL pipeline vs. runtime API vs. UI
- Serverless micro-services (Next.js API routes) for search & chat endpoints
- Vector search pattern using pgvector and cosine similarity
- Modular UI components (Answer, Navbar, Player)

d. Key technologies
- Next.js + React + TypeScript + Tailwind CSS
- Supabase (Postgres + pgvector)
- OpenAI Embeddings & ChatCompletion API
- Python Whisper for audio transcription & embedding
- Node.js scripts for scraping & embedding

##### Elements
- Labels: “Scraper”, “Text Embedder”, “Audio Processor”, “Supabase DB”, “Next.js API”, “React UI”, “OpenAI API”, “Podcast Source”
- ETL → DB
- UI → API → DB & OpenAI → API → UI
- Audio pipeline → JSON → DB
- API services (blue rectangles)
- UI components (orange rectangles)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Services
    subgraph "External Services" 
        direction TB
        OpenAI["OpenAI API"]:::external
        PodcastSrc["Naval’s Blog & Podcast Source"]:::external
    end

    %% Data ETL Layer
    subgraph "Data Ingestion & ETL Pipeline" 
        direction TB
        Scraper["Scraper"]:::etl
        TextEmbedder["Text Embedder"]:::etl
        AudioProcessor["Audio Processor (Whisper pipeline)"]:::etl
        AudioEmbedder["Audio Embedder"]:::etl
    end

    %% Database Layer
    subgraph "Database & Storage" 
        direction TB
        SupabaseDB[(Supabase Postgres + pgvector)]:::db
        Schema["DB Schema"]:::db
    end

    %% API Layer
    subgraph "Next.js API (Serverless)" 
        direction TB
        SearchAPI["/api/search"]:::api
        AnswerAPI["/api/answer"]:::api
        Utils["Shared Utilities"]:::shared
        Types["Type Definitions"]:::shared
    end

    %% Frontend Layer
    subgraph "React UI" 
        direction TB
        AppWrapper["App Wrapper"]:::ui
        Document["Custom Document"]:::ui
        MainPage["Main Page"]:::ui
        Navbar["Navbar"]:::ui
        AnswerComp["Answer Component"]:::ui
        Player["Player Component"]:::ui
        Footer["Footer Component"]:::ui
        Styles["Global Styles"]:::ui
        Tailwind["Tailwind Config"]:::config
        NextConfig["Next.js Config"]:::config
    end

    %% ETL flow
    Scraper -->|"outputs JSON"| TextEmbedder
    TextEmbedder -->|"embeddings → INSERT"| SupabaseDB
    AudioProcessor -->|"transcripts JSON"| AudioEmbedder
    AudioEmbedder -->|"embeddings → INSERT"| SupabaseDB

    %% Schema
    Schema --> SupabaseDB

    %% API interactions
    SearchAPI -->|"vector search"| SupabaseDB
    SearchAPI -->|"call embeddings"| OpenAI
    AnswerAPI -->|"vector search"| SupabaseDB
    AnswerAPI -->|"call chat completion"| OpenAI
    Utils --> SearchAPI
    Utils --> AnswerAPI
    Types --> SearchAPI
    Types --> AnswerAPI

    %% Frontend to API
    MainPage -->|"calls /api/search"| SearchAPI
    MainPage -->|"calls /api/answer"| AnswerAPI
    Navbar --> MainPage
    AnswerComp --> MainPage
    Player --> MainPage
    Footer --> MainPage

    %% UI structure
    AppWrapper --> Document
    Document --> MainPage
    MainPage --> Navbar
    MainPage --> AnswerComp
    MainPage --> Player
    MainPage --> Footer
    Styles --> MainPage
    Tailwind --> Styles
    NextConfig --> AppWrapper

    %% External source to ETL
    PodcastSrc --> AudioProcessor
    PodcastSrc --> Scraper

    %% Click Events
    click Scraper "https://github.com/mckaywrigley/naval-gpt/blob/main/scripts/scrape.ts"
    click TextEmbedder "https://github.com/mckaywrigley/naval-gpt/blob/main/scripts/embed-text.ts"
    click AudioProcessor "https://github.com/mckaywrigley/naval-gpt/blob/main/scripts/main.py"
    click AudioEmbedder "https://github.com/mckaywrigley/naval-gpt/blob/main/scripts/embed-audio.ts"
    click Schema "https://github.com/mckaywrigley/naval-gpt/blob/main/schema.sql"
    click MainPage "https://github.com/mckaywrigley/naval-gpt/blob/main/pages/index.tsx"
    click SearchAPI "https://github.com/mckaywrigley/naval-gpt/blob/main/pages/api/search.ts"
    click AnswerAPI "https://github.com/mckaywrigley/naval-gpt/blob/main/pages/api/answer.ts"
    click Utils "https://github.com/mckaywrigley/naval-gpt/blob/main/utils/index.ts"
    click Types "https://github.com/mckaywrigley/naval-gpt/blob/main/types/index.ts"
    click AnswerComp "https://github.com/mckaywrigley/naval-gpt/blob/main/components/Answer/Answer.tsx"
    click Navbar "https://github.com/mckaywrigley/naval-gpt/blob/main/components/Navbar.tsx"
    click Player "https://github.com/mckaywrigley/naval-gpt/blob/main/components/Player.tsx"
    click Footer "https://github.com/mckaywrigley/naval-gpt/blob/main/components/Footer.tsx"
    click Styles "https://github.com/mckaywrigley/naval-gpt/blob/main/styles/globals.css"
    click Tailwind "https://github.com/mckaywrigley/naval-gpt/blob/main/tailwind.config.js"
    click NextConfig "https://github.com/mckaywrigley/naval-gpt/blob/main/next.config.js"
    click AppWrapper "https://github.com/mckaywrigley/naval-gpt/blob/main/pages/_app.tsx"
    click Document "https://github.com/mckaywrigley/naval-gpt/blob/main/pages/_document.tsx"

    %% Styles
    classDef etl fill:#d2f8d2,stroke:#2ecc71,stroke-width:2px
    classDef db fill:#f0e68c,stroke:#f1c40f,stroke-width:2px
    classDef api fill:#aed6f1,stroke:#3498db,stroke-width:2px
    classDef ui fill:#fadbd8,stroke:#e74c3c,stroke-width:2px
    classDef shared fill:#d5dbdb,stroke:#7f8c8d,stroke-width:2px
    classDef config fill:#f5cba7,stroke:#e67e22,stroke-width:2px
    classDef external fill:none,stroke-dasharray: 5 5,stroke:#95a5a6,stroke-width:2px
```
