# codehash001/documatey architecture

> Architecture diagram of the GitHub repository codehash001/documatey, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/codehash001/documatey
- Repository: https://github.com/codehash001/documatey

## Overview

##### Project Type and Purpose
- This is a full-stack Next.js (React + TypeScript) application providing a vector-search assistant: it ingests documentation (URL or text), chunks it, computes embeddings via Google Generative AI, stores/retrieves vectors in TiDB Serverless, and offers a chat-style, step-wise RAG (retrieval-augmented generation) wizard.

##### File Structure Analysis
- app/
- api/chat, clarify, plan, search, index, status – Next.js API routes (serverless functions).
- layout.tsx, page.tsx, globals.css – root layout, home page UI.
- components/ui/… – reusable UI building blocks (buttons, cards, inputs, theme toggle).
- components/wizard.tsx – multi-step RAG UI controlling indexing, clarifications, plannning, chat.
- lib/… – core business logic modules:
- chunk.ts – text splitting.
- embed.ts – calling Google embeddings.
- search.ts – vector query logic.
- llm.ts – calling Google Gemini LLM.
- tidb.ts – TiDB database connection and upsert/index management.
- utils.ts – shared helpers.
- next.config.ts, tsconfig.json, ESLint/PostCSS configs.

##### README Insights
- Env vars configure TiDB connection & Google API keys.
- Describes the three-step user flow: index docs, clarify goal, generate plan with citations; context-aware chat via step IDs.
- Highlights TiDB HNSW vector index creation on embedded_documents table.

a. Main Components
- Frontend (Next.js app directory + components).
- API Layer (Next.js serverless functions under app/api/*).
- Business Logic Modules (lib/*.ts).
- Vector Store (TiDB cloud cluster).
- Embedding & LLM Service (Google Generative AI endpoints).
b. Relationships & Data Flows
1. User interacts with the Wizard UI → triggers REST calls to API routes.
2. API route “/api/index” → uses lib/chunk → lib/embed → lib/tidb to upsert vectors.
3. API route “/api/search” or “/api/chat” → uses lib/search → optionally lib/llm for RAG responses.
4. TiDB returns nearest-neighbor vectors → API returns JSON/Markdown to front end.
5. Frontend renders responses and maintains step state.
c. Architectural Patterns
- Serverless functions (One route per business use-case).
- Layered separation: UI → API → Service modules → Data store.
- Shared library modules in lib/*.
- RAG pipeline: chunk → embed → store → retrieve → generate.
d. Key Technologies
- Next.js App Router (app/ directory)
- React + TypeScript + CSS (components/ui)
- TiDB Serverless (MySQL-compatible, HNSW index)
- Google Generative AI (text-embedding-004, Gemini LLM)
- Node 20.x, npm scripts (dev, build, start)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% UI Layer
    subgraph "Browser UI"
        subgraph "Next.js App Router"
            Layout["layout.tsx"]:::ui
            Page["page.tsx"]:::ui
            Globals["globals.css"]:::ui
        end
        Wizard["Wizard UI"]:::ui
        subgraph "Reusable UI Components"
            Beam["beam-background.tsx"]:::ui
            Button["button.tsx"]:::ui
            Card["card.tsx"]:::ui
            Input["input.tsx"]:::ui
            Select["select.tsx"]:::ui
            Textarea["textarea.tsx"]:::ui
            Theme["theme-toggle.tsx"]:::ui
        end
    end

    %% API Layer
    subgraph "API Layer"
        IndexAPI["/api/index"]:::api
        ClarifyAPI["/api/clarify"]:::api
        PlanAPI["/api/plan"]:::api
        SearchAPI["/api/search"]:::api
        ChatAPI["/api/chat"]:::api
        StatusAPI["/api/status"]:::api
    end

    %% Service Modules
    subgraph "Service Modules"
        Chunk["chunk.ts"]:::service
        Embed["embed.ts"]:::service
        SearchLib["search.ts"]:::service
        LLM["llm.ts"]:::service
        TiDB["tidb.ts"]:::service
        Utils["utils.ts"]:::service
    end

    %% External Services
    subgraph "External Services"
        TiDBCloud["TiDB Serverless"]:::external
        GoogleEmbed["Google Embeddings"]:::external
        GoogleLLM["Google Gemini LLM"]:::external
        EnvConfig["Environment Config"]:::external
    end

    %% Configuration Files
    subgraph "Configuration Files"
        nextConfig["next.config.ts"]:::external
        tsconfig["tsconfig.json"]:::external
        postcss["postcss.config.mjs"]:::external
        eslint["eslint.config.mjs"]:::external
        pkgjson["package.json"]:::external
        lockjson["package-lock.json"]:::external
    end

    %% User Flow
    User["User"]:::external -->|interacts| Wizard
    Wizard -->|Index docs| IndexAPI
    Wizard -->|Clarify goal| ClarifyAPI
    Wizard -->|Generate plan| PlanAPI
    Wizard -->|Chat message| ChatAPI

    %% Index Flow
    IndexAPI -->|chunk text| Chunk
    Chunk -->|embed vectors| Embed
    Embed -->|upsert vectors| TiDB
    TiDB -->|connect & upsert| TiDBCloud

    %% Clarify Flow
    ClarifyAPI -->|clarify via LLM| LLM
    LLM -->|request| GoogleLLM

    %% Plan Flow
    PlanAPI -->|search NN| SearchLib
    SearchLib -->|search NN| TiDBCloud
    PlanAPI -->|generate plan| LLM
    LLM -->|request| GoogleLLM

    %% Search Flow
    SearchAPI -->|search NN| SearchLib
    SearchLib -->|search NN| TiDBCloud
    SearchAPI -->|synthesize| LLM
    LLM -->|request| GoogleLLM

    %% Chat Flow
    ChatAPI -->|retrieve context| SearchLib
    SearchLib -->|search NN| TiDBCloud
    ChatAPI -->|generate response| LLM
    LLM -->|request| GoogleLLM

    %% Env Config Flows
    EnvConfig --> TiDBCloud
    EnvConfig --> GoogleEmbed
    EnvConfig --> GoogleLLM

    %% Configuration references
    EnvConfig --> nextConfig
    EnvConfig --> tsconfig
    EnvConfig --> postcss
    EnvConfig --> eslint
    EnvConfig --> pkgjson
    EnvConfig --> lockjson

    %% Click Events
    click Layout "https://github.com/codehash001/documatey/blob/main/app/layout.tsx"
    click Page "https://github.com/codehash001/documatey/blob/main/app/page.tsx"
    click Globals "https://github.com/codehash001/documatey/blob/main/app/globals.css"
    click Wizard "https://github.com/codehash001/documatey/blob/main/components/wizard.tsx"
    click Beam "https://github.com/codehash001/documatey/blob/main/components/ui/beam-background.tsx"
    click Button "https://github.com/codehash001/documatey/blob/main/components/ui/button.tsx"
    click Card "https://github.com/codehash001/documatey/blob/main/components/ui/card.tsx"
    click Input "https://github.com/codehash001/documatey/blob/main/components/ui/input.tsx"
    click Select "https://github.com/codehash001/documatey/blob/main/components/ui/select.tsx"
    click Textarea "https://github.com/codehash001/documatey/blob/main/components/ui/textarea.tsx"
    click Theme "https://github.com/codehash001/documatey/blob/main/components/ui/theme-toggle.tsx"
    click IndexAPI "https://github.com/codehash001/documatey/blob/main/app/api/index/route.ts"
    click ClarifyAPI "https://github.com/codehash001/documatey/blob/main/app/api/clarify/route.ts"
    click PlanAPI "https://github.com/codehash001/documatey/blob/main/app/api/plan/route.ts"
    click SearchAPI "https://github.com/codehash001/documatey/blob/main/app/api/search/route.ts"
    click ChatAPI "https://github.com/codehash001/documatey/blob/main/app/api/chat/route.ts"
    click StatusAPI "https://github.com/codehash001/documatey/blob/main/app/api/status/route.ts"
    click Chunk "https://github.com/codehash001/documatey/blob/main/lib/chunk.ts"
    click Embed "https://github.com/codehash001/documatey/blob/main/lib/embed.ts"
    click SearchLib "https://github.com/codehash001/documatey/blob/main/lib/search.ts"
    click LLM "https://github.com/codehash001/documatey/blob/main/lib/llm.ts"
    click TiDB "https://github.com/codehash001/documatey/blob/main/lib/tidb.ts"
    click Utils "https://github.com/codehash001/documatey/blob/main/lib/utils.ts"
    click nextConfig "https://github.com/codehash001/documatey/blob/main/next.config.ts"
    click tsconfig "https://github.com/codehash001/documatey/blob/main/tsconfig.json"
    click postcss "https://github.com/codehash001/documatey/blob/main/postcss.config.mjs"
    click eslint "https://github.com/codehash001/documatey/blob/main/eslint.config.mjs"
    click pkgjson "https://github.com/codehash001/documatey/blob/main/package.json"
    click lockjson "https://github.com/codehash001/documatey/blob/main/package-lock.json"

    %% Styles
    classDef ui fill:#AEDFF7,stroke:#333,color:#000
    classDef api fill:#C2F0C2,stroke:#333,color:#000
    classDef service fill:#FFF2CC,stroke:#333,color:#000
    classDef external fill:#FDD9B5,stroke:#333,color:#000
```
