# themattberman/gpt4-pdf-chatbot-langchain architecture

> Architecture diagram of the GitHub repository themattberman/gpt4-pdf-chatbot-langchain, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/themattberman/gpt4-pdf-chatbot-langchain
- Repository: https://github.com/themattberman/gpt4-pdf-chatbot-langchain

## Overview

##### Main Components
- Frontend (User Interface):
- Located in the “pages” and “components” directories.
- The “pages/_app.tsx”, “pages/index.tsx” and “pages/_document.tsx” files indicate a Next.js application rendering a chat interface to interact with users.
- UI subcomponents are in “components/ui” (e.g., LoadingDots, TextArea, accordion) which build the interactive chat experience.
- Backend/API Layer:
- Found under “pages/api/chat.ts”. This API endpoint receives user queries, routes them to the LangChain (and underlying OpenAI) logic, and sends back responses.
- PDF Ingestion & Processing:
- The “scripts/ingest-data.ts” script is used to load and convert PDF files (located in the “docs” folder) into embeddings.
- The “utils/customPDFLoader.ts” supports this by processing PDF contents.
- Declarations in “declarations/pdf-parse.d.ts” help with type-safety when parsing PDFs.
- External Services & Integrations:
- OpenAI: Used for generating responses (configured in “utils/openai-client.ts” and “utils/makechain.ts”).
- Pinecone: The vector store for embedding storage and retrieval (see “config/pinecone.ts” and “utils/pinecone-client.ts”).
- LangChain: Although not explicitly detailed in the file tree, it’s integrated via utilities and is central to orchestrating the chatbot workflow.

##### Data Flow & Relationships
- User Interaction: The user enters a question via the frontend chat interface.
- API Request: The frontend sends the query to the backend API endpoint (pages/api/chat.ts).
- Chat Chain & External Calls:
- The API uses utilities (e.g., makechain.ts) to construct a conversation chain and interacts with the OpenAI API to get a response.
- It may query Pinecone (via the pinecone client) to retrieve relevant embedded content from ingested PDF documents.
- PDF Data Ingestion:
- The “scripts/ingest-data.ts” reads PDF files from the “docs” directory, processes them using a custom PDF loader, and sends the content for embedding.
- Embedded vectors are stored in Pinecone under a specified namespace.
- Response: The processed answer is sent back through the API to be rendered on the chat interface.

##### Architectural Patterns and Design Principles
- Separation of Concerns: The project cleanly separates the presentation layer (Next.js frontend), the application logic (API routes and utilities), and external integrations (Pinecone, OpenAI, LangChain).
- Modular Architecture: Each major functionality (UI components, PDF processing, API handling, external service integration) resides in its dedicated folder/module.
- Configurability & Environment-Specific Setup: The use of an “.env” file (with an example provided) along with configuration files under “config” ensures that service keys and runtime configurations are managed separately.

##### Relevant Technologies
- Frameworks/Tools: Next.js for server-side rendering and routing, Typescript for static typing.
- Libraries: LangChain for orchestrating LLM apps, Pinecone for vector storage, and OpenAI’s API for generating chat responses.
- Utilities: Various utility modules (like “utils/cn.ts” for conditional classnames, “tsconfig.json” for TypeScript configuration) to support building a maintainable application.

## Diagram (Mermaid)

```mermaid
flowchart TD
    User["User"]:::external

    subgraph "Frontend/UI"
        FE1["pages/_app.tsx"]:::frontend
        FE2["pages/index.tsx"]:::frontend
        FE3["pages/_document.tsx"]:::frontend
        FE4["components/ui/LoadingDots.tsx"]:::frontend
        FE5["components/ui/TextArea.tsx"]:::frontend
        FE6["components/ui/accordion.tsx"]:::frontend
    end

    subgraph "Backend/API Layer"
        BE["pages/api/chat.ts"]:::backend
    end

    subgraph "PDF Ingestion Module"
        PDF1["scripts/ingest-data.ts"]:::processing
        PDF2["utils/customPDFLoader.ts"]:::processing
        PDF3["declarations/pdf-parse.d.ts"]:::processing
    end

    subgraph "OpenAI Integration (LangChain)"
        OAI1["utils/openai-client.ts"]:::openai
        OAI2["utils/makechain.ts"]:::openai
        OAI_API["OpenAI_API"]:::external
    end

    subgraph "Pinecone Integration"
        PC1["config/pinecone.ts"]:::pinecone
        PC2["utils/pinecone-client.ts"]:::pinecone
        PineconeSV["Pinecone_Service"]:::external
    end

    %% Data Flows
    User -->|"User_Query"| FE2
    FE2 -->|"API_Request"| BE
    BE -->|"LLM_Request"| OAI1
    OAI1 -->|"Call_API"| OAI_API
    OAI_API -->|"Return_Response"| OAI1
    BE -->|"Embedding_Query"| PC2
    PC2 -->|"Pinecone_Query"| PineconeSV
    PineconeSV -->|"Return_Embeddings"| PC2
    PDF1 -->|"Index_Data"| PC2

    %% Styles
    classDef frontend fill:#BBDEFB,stroke:#0D47A1,stroke-width:2px;
    classDef backend fill:#C8E6C9,stroke:#2E7D32,stroke-width:2px;
    classDef processing fill:#FFF9C4,stroke:#F9A825,stroke-width:2px;
    classDef openai fill:#FFCDD2,stroke:#C62828,stroke-width:2px;
    classDef pinecone fill:#D1C4E9,stroke:#512DA8,stroke-width:2px;
    classDef external fill:#E0E0E0,stroke:#9E9E9E,stroke-width:2px;

    %% Click Events
    click FE1 "https://github.com/themattberman/gpt4-pdf-chatbot-langchain/blob/main/pages/_app.tsx"
    click FE2 "https://github.com/themattberman/gpt4-pdf-chatbot-langchain/blob/main/pages/index.tsx"
    click FE3 "https://github.com/themattberman/gpt4-pdf-chatbot-langchain/blob/main/pages/_document.tsx"
    click FE4 "https://github.com/themattberman/gpt4-pdf-chatbot-langchain/blob/main/components/ui/LoadingDots.tsx"
    click FE5 "https://github.com/themattberman/gpt4-pdf-chatbot-langchain/blob/main/components/ui/TextArea.tsx"
    click FE6 "https://github.com/themattberman/gpt4-pdf-chatbot-langchain/blob/main/components/ui/accordion.tsx"
    click BE "https://github.com/themattberman/gpt4-pdf-chatbot-langchain/blob/main/pages/api/chat.ts"
    click PDF1 "https://github.com/themattberman/gpt4-pdf-chatbot-langchain/blob/main/scripts/ingest-data.ts"
    click PDF2 "https://github.com/themattberman/gpt4-pdf-chatbot-langchain/blob/main/utils/customPDFLoader.ts"
    click PDF3 "https://github.com/themattberman/gpt4-pdf-chatbot-langchain/blob/main/declarations/pdf-parse.d.ts"
    click OAI1 "https://github.com/themattberman/gpt4-pdf-chatbot-langchain/blob/main/utils/openai-client.ts"
    click OAI2 "https://github.com/themattberman/gpt4-pdf-chatbot-langchain/blob/main/utils/makechain.ts"
    click PC1 "https://github.com/themattberman/gpt4-pdf-chatbot-langchain/blob/main/config/pinecone.ts"
    click PC2 "https://github.com/themattberman/gpt4-pdf-chatbot-langchain/blob/main/utils/pinecone-client.ts"
```
