How chroma-core/gpt4-pdf-chatbot-langchain-chroma works
Project type and purpose
• This is a full-stack Next.js application that builds a ChatGPT-style chatbot for querying multiple PDF files.
• The primary libraries are Next.js (frontend + serverless API), LangChain (chain creation for LLM workflows), Chroma (vector store for embeddings), and OpenAI (GPT-4/other models).
File structure
• Top-level directories:
• components – React UI components and layout
• pages – Next.js pages and API routes (pages/api/chat.ts)
• scripts – data ingestion script (scripts/ingest-data.ts)
• utils – helper functions and chain setup (utils/makechain.ts, utils/customPDFLoader.ts)
• config – Chroma configuration (config/chroma.ts)
• public, styles, types, declarations – static assets, styling, TypeScript types
• Key config files: .env.example, next.config.js, tsconfig.json, package.json
• The ingestion process is decoupled into a standalone script that runs in Node, whereas the main app uses Next.js serverless functions.
README for additional insights
• Describes a two-step workflow:
a. Convert PDF files into embeddings via the ingestion script and store them in Chroma (Docker container).
b. Run the Next.js app, where client chat input hits a serverless API that retrieves relevant embeddings from Chroma, adds context to a prompt, and calls OpenAI for an answer.
• Mentions key env vars (OPENAI_API_KEY, COLLECTION_NAME) and Docker for Chroma.
Main components
a. Frontend UI (Next.js pages + React components)
b. Backend API (Next.js pages/api/chat.ts)
c. Ingestion service (Node.js script scripts/ingest-data.ts)
d. PDF storage (local docs folder)
e. Vector database (Chroma running in Docker)
f. External LLM service (OpenAI API)
Relationships and data flow
• Ingestion flow: PDF files → custom PDF loader → LangChain splitter/embeddings → Chroma vector store
• Runtime flow: User in browser → Next.js frontend → API route /api/chat → LangChain retrieval via Chroma → constructs prompt → OpenAI API → response → frontend displays answer
• Environment: .env provides API keys and collection name injected into both ingestion script and API route
Architectural patterns and design principles
• Separation of concerns: data ingestion vs. real-time chat
• Serverless functions for API endpoints (Next.js pattern)
• Use of a vector database for retrieval-augmented generation (RAG)
• Functional composition in LangChain (chains, prompt templates)
• Containerization of Chroma DB for easy local deployment