How pipedreamhq/mcp works
Project type and purpose
• This is a monorepo containing three examples of integrating with Pipedream’s dynamic Model Context Protocol (MCP) server:
• A full-stack Next.js “Chat App” under examples/chat
• A CLI “AI SDK” example under examples/ai-sdk
• A CLI “OpenAI SDK” example under examples/openai-sdk
• The primary focus is the Chat App Example, a production-ready Next.js web application demonstrating real-time chat, authentication, persistence, and dynamic tool discovery via Pipedream’s MCP server.
File structure
• Top-level uses pnpm workspaces; shared libs and configs reside under examples/shared.
• examples/chat directory follows Next.js 13+ App Router structure:
• app/… contains React server/client components and API routes (/api/chat, /api/history, /api/files, /api/suggestions, /api/vote, /auth)
• components/… houses UI primitives, higher-level widgets, and shared hooks
• lib/… holds domain logic: AI providers, tool definitions, DB schema and migrations (Drizzle ORM), auth utils
• middleware.ts for edge logic
• tests/ cover integration and unit tests
• config files: next.config.ts, tailwind.config.ts, drizzle.config.ts, Docker-compose.yml
• docs/ guide on setup, models, artifacts, migrations
• examples/ai-sdk and examples/openai-sdk are small TS CLI apps demonstrating MCP calls in a single Node process.
README for additional insights
• Emphasizes dynamic tool discovery per MCP spec
• Shows environment variables for auth, persistence, and LLM keys
• Describes three example entrypoints: pnpm chat, pnpm ai-sdk, pnpm openai-sdk
Main components
a. Frontend (Browser)
• Next.js React pages & components
• Authentication UI (NextAuth sign-in/register)
• Chat interface, file upload, document preview, code editor widgets
b. Backend (Next.js Server)
• Server-side rendering & API routes
• NextAuth OAuth callbacks to Pipedream Connect
• API endpoints for chat streaming, document management, file uploads, suggestions, vote tracking, healthcheck
• Middleware for session handling
c. Data Layer
• Postgres (or compatible) via Drizzle ORM
• Tables: users, chats, messages, artifacts, migrations metadata
d. MCP Service Integration
• Pipedream MCP Server (external) for dynamic tool discovery & execution
• Wrapper in lib/ai/providers.ts and lib/pd-backend-client.ts
e. External Services invoked via MCP
• OpenAI API (LLM provider)
• Third-party APIs (Slack, Weather, custom Pipedream integrations)
f. CLI Examples
• ai-sdk and openai-sdk processes that invoke MCP calls directly
Relationships and data flow
• Browser → Next.js API → Drizzle DB (for persistence)
• Browser → Next.js API → Pipedream MCP Server → External APIs/LLMs → Next.js API → Browser
• NextAuth flow: Browser ↔ NextAuth API ↔ Pipedream Connect OAuth endpoints
• Shared code: examples/shared is imported by both CLI examples and chat back end
• CLI flow: Node process → MCP Server → external APIs → STDOUT
Architectural patterns and design principles
• Layered architecture: UI layer, API layer, service/integration layer, data layer
• Separation of concerns: UI components vs. business logic in lib/ vs. data access in lib/db
• Dynamic service discovery: MCP server acts as a registry and executor of tools at runtime
• Serverless-style API routes: each /api/… route is its own handler
• Test coverage: integration tests for API routes and prompts
• Browser UI → Next.js Server (API + SSR) → Drizzle/Postgres
• Next.js Server → Pipedream MCP Server → External APIs/LLMs
• NextAuth OAuth flow → Pipedream Connect
• CLI examples flow separate from Chat App
• Notes on dynamic tool discovery as central pattern