How emojumprepo/worldseek-memory works
Project type and purpose
• This is a full-stack AI Agent Platform (“FastGPT”) in a monorepo, combining:
• A Next.js-based frontend (projects/app) for UI & SSR/CSR
• A Node.js API layer (pages/api) backed by a “service” module
• Background workers for CPU/IO-heavy tasks (service/worker, BullMQ)
• Microservices (projects/mcp_server, projects/sandbox)
• Plugin & template runtimes (packages/plugins, packages/templates)
• A documentation site (docSite)
• Helm/Docker deployment manifests
File structure
• Top-level directories denote layers:
• packages/global, packages/service, packages/web: shared libraries
• packages/plugins, packages/templates: dynamic extensions
• projects/app: the main Next.js UI
• projects/mcp_server & sandbox: standalone microservices
• service/worker: background job processors
• service/common, core, support: layered backend business logic
• deploy/docker & deploy/helm: Kubernetes/Docker deployment
README for insights
• Core features: workflow orchestration, knowledge base storage (MongoDB + vector DB via PGVector/Milvus), OpenAPI interface, embeddable chat, plugin & template marketplace
• Tech stack: Next.js, TypeScript, ChakraUI, MongoDB, PostgreSQL (with PGVector) or Milvus, Redis, BullMQ, Helm/Kubernetes
Main components & their interactions
a. Frontend (Next.js UI)
• Renders pages & components (projects/app/src/pages & packages/web)
• Calls API endpoints under /api (hosted in Next.js)
• Embeds chat widget via an Iframe/API
b. API Server (Next.js API + service module)
• HTTP endpoints implement core operations (chat, dataset, app, workflow)
• Uses service/common for middleware (CORS, auth, rate-limit)
• Delegates to service/core for business logic (AI calls, dataset management, workflow dispatch)
• Persists data in MongoDB (sessions, logs, configs) and Postgres/Milvus (vector indices)
• Uses Redis for caching & BullMQ as a job queue
c. Background Workers
• service/worker actors consume BullMQ queues
• Perform tasks like embedding generation, file parsing (PDF→text), transcription, vector indexing
• Write results back to DB or signal completion
d. Microservices
• mcp_server: lightweight proxy for MCP tools, external integrations
• sandbox: code‐sandbox service for secure JS execution in workflows
e. Plugin & Template Runtime
• packages/plugins: dynamically loadable user plugins run in a worker/VM
• packages/templates: prebuilt prompt templates shown in UI
f. External Services
• AI providers via one-api or custom aiproxy
• Third-party data sources (webcrawler, OpenAPI integrations, outLink)
• Authentication/SSO via external identity providers
g. Data Stores
• MongoDB: primary document store (user data, chat logs, configs)
• PostgreSQL + PGVector or Milvus: vector databases for semantic search
• Redis: cache & BullMQ backend
Architectural patterns & design principles
• Layered architecture: common → core → support → third-provider modules
• Monorepo with pnpm workspaces for code sharing
• Microservices for isolation of sandbox and MCP proxy
• Serverless‐style API in Next.js for rapid iteration
• Event-driven background jobs via BullMQ queues
• Plugin architecture: user‐provided code runs in isolated runtime
• Kubernetes/Helm for deployment and scaling