How botpress/v12 works
Project type and purpose
• Botpress is a full-stack, extensible chatbot development platform.
• It consists of a Node.js/TypeScript server that hosts a plugin-based “core” plus dozens of feature modules, and a React-based admin/studio UI (and a lightweight UI for embedding).
• The platform offers built-in natural language processing (NLU), messaging channel integrations, a flow designer, and a module/plugin ecosystem.
File structure
• Top-level “packages/bp” contains the core server code (Express-style HTTP API, real-time socket, module loader, DI container).
• “modules/*” holds individual features (analytics, basic-skills, builtin, nlu, hitl, channel-web, code-editor, etc.) each with its own backend API, hooks, content-types, views (full & lite), migrations, translations.
• “packages/ui-admin” and “packages/ui-lite” are the React-Redux admin and lightweight embeddable web clients.
• “examples” and “bench” folders illustrate sample bots, custom components, stress tests.
• Config files include Dockerfile, Kubernetes/heroku.yml, CI pipeline workflows under .github/workflows.
• Build tools: Gulp, Webpack (module-builder), Babel, TS config, migration scripts (Knex).
README for additional insights
• Deploy via binary, Docker image, or source.
• Plugin architecture: modules loaded dynamically at runtime.
Main components
a. Core Server (Node.js/TypeScript)
• HTTP API Layer (Express + custom routers)
• Real-time Layer (Socket.io JWT gateway)
• Module Loader (Inversify DI, plugin registry)
• Event Engine & Job Queue (in-memory + Redis for distributed jobs)
• BPM/Flow Engine (Dialog engine, instruction queue)
• NLU Service (embedded or external)
• Action Server (local or external process for user code)
• Storage:
• Relational DB (PostgreSQL/SQLite via Knex + migrations)
• File System driver (ghost FS for bot assets)
• Distributed sub-services: Redis for locks/queues, optional message broker.
b. Modules
• Each module registers backend API endpoints, database migrations, hooks, content types, views.
• Key modules: basic-skills, analytics, broadcasting, channel-web (webchat), nlu, hitl/hitlnext (human-in-the-loop), code-editor, uipath, etc.
c. Front-end Clients
• UI-Admin (React+Redux SPA for bot & workspace administration, health, channels, modules)
• UI-Lite (embeddable React component for chat UI)
• Channel-Web (fully featured web chat client)
d. External Integrations
• Messaging Channels (Slack, Messenger, Twilio, SMS, custom brokers)
• External NLU Engines (Rasa, Dialogflow, LUIS)
• Webhooks & custom modules via SDK
Relationships and interactions
• Clients ↔ HTTP API (REST/JSON) for configuration, content, user management, analytics.
• Chat clients ↔ Real-time Socket (bi-directional chat events).
• Core Server ↔ Database for persistence (config, dialogs, events, metrics).
• Core Server ↔ File system for bot assets, media (ghost FS).
• Core Server ↔ Redis for distributed locks, job queues.
• Core Server ↔ NLU component (local module “nlu” or external service).
• Core Server ↔ Action Server for executing user-provided code.
• Core Server ↔ Module plug-ins (dynamically loaded at startup).
• CI/CD & Build Pipeline ↔ Docker Hub / Artifactory.
Architectural patterns & design principles
• Plugin-based modular architecture (Inversify DI, dynamic module loader).
• Layered design: API layer, service layer, persistence layer, real-time layer.
• CQRS/Event-driven for dialog/instruction queue.
• Separation of concerns: core vs modules vs front-end.
• Micro-frontend pattern: each module can supply its own views in the admin UI.
• Containerization & Infrastructure as code (Docker, Kubernetes manifests).
Detailed capture of essential elements
• Core boot sequence: Inversify container loads modules → runs migrations → starts HTTP & real-time servers.
• Module lifecycle: module declares hooks, actions, views → core invokes hooks upon events.
• NLU pipeline: user message → core preprocessing → NLU module → intent/entities → dialog engine.
• Action execution: dialog engine enqueues instructions → instruction processor → action server (JS sandbox).
• Event storage: every incoming/outgoing event persisted in events table.
• UI updates: front-end subscribes to real-time events for chat and admin notifications.