How nalindalal/zapier works
Project’s Main Components
• Frontend – A Next.js application (located under the “frontend” folder) that provides the user interface for login, signup, dashboard, and zap creation. It interfaces with the backend through API calls (using axios).
• Primary Backend – An Express.js-based service (in the “primary-backend” folder) that sets up routes (e.g., user and zap routers) to handle core requests from the frontend. It also integrates schema-based validations (with libraries like zod) and authentication logic.
• Hooks – A component that initializes webhooks and sets up database interactions using Prisma. It includes the initial schema (User, Zap, Trigger, Action, etc.) and represents the first entry point for webhook-related processes.
• Processor – A Kafka-based service (in the “processor” folder) that is responsible for consuming and processing asynchronous events from Kafka topics (like “zap-events”). It further processes the data received (e.g., by performing transformations or filtering) before passing it along.
• Worker – A background job handler (in the “worker” folder) that takes on additional processing tasks such as sending emails, parsing data, and interacting with external services (like Solana blockchain operations).
Interactions and Data Flow
• The Frontend issues API calls to the Primary Backend. The frontend also may trigger events that eventually result in webhooks being created.
• The Primary Backend receives these requests and interacts with the database using Prisma. It may also forward events to the asynchronous processing layer.
• The Hooks component manages webhook registrations and directly deals with webhook events (e.g., an event fired from GitHub), ensuring that the correct trigger states are updated and data is recorded.
• When a webhook event occurs, it is communicated asynchronously—typically published into the Kafka message broker.
• The Processor service consumes these Kafka messages, processes them, and may prepare specific payloads for further actions.
• The Worker module picks up processing tasks (e.g., sending emails, parsing data, executing Solana-related actions) based on outputs from the processor or through scheduled triggers.
• A PostgreSQL database (run in Docker in different modules) is central to storing the schema-driven data. Ensure you capture that each service uses Prisma to interact with a shared or replicated DB.
Architectural Patterns and Design Principles
• Separation of Concerns – Notice how the frontend, API/backend, webhooks, asynchronous processing (processor) and background tasks (worker) are separated into different modules.
• Asynchronous Messaging – The use of Kafka clearly defines a decoupled, event-driven architecture for handling webhook events.
• Monorepo Structure – Emphasize that while each module (frontend, hooks, primary-backend, processor, worker) exists independently, they are orchestrated together as part of a larger system.