# sohila-khaled-abbas/n8n-production-queue-setup architecture

> Architecture diagram of the GitHub repository sohila-khaled-abbas/n8n-production-queue-setup, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sohila-khaled-abbas/n8n-production-queue-setup
- Repository: https://github.com/sohila-khaled-abbas/n8n-production-queue-setup

## Overview

This repository packages a self-hosted n8n automation stack and an optional AI workflow-building assistant. The main runtime path is browser or webhook input into n8n, queued execution through Redis, and worker execution with task-runner sidecars; queue depth also drives autoscaling and monitoring. PostgreSQL persists n8n state, while the assistant generates workflows, keeps chat history, and can ingest workflows and credentials through the n8n API. Runtime wiring is documented in the README; assistant module relationships are supported by the excerpts and SOURCE INDEX. Some integrations and backup behavior are documented but their detailed implementation was not sampled.

Main entry points: [`docker-compose.yml`](https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/docker-compose.yml) (n8n workers), [`autoscaler/autoscaler.py`](https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/autoscaler/autoscaler.py) (Worker autoscaler), [`mcp_server/llm_generator.py`](https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/mcp_server/llm_generator.py) (Workflow generator), [`mcp_server/server.py`](https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/mcp_server/server.py) (Assistant chat API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_execution["Workflow execution"]
  node_n8n_main["n8n editor and API<br/>[docker-compose.yml]"]
  node_webhook_processor["Webhook processor<br/>[docker-compose.yml]"]
  node_workers["n8n workers<br/>[docker-compose.yml]"]
  node_task_runners["Task runners"]
  node_workflow_assets["Workflow portfolio"]
end

subgraph group_operations["Scaling and operations"]
  node_autoscaler["Worker autoscaler<br/>[autoscaler.py]"]
  node_queue_monitor["Queue monitor"]
  node_backup_service["Scheduled backups<br/>[docker-compose.yml]"]
end

subgraph group_assistant["AI assistant"]
  node_assistant_ui_api["Assistant chat API<br/>[server.py]"]
  node_llm_generator["Workflow generator<br/>[llm_generator.py]"]
  node_chat_database[("Chat history store<br/>[database.py]")]
  node_n8n_client["n8n API client<br/>[n8n_client.py]"]
end

subgraph group_services["State and integrations"]
  node_redis_queue[("Execution queue<br/>[docker-compose.yml]")]
  node_postgres[("PostgreSQL persistence<br/>[docker-compose.yml]")]
  node_vector_db[("Qdrant vector database<br/>[docker-compose.yml]")]
  node_waha["WhatsApp gateway<br/>[docker-compose.yml]"]
end

node_browser(("Browser users"))
node_webhook_clients(("Webhook callers"))
node_external_models["External model APIs"]
node_cloud_backup["Cloud backup target"]

node_browser -->|"uses editor"| node_n8n_main
node_webhook_clients -->|"sends webhooks"| node_webhook_processor
node_webhook_processor -->|"enqueues executions"| node_redis_queue
node_n8n_main -->|"enqueues executions"| node_redis_queue
node_workers -->|"dequeues jobs"| node_redis_queue
node_workers -->|"runs with sidecars"| node_task_runners
node_n8n_main -->|"persists state"| node_postgres
node_autoscaler -->|"reads queue depth"| node_redis_queue
node_autoscaler -->|"scales replicas"| node_workers
node_autoscaler -->|"scales alongside"| node_task_runners
node_queue_monitor -->|"monitors depth"| node_redis_queue
node_backup_service -->|"backs up data"| node_postgres
node_backup_service -->|"uploads backups"| node_cloud_backup
node_assistant_ui_api -->|"reads and writes history"| node_chat_database
node_assistant_ui_api -->|"requests generation"| node_llm_generator
node_llm_generator -->|"uses workflow context"| node_n8n_client
node_n8n_client -->|"fetches workflows and credentials"| node_n8n_main
node_llm_generator -.->|"calls model APIs"| node_external_models
node_workers -.->|"supports RAG workflows"| node_vector_db
node_workers -.->|"supports WhatsApp workflows"| node_waha
node_workflow_assets -.->|"imports workflows"| node_n8n_main

click node_n8n_main "https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/docker-compose.yml"
click node_webhook_processor "https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/docker-compose.yml"
click node_redis_queue "https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/docker-compose.yml"
click node_workers "https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/docker-compose.yml"
click node_task_runners "https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/n8n-task-runners.json"
click node_postgres "https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/docker-compose.yml"
click node_autoscaler "https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/autoscaler/autoscaler.py"
click node_queue_monitor "https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/monitor/monitor_redis_queue.py"
click node_backup_service "https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/docker-compose.yml"
click node_assistant_ui_api "https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/mcp_server/server.py"
click node_llm_generator "https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/mcp_server/llm_generator.py"
click node_chat_database "https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/mcp_server/database.py"
click node_n8n_client "https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/mcp_server/n8n_client.py"
click node_vector_db "https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/docker-compose.yml"
click node_waha "https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/docker-compose.yml"
click node_workflow_assets "https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/tree/main/workflows"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n8n_main,node_webhook_processor,node_workers,node_task_runners,node_workflow_assets,node_browser toneBlue
class node_autoscaler,node_queue_monitor,node_backup_service toneAmber
class node_assistant_ui_api,node_llm_generator,node_chat_database,node_n8n_client,node_webhook_clients,node_external_models toneMint
class node_redis_queue,node_postgres,node_vector_db,node_waha toneRose
class node_cloud_backup toneIndigo
```

## Components

### Workflow execution

- [n8n editor and API](https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/docker-compose.yml): `docker-compose.yml`
- [Webhook processor](https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/docker-compose.yml): `docker-compose.yml`
- [n8n workers](https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/docker-compose.yml): `docker-compose.yml`
- [Task runners](https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/n8n-task-runners.json): `n8n-task-runners.json`
- [Workflow portfolio](https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/tree/main/workflows): `workflows`

### Scaling and operations

- [Worker autoscaler](https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/autoscaler/autoscaler.py): `autoscaler/autoscaler.py`
- [Queue monitor](https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/monitor/monitor_redis_queue.py): `monitor/monitor_redis_queue.py`
- [Scheduled backups](https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/docker-compose.yml): `docker-compose.yml`

### AI assistant

- [Assistant chat API](https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/mcp_server/server.py): `mcp_server/server.py`
- [Workflow generator](https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/mcp_server/llm_generator.py): `mcp_server/llm_generator.py`
- [Chat history store](https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/mcp_server/database.py): `mcp_server/database.py`
- [n8n API client](https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/mcp_server/n8n_client.py): `mcp_server/n8n_client.py`

### State and integrations

- [Execution queue](https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/docker-compose.yml): `docker-compose.yml`
- [PostgreSQL persistence](https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/docker-compose.yml): `docker-compose.yml`
- [Qdrant vector database](https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/docker-compose.yml): `docker-compose.yml`
- [WhatsApp gateway](https://github.com/sohila-khaled-abbas/n8n-production-queue-setup/blob/main/docker-compose.yml): `docker-compose.yml`

### Other components

- **Browser users**
- **Webhook callers**
- **External model APIs**
- **Cloud backup target**

## Connections

- Browser users → n8n editor and API: uses editor (evidence: `README.md`)
- Webhook callers → Webhook processor: sends webhooks (evidence: `README.md`)
- Webhook processor → Execution queue: enqueues executions (evidence: `README.md`)
- n8n editor and API → Execution queue: enqueues executions (evidence: `README.md`)
- n8n workers → Execution queue: dequeues jobs (evidence: `README.md`)
- n8n workers → Task runners: runs with sidecars (evidence: `README.md`)
- n8n editor and API → PostgreSQL persistence: persists state (evidence: `README.md`)
- Worker autoscaler → Execution queue: reads queue depth (evidence: `README.md`)
- Worker autoscaler → n8n workers: scales replicas (evidence: `README.md`)
- Worker autoscaler → Task runners: scales alongside (evidence: `README.md`)
- Queue monitor → Execution queue: monitors depth (evidence: `README.md`)
- Scheduled backups → PostgreSQL persistence: backs up data (evidence: `README.md`)
- Scheduled backups → Cloud backup target: uploads backups (evidence: `README.md`)
- Assistant chat API → Chat history store: reads and writes history (evidence: `mcp_server/server.py`)
- Assistant chat API → Workflow generator: requests generation (evidence: `mcp_server/server.py`)
- Workflow generator → n8n API client: uses workflow context (evidence: `mcp_server/llm_generator.py`)
- n8n API client → n8n editor and API: fetches workflows and credentials (evidence: `mcp_server/n8n_client.py`)
- Workflow generator → External model APIs: calls model APIs (evidence: `README.md`)
- n8n workers → Qdrant vector database: supports RAG workflows (evidence: `README.md`)
- n8n workers → WhatsApp gateway: supports WhatsApp workflows (evidence: `README.md`)
- Workflow portfolio → n8n editor and API: imports workflows (evidence: `README.md`)