# apify/n8n-nodes-apify-content-crawler architecture

> Architecture diagram of the GitHub repository apify/n8n-nodes-apify-content-crawler, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/apify/n8n-nodes-apify-content-crawler
- Repository: https://github.com/apify/n8n-nodes-apify-content-crawler

## Overview

1. **Repository purpose**
   - This is an **n8n community node package** that wraps a single Apify actor: **Website Content Crawler**.
   - Its job is to let n8n workflows trigger a crawl, pass input options to Apify, and consume the returned content data.

2. **Primary boundary: n8n vs Apify**
   - The core architectural boundary is between **n8n’s node execution runtime** and the **external Apify platform**.
   - The node is the adapter layer: it translates n8n parameters and credentials into an Apify actor run, then maps the actor output back into n8n items.

3. **Main runtime artifact**
   - The central implementation lives in `nodes/ApifyContentCrawler/ApifyContentCrawler.node.ts`.
   - Its node metadata is defined separately in `ApifyContentCrawler.node.json`, which is typical for n8n community nodes and keeps UI/operation definitions distinct from execution logic.

4. **Operation model**
   - The node exposes a **single operation**: **Run Crawler**.
   - Input configuration is organized under `resources/properties/`, with separate property definitions for **single URL crawl** and **multiple URL crawl** modes.
   - This indicates the node’s UI/input schema is intentionally split by crawl shape rather than by multiple unrelated features.

5. **Execution path**
   - The execution flow is: **n8n workflow input → node parameter parsing → Apify actor invocation → result collection → n8n output items**.
   - `helpers/executeActor.ts` appears to encapsulate the actual Apify execution call.
   - `helpers/genericFunctions.ts` and `helpers/consts.ts` likely provide shared transformation, validation, or naming logic used by the node implementation.

6. **Authentication subsystem**
   - The repository supports two credential types in `credentials/`:
     - `ApifyApi.credentials.ts` for **API key** authentication
     - `ApifyOAuth2Api.credentials.ts` for **OAuth2**
   - This is an important architectural choice: the node is not tied to one auth model, and the credential layer is separated from the node logic, following n8n’s credential boundary.

7. **Input/UX structure**
   - The `resources/resources.ts` and `resources/properties/*` files define how the node is presented in the n8n editor.
   - The README’s screenshots and the separation into resource/property files suggest the node uses n8n’s standard declarative configuration model rather than custom UI code.

8. **Test architecture**
   - Tests live alongside the node under `nodes/ApifyContentCrawler/__tests__/`.
   - The test suite includes workflow-level fixtures and utilities such as:
     - workflow execution helpers
     - credential helpers
     - result extraction helpers
     - workflow JSON fixtures
   - This suggests the node is verified through **integration-style n8n execution tests**, not just unit tests, which is important for a node that depends on credential handling and workflow wiring.

9. **Packaging and release pipeline**
   - The repo is set up as a distributable npm package, with release automation in `.github/workflows/publish.yml`.
   - Release flow described in the README indicates the package version is updated and published to npm from GitHub Actions after a tagged GitHub Release.
   - `package-lock.json`, `package.json`, and CI workflows show this is meant to be shipped as a versioned integration artifact, not run as a standalone service.

10. **Quality gates and developer tooling**
   - The project uses `tsconfig.json`, `eslint.config.mjs`, `.prettierrc.js`, `jest.config.js`, and `gulpfile.js`.
   - This implies a TypeScript-first codebase with linting, formatting, test execution, and a build step integrated into CI.
   - The `.github/workflows/ci.yml` likely runs validation before release, while `claude-md-maintenance.yml` is repo maintenance automation.

11. **Documentation and operational assumptions**
   - The README documents n8n setup, local development, credential setup, and the need for a public `WEBHOOK_URL` when running locally with external triggers.
   - Even though this node is not itself a trigger, that note reflects the broader n8n deployment environment this package assumes.

12. **Diagram-friendly mental model**
   - A useful architecture diagram would show:
     - **n8n workflow / AI Agent tool** on the left
     - **ApifyContentCrawler node** as the adapter in the middle
     - **Apify API / Website Content Crawler actor** on the right
     - **Credentials** as a separate trusted input into the node
     - **Test/CI/release pipeline** as a secondary support track around the package
   - The key data boundary is the actor input/output payload; everything else is adapter, config, or packaging around that flow.

Main entry points: [`nodes/ApifyContentCrawler/ApifyContentCrawler.node.ts`](https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/ApifyContentCrawler.node.ts) (Crawler Node), [`.github/workflows/ci.yml`](https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/.github/workflows/ci.yml) (CI), [`nodes/ApifyContentCrawler/resources/resources.ts`](https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/resources/resources.ts) (Resources), [`nodes/ApifyContentCrawler/__tests__/Apify.node.spec.ts`](https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/__tests__/Apify.node.spec.ts) (Node Tests)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Runtime"]
  node_node["Crawler Node<br/>n8n node"]
  node_meta["Node Meta<br/>n8n manifest"]
  node_resources["Resources<br/>n8n UI schema<br/>[resources.ts]"]
  node_single["Single URL<br/>input schema<br/>[scrapeSingle.ts]"]
  node_multiple["Multiple URLs<br/>input schema<br/>[scrapeMultiple.ts]"]
  node_exec["Actor Exec<br/>Apify bridge<br/>[executeActor.ts]"]
  node_helpers["Helpers<br/>shared logic"]
  node_consts["Constants<br/>shared config<br/>[consts.ts]"]
end

subgraph group_support["Support"]
  node_package["Package<br/>[package.json]"]
  node_api_cred["API Key<br/>credential"]
  node_oauth_cred["OAuth2<br/>credential"]
  node_tests["Node Tests<br/>integration tests<br/>[Apify.node.spec.ts]"]
  node_fixtures["Fixtures<br/>test data"]
  node_ci["CI<br/>github actions<br/>[ci.yml]"]
  node_publish["Publish<br/>github actions<br/>[publish.yml]"]
  node_build["Build<br/>tooling<br/>[gulpfile.js]"]
  node_lint["Lint/Format<br/>tooling<br/>[eslint.config.mjs]"]
  node_docs["README<br/>documentation<br/>[README.md]"]
end

node_apify_api[("Apify API<br/>external platform")]

node_package -->|"ships"| node_node
node_node -->|"uses"| node_meta
node_node -->|"loads"| node_resources
node_resources -->|"defines"| node_single
node_resources -->|"defines"| node_multiple
node_node -->|"uses"| node_helpers
node_helpers -->|"shares"| node_consts
node_node -->|"calls"| node_exec
node_exec -->|"invokes"| node_apify_api
node_api_cred -->|"auth"| node_node
node_oauth_cred -->|"auth"| node_node
node_tests -->|"verifies"| node_node
node_tests -->|"uses"| node_fixtures
node_ci -->|"runs"| node_tests
node_ci -->|"runs"| node_build
node_ci -->|"runs"| node_lint
node_publish -->|"releases"| node_package
node_docs -->|"explains"| node_node

click node_package "https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/package.json"
click node_node "https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/ApifyContentCrawler.node.ts"
click node_meta "https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/ApifyContentCrawler.node.json"
click node_resources "https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/resources/resources.ts"
click node_single "https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/resources/properties/scrapeSingle.ts"
click node_multiple "https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/resources/properties/scrapeMultiple.ts"
click node_exec "https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/helpers/executeActor.ts"
click node_helpers "https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/helpers/genericFunctions.ts"
click node_consts "https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/helpers/consts.ts"
click node_api_cred "https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/credentials/ApifyApi.credentials.ts"
click node_oauth_cred "https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/credentials/ApifyOAuth2Api.credentials.ts"
click node_tests "https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/__tests__/Apify.node.spec.ts"
click node_fixtures "https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/__tests__/workflows/actors/run-actor-advanced.workflow.json"
click node_ci "https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/.github/workflows/ci.yml"
click node_publish "https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/.github/workflows/publish.yml"
click node_build "https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/gulpfile.js"
click node_lint "https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/eslint.config.mjs"
click node_docs "https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/README.md"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_node,node_meta,node_resources,node_single,node_multiple,node_exec,node_helpers,node_consts toneBlue
class node_package,node_api_cred,node_oauth_cred,node_tests,node_fixtures,node_ci,node_publish,node_build,node_lint,node_docs toneAmber
class node_apify_api toneNeutral
```

## Components

### Runtime

The n8n node, its inputs, helpers, and external Apify boundary.

- [Crawler Node](https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/ApifyContentCrawler.node.ts): n8n node · Main execution entry point that runs the Apify crawler from n8n workflows. · `nodes/ApifyContentCrawler/ApifyContentCrawler.node.ts`
- [Node Meta](https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/ApifyContentCrawler.node.json): n8n manifest · Declarative node metadata used by the n8n editor and runtime. · `nodes/ApifyContentCrawler/ApifyContentCrawler.node.json`
- [Resources](https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/resources/resources.ts): n8n UI schema · Resource registry that organizes the node's editor-facing input model. · `nodes/ApifyContentCrawler/resources/resources.ts`
- [Single URL](https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/resources/properties/scrapeSingle.ts): input schema · Properties for crawling one URL at a time. · `nodes/ApifyContentCrawler/resources/properties/scrapeSingle.ts`
- [Multiple URLs](https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/resources/properties/scrapeMultiple.ts): input schema · Properties for crawling a URL set or batch. · `nodes/ApifyContentCrawler/resources/properties/scrapeMultiple.ts`
- [Actor Exec](https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/helpers/executeActor.ts): Apify bridge · Encapsulates the actor run and result retrieval against Apify. · `nodes/ApifyContentCrawler/helpers/executeActor.ts`
- [Helpers](https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/helpers/genericFunctions.ts): shared logic · Common constants and transformation functions used by the node. · `nodes/ApifyContentCrawler/helpers/genericFunctions.ts`
- [Constants](https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/helpers/consts.ts): shared config · Shared names and fixed values for the crawler integration. · `nodes/ApifyContentCrawler/helpers/consts.ts`

### Support

Credentials, tests, build, CI, and release tooling around the package.

- [Package](https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/package.json): npm package · Distributable n8n community node package for the Website Content Crawler actor. · `package.json`
- [API Key](https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/credentials/ApifyApi.credentials.ts): credential · API key credential for Apify authentication. · `credentials/ApifyApi.credentials.ts`
- [OAuth2](https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/credentials/ApifyOAuth2Api.credentials.ts): credential · OAuth2 credential for Apify authentication. · `credentials/ApifyOAuth2Api.credentials.ts`
- [Node Tests](https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/__tests__/Apify.node.spec.ts): integration tests · n8n execution tests for the crawler node and its workflow wiring. · `nodes/ApifyContentCrawler/__tests__/Apify.node.spec.ts`
- [Fixtures](https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/nodes/ApifyContentCrawler/__tests__/workflows/actors/run-actor-advanced.workflow.json): test data · Workflow JSON and helper fixtures used by the integration tests. · `nodes/ApifyContentCrawler/__tests__/workflows/actors/run-actor-advanced.workflow.json`
- [CI](https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/.github/workflows/ci.yml): github actions · Validation workflow for build, lint, and tests. · `.github/workflows/ci.yml`
- [Publish](https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/.github/workflows/publish.yml): github actions · Release workflow that publishes the package to npm. · `.github/workflows/publish.yml`
- [Build](https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/gulpfile.js): tooling · TypeScript build and package task runner. · `gulpfile.js`
- [Lint/Format](https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/eslint.config.mjs): tooling · Project linting and formatting configuration. · `eslint.config.mjs`
- [README](https://github.com/apify/n8n-nodes-apify-content-crawler/blob/master/README.md): documentation · Usage, setup, and release guidance for the package. · `README.md`

### Other components

- **Apify API**: external platform · External Apify platform used to run the Website Content Crawler actor.

## Connections

- Package → Crawler Node: ships · The npm package delivers the n8n node implementation.
- Crawler Node → Node Meta: uses · Node runtime is paired with separate editor/runtime metadata.
- Crawler Node → Resources: loads · Execution logic consumes the declarative resource model.
- Resources → Single URL: defines · Single-URL crawl mode properties are registered here.
- Resources → Multiple URLs: defines · Multiple-URL crawl mode properties are registered here.
- Crawler Node → Helpers: uses · Node logic reuses shared transformation and validation helpers.
- Helpers → Constants: shares · Shared constants support the node's internal conventions.
- Crawler Node → Actor Exec: calls · The node delegates the Apify run and result collection to the executor.
- Actor Exec → Apify API: invokes · Executor sends the crawl request to the external Apify platform.
- API Key → Crawler Node: auth · API key credentials are consumed by the node when configured.
- OAuth2 → Crawler Node: auth · OAuth2 credentials are consumed by the node when configured.
- Node Tests → Crawler Node: verifies · Integration tests exercise the node through n8n workflow execution.
- Node Tests → Fixtures: uses · Workflow fixtures provide realistic execution scenarios.
- CI → Node Tests: runs · CI validates the node through automated test execution.
- CI → Build: runs · CI includes build validation for the distributable package.
- CI → Lint/Format: runs · CI applies linting and formatting checks.
- Publish → Package: releases · Release workflow publishes the versioned npm package.
- README → Crawler Node: explains · Documentation describes how to configure and run the node.
