How firecrawl/firecrawl-mcp-server works
1. What this repo is
- A TypeScript Node.js MCP server that exposes Firecrawl web-retrieval capabilities to MCP clients.
- It is packaged for both stdio execution (npx firecrawl-mcp) and an optional streamable HTTP local mode (/mcp).
2. Primary runtime boundary
- The application entrypoint is src/index.ts.
- The server sits between an MCP client (Claude Desktop, Cursor, VS Code, Windsurf, etc.) and the Firecrawl backend/API.
- The codebase is small and focused: there are no separate frontend or database subsystems.
3. MCP tool layer
- The main architecture is a tool registry: each MCP tool maps to a Firecrawl capability.
- Core tool groups are:
- single-page retrieval: scrape
- multi-page retrieval: batch_scrape, crawl
- discovery/search: map, search
- structured extraction: extract
- autonomous research: agent
- browser interaction: browser_* legacy/deprecated tools
4. Data flow for synchronous operations
- For direct requests like scrape or search, the client sends a tool call with parameters.
- The server forwards that request to Firecrawl, transforms the response into MCP-compatible content, and returns it immediately.
- This is the simplest request/response path and is the default for most usage.
5. Asynchronous job flow
- Some operations are intentionally async because they can take a long time or produce large outputs:
- batch_scrape
- crawl
- agent
- some browser/session flows
- These tools return a job ID / operation ID first.
- Companion status tools (check_batch_status, check_crawl_status, agent_status) poll Firecrawl until the job completes and then return the final payload.
6. External service dependency
- The server’s core dependency is Firecrawl Cloud or a self-hosted Firecrawl instance.
- FIRECRAWL_API_KEY is required for cloud usage; FIRECRAWL_API_URL switches the server to a custom/self-hosted endpoint.
- The repo itself does not store content; Firecrawl is the execution engine for web access, crawling, extraction, and browser automation.
7. Transport and client integration
- The server is designed to be embedded in external developer tools as an MCP provider.
- Default transport is local process I/O; a local HTTP mode is available for environments that prefer a URL-based MCP endpoint.
- The README and smithery.yaml indicate multiple installer/integration targets, but they all consume the same server process.
8. Reliability and throttling
- The server includes retry/backoff configuration and rate-limit handling for Firecrawl API calls.
- Batch operations are designed with queueing/parallelism in mind, while the server limits retries and spacing via environment variables.
- This matters architecturally because the server is not a thin proxy; it actively manages operational resilience.
9. Monitoring and operational feedback
- The server emits logs for operation lifecycle, retries, and credit usage thresholds.
- Credit monitoring is a first-class concern in the architecture because Firecrawl usage is metered and long-running jobs can consume credits quickly.
- Errors are surfaced back through MCP responses as structured failures, not just thrown exceptions.
10. Legacy browser subsystem
- The browser_* tools are explicitly marked deprecated.
- They represent an older cloud-browser/session model and are retained for compatibility, but the preferred path is now scrape plus interact.
- In an architecture diagram, this should be shown as a legacy branch, not the mainline flow.
11. Repository packaging and delivery
- The repo is built for distribution via npm and container images.
- Dockerfile, Dockerfile.service, and the docker/ assets show that the server can be deployed as a containerized service as well as a CLI package.
- GitHub Actions workflows handle CI, image builds, staging, publishing, and registry updates.
12. Code organization signals
- src/types/fastmcp.d.ts suggests local type augmentation for the MCP runtime/library.
- src/legacy/index.md indicates older documentation/behavior kept alongside the current server.
- Tests and build tooling (jest, TypeScript, ESLint, Prettier) support a single-codebase service rather than a multi-package platform.
Main entry points: index.ts Server entry · ci.yml CI/CD · Dockerfile Containers · index.md Browser tools
Components 17
Runtime
MCP server execution path and tool handling
- Server entry · Application bootstrap and runtime wiring.
src/index.ts - Tool registry · Maps MCP tool names to Firecrawl capabilities.
- Sync tools · Scrape, search, map, and extract paths that return immediately.
- Async jobs · Batch scrape, crawl, agent, and long-running browser/session flows.
- Status checks · check_batch_status, check_crawl_status, and agent_status.
- Response mapper · Transforms Firecrawl payloads into MCP-compatible content and errors.
Integration
Client transports and external Firecrawl dependency
- MCP client · Claude Desktop, Cursor, VS Code, Windsurf, etc.
- Stdio mode · Default local process I/O server mode.
- HTTP mode · Optional streamable local HTTP endpoint.
- Firecrawl API · Cloud or self-hosted Firecrawl instance used for retrieval and automation.
Ops
Resilience, logging, and credit monitoring
- Retries & limits · Backoff, retry spacing, and rate-limit handling for Firecrawl calls.
- Credit monitor · Logs usage thresholds and metered-credit warnings.
Delivery
Packaging, images, and CI/CD
- CI/CD · Workflows for testing, images, staging, publishing, and registry updates.
.github/workflows/ci.yml - Containers · Dockerfile, service image, and runtime container assets.
Dockerfile - NPM package · Distributed as an installable CLI/server package.
package.json - Build tools · TypeScript, Jest, ESLint, and Prettier support for the single codebase.
tsconfig.json
Legacy
Deprecated browser tooling kept for compatibility
- Browser tools · Deprecated browser/session compatibility layer.
src/legacy/index.md
Connections 23
- MCP client → Stdio mode: calls · Client launches or attaches to the default stdio server.
- MCP client → HTTP mode: connects · Clients can also use the optional local HTTP endpoint.
- Stdio mode → Server entry: starts · stdio transport boots the server process.
- HTTP mode → Server entry: starts · HTTP transport uses the same server entrypoint.
- Server entry → Tool registry: registers · Bootstraps MCP tools and server behavior.
- Tool registry → Sync tools: routes · Direct tool calls go through the synchronous path.
- Tool registry → Async jobs: routes · Long-running operations are submitted as jobs.
- Tool registry → Browser tools: includes · Deprecated browser tools remain available for compatibility.
- Sync tools → Response mapper: formats · Maps Firecrawl responses into MCP content.
- Async jobs → Status checks: hands off · Job-based tools expose companion polling/status operations.
- Status checks → Response mapper: formats · Poll results are converted to final MCP responses.
- Sync tools → Firecrawl API: requests · Synchronous retrieval and extraction calls reach Firecrawl.
- Async jobs → Firecrawl API: submits · Async jobs are created in Firecrawl.
- Status checks → Firecrawl API: polls · Status tools query Firecrawl until completion.
- Browser tools → Firecrawl API: drives · Legacy browser/session flows still depend on Firecrawl.
- Server entry → Retries & limits: uses · Operational retry and rate-limit behavior wraps API calls.
- Retries & limits → Firecrawl API: protects · Retries, backoff, and spacing smooth API interaction.
- Server entry → Credit monitor: logs · Emits credit and lifecycle signals.
- Credit monitor → Firecrawl API: observes · Tracks metered usage implications of Firecrawl operations.
- CI/CD → Containers: builds · Workflows produce service and image artifacts.
- CI/CD → NPM package: publishes · Workflows handle package release and registry updates.
- Build tools → Server entry: supports · TypeScript and test/lint tooling support the server code.
- Containers → Server entry: packages · Container images bundle the server runtime.