How carr1005/lighthouse-batch-parallel works
Project type and purpose
• This is a Node.js‑based library + CLI tool (“lighthouse-batch-parallel”) for running Google Lighthouse audits on a batch of URLs in parallel.
• It can be required as a module in other Node.js apps or invoked as a standalone command‑line utility.
File structure
• Top‑level files and directories:
• index.js – the core module entry point exporting lighthouseBatchParallel
• worker.js – implements a single audit worker (spawns headless Chrome, runs Lighthouse)
• lighthouse-batch-parallel.js – CLI wrapper invoking the core module
• utils.js – shared utilities (parsing CSV/JSON, formatting output, event wiring)
• assets/ – default and example input/custom‑audit configurations
• package.json, .eslintrc.json, README.md, LICENSE, .gitignore
• Patterns:
• Worker Pool / Producer‑Consumer for parallel audits
• Event‑driven (EventEmitter) for streaming results and progress
• Modular separation: CLI vs core API vs worker implementation
• Input/Output abstraction supporting streams, files, and various formats (CSV, JSON, JS objects)
README for additional insights
• Core API: lighthouseBatchParallel(parameters) → EventEmitter
• Configurable params: input source, custom audits, throttling, output format, number of workers
• Events emitted: ‘data’, ‘error’, ‘end’
• CLI options mirror core parameters and handle file I/O
Main components
a. Front‑end / Invocation layers
• CLI Script (lighthouse-batch-parallel.js)
• Consumer code calling require('lighthouse-batch-parallel')
b. Core Orchestrator
• index.js – parses args/params, loads utils, initializes task queue and worker pool, wires up EventEmitter
c. Worker Pool
• worker.js – each process/thread:
– launches headless Chrome instance via Lighthouse Node API
– runs audit for one URL
– returns audit report back to orchestrator
d. Utilities
• utils.js – file/stream parsing (CSV/JSON), output formatting, progress calculation
• assets/defaultConfig.js – default audit configuration
e. External dependencies
• Lighthouse package (runs audits under the hood)
• Headless Chrome (controlled via DevTools Protocol)
• Node.js EventEmitter, child_process or worker_threads (depending on implementation)
f. Input and Output flows
• Input sources:
– File (CSV/JSON) → parsed to stream of URL tasks
– In‑memory JS array or CSV string stream
• Output sinks:
– Stream/event callbacks (data, error, end)
– Files (CSV/JSON) when used via CLI
Relationships and interactions
• The CLI or consumer invokes the orchestrator (index.js).
• The orchestrator:
1. Parses inputs and custom‑audit configs via utils.
2. Enqueues tasks into a central queue.
3. Spawns N worker instances (workersNum) in parallel.
4. Distributes tasks to workers (round‑robin or queue pull).
5. Listens for worker responses (audit report or error).
6. Emits ‘data’ or ‘error’ events back to caller.
7. On completion of all tasks, emits ‘end’.
• Each worker:
• Receives a task (URL + device + audit config)
• Launches a headless Chrome via Lighthouse API
• Runs audit, returns results (scores, metrics)
• Utils handle serialization and formatting at each stage.
Architectural patterns and design principles
• Producer‑Consumer (Orchestrator produces tasks, workers consume)
• Event‑Driven: decoupling of core logic from consumers via EventEmitter
• Strategy / Plugin: custom audit configuration loaded at runtime
• Single Responsibility:
• CLI wrapper (arg parsing, file I/O)
• Core orchestrator (parallelization, eventing)
• Worker (single audit execution)
• Utils (parsing/formatting)
Relevant technologies and libraries
• Node.js (v10+)
• Google Chrome Lighthouse (Node module)
• child_process or worker_threads (to parallelize)
• CSV / JSON parsing libraries
• EventEmitter (built‑in)
• npm scripts (for packaging)
• ESLint (code quality)