# mend3/puppeteer-di-solid-ts architecture

> Architecture diagram of the GitHub repository mend3/puppeteer-di-solid-ts, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mend3/puppeteer-di-solid-ts
- Repository: https://github.com/mend3/puppeteer-di-solid-ts

## Overview

##### Project type and purpose
- This is a specialized command-line web crawler built in TypeScript using Puppeteer.
- It follows SOLID design principles to remain modular, testable, and maintainable.
- It supports both local browser launches and remote browser connections (via a Docker-based browserless service).

##### File structure
- Top-level files:
- package.json / package-lock.json — Node.js project metadata and dependencies
- docker-compose.yaml — orchestration for a remote browser service (browserless)
- tsconfig.json — TypeScript compiler options
- .gitignore, LICENSE, ABOUT.md, README.md — documentation and config
- src/puppeteer.ts — the single entry point for the crawler’s logic
- The minimal directory layout and reliance on SOLID suggests the code is organized in distinct classes/modules inside puppeteer.ts (or imported there) for launching browsers, navigating pages, extracting data, etc.

##### README for additional insights
- Configuration via environment variables for REMOTE_BROWSER and EXECUTABLE_PATH
- npm scripts:
- start — run with local Chrome
- start:remote — pull up browserless via Docker compose and connect
- build — generate docs and produce a dist folder
- Core technologies: Puppeteer, TypeScript, Docker

##### Main components
a. Internal Components
- Crawler Engine (core orchestrator)
- Browser Launcher Module
– Local Launcher (executes /usr/bin/google-chrome or provided path)
– Remote Launcher (connects to browserless host:port)
- Page Processor (navigation, DOM evaluation, data extraction)
- Configuration Manager (reads env vars, sets exec path or remote host)
- Build & Docs Generator (npm build step, TypeDoc output)
b. External Services
- Local Chrome Browser
- Browserless Docker Service (defined in docker-compose.yaml)
- File System / Dist folder (output artifacts, logs, docs)
c. Relationships and data flows
- Configuration Manager feeds settings into Browser Launcher
- Crawler Engine invokes Browser Launcher → obtains Puppeteer Browser instance
- Browser instance passed to Page Processor → extracts data → writes to FS or forwards to consumer
- Build step is separate—reads src, generates docs, outputs to dist

##### Important architectural patterns
- SOLID modularity: separate classes for each responsibility
- Dependency injection in TypeScript: inject Launcher into Engine, inject Config into Launcher
- Strategy pattern for Local vs. Remote launcher
- Single entry point (puppeteer.ts) that wires everything together

##### Detailed element checklist
- Components clearly labeled (e.g., “LocalLauncher,” “RemoteLauncher,” “CrawlerEngine”)
- Data flows annotated (e.g., “Page HTML,” “Config settings”)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Internal Modules
    subgraph "Internal Modules"
        direction TB
        CrawlerEngine["CrawlerEngine"]:::core
        ConfigManager["ConfigurationManager"]:::util
        LocalLauncher["LocalLauncher"]:::core
        RemoteLauncher["RemoteLauncher"]:::core
        PageProcessor["PageProcessor"]:::core
    end

    %% External Services
    subgraph "External Services"
        direction TB
        ChromeBrowser["Local Chrome Browser"]:::external
        BrowserlessService["Browserless Docker Service"]:::external
    end

    %% Utilities
    subgraph "Utilities & Tools"
        direction TB
        BuildDocsGenerator["Build & Docs Generator"]:::util
        ProjectMetadata["Project Metadata & Dependencies"]:::util
        DocumentationConfig["Documentation & Config"]:::util
    end

    %% Relationships
    ConfigManager -->|"provides Config settings"| LocalLauncher
    ConfigManager -->|"provides Config settings"| RemoteLauncher

    CrawlerEngine -->|"uses Launcher"| LocalLauncher
    CrawlerEngine -->|"uses Launcher"| RemoteLauncher
    LocalLauncher -->|"launch browser"| ChromeBrowser
    RemoteLauncher -->|"connect"| BrowserlessService

    CrawlerEngine -->|"passes Browser instance"| PageProcessor
    PageProcessor -->|"extracts Page HTML"| FileSystem[(File System / Dist Folder)]:::external

    ProjectMetadata -->|"defines build scripts"| BuildDocsGenerator
    BuildDocsGenerator -->|"generates docs & dist"| FileSystem

    %% Click Events
    click CrawlerEngine "https://github.com/mend3/puppeteer-di-solid-ts/blob/main/src/puppeteer.ts"
    click ConfigManager "https://github.com/mend3/puppeteer-di-solid-ts/blob/main/src/puppeteer.ts"
    click LocalLauncher "https://github.com/mend3/puppeteer-di-solid-ts/blob/main/src/puppeteer.ts"
    click RemoteLauncher "https://github.com/mend3/puppeteer-di-solid-ts/blob/main/src/puppeteer.ts"
    click PageProcessor "https://github.com/mend3/puppeteer-di-solid-ts/blob/main/src/puppeteer.ts"
    click BrowserlessService "https://github.com/mend3/puppeteer-di-solid-ts/blob/main/docker-compose.yaml"
    click BuildDocsGenerator "https://github.com/mend3/puppeteer-di-solid-ts/blob/main/package.json"
    click ProjectMetadata "https://github.com/mend3/puppeteer-di-solid-ts/blob/main/package.json"
    click DocumentationConfig "https://github.com/mend3/puppeteer-di-solid-ts/blob/main/README.md"

    %% Styles
    classDef core fill:#D0E8FF,stroke:#0366D6,stroke-width:1px
    classDef external fill:#E6FFEA,stroke:#28A745,stroke-width:1px
    classDef util fill:#F2F2F2,stroke:#666666,stroke-width:1px
```
