# livingflame/bigpipe architecture

> Architecture diagram of the GitHub repository livingflame/bigpipe, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/livingflame/bigpipe
- Repository: https://github.com/livingflame/bigpipe

## Overview

This repo is a **PHP implementation/demo of the “BigPipe” pattern** (popularized by Facebook): the server **decomposes a page into “pagelets”**, streams an initial HTML “shell” quickly, then **pipelines pagelet rendering and delivery** to the browser, where a **client-side BigPipe runtime** inserts pagelet HTML and loads required JS/CSS dependencies.

Key evidence:
- `BigPipe.php`, `Pagelet.php`: core server-side BigPipe + pagelet abstractions
- `DependencySort.php`: indicates dependency ordering/toposort (likely for resources or pagelets)
- `ResourceDataCache.php` + `registered_resource/*.json`: resource registry + caching of resource metadata
- `static/js/BigPipe.js`: browser-side runtime that receives pagelet payloads and injects them into the DOM
- `example.php`: entrypoint/demo page wiring everything together
- `static/css/*`, `static/js/*`: served assets and “delayed” JS/CSS examples

##### Server-side (PHP)
- **Application entrypoint / demo**
- `example.php` (request handler that constructs pagelets, registers resources, triggers streaming)
- **BigPipe orchestrator**
- `BigPipe.php` (main controller: manages pagelets, emits initial HTML, streams pagelet payloads)
- **Pagelet model**
- `Pagelet.php` (represents a chunk: id/container, render callback/template, resource deps)
- **Dependency / ordering**
- `DependencySort.php` (sorts dependencies—commonly used to ensure correct JS/CSS load order)
- **Resource metadata + caching**
- `ResourceDataCache.php` (loads/parses resource registry JSON and caches it)
- `registered_resource/*.json` (resource definitions; likely include type=css/js, uri, deps)

##### Client-side (Browser)
- **BigPipe runtime**
- `static/js/BigPipe.js` (receives streamed pagelet data, inserts HTML, triggers resource loads)
- **3rd-party lib**
- `static/js/jquery-3.1.0.js`
- **Static assets**
- `static/css/*.php` and `static/js/delayJS*.php` (PHP endpoints that output CSS/JS; used to simulate delay)
- `bigPipe.css`, `static/css/test.css` etc.

Shows the runtime blocks: Browser, Web Server (PHP), BigPipe library, Resource Registry, Static asset endpoints.

Shows *streaming + pipelining*: initial HTML flush, then multiple pagelet payloads delivered over time, browser inserts them, loads dependencies.

##### *(Optional but valuable)* Dependency graph / flow
Shows how `registered_resource/*.json` and `DependencySort.php` govern final resource load order.

##### Browser BigPipe.js → DOM: insert pagelet HTML into container(s)
8. **Browser BigPipe.js → Resource loader**: request needed CSS/JS (from `static/...` endpoints or files), respecting dependencies

##### Resource-specific interactions
- **BigPipe** decides which resources a pagelet needs (based on JSON registry + pagelet declaration).
- **Browser** loads missing resources; subsequent pagelets can reuse already-loaded resources (resource caching / dedupe concept—worth drawing as a note).

##### Architectural patterns/design principles to highlight (specific to this project)
- **Streaming HTML response**: server flushes partial output (shell first, pagelets later).
- **Page composition via “pagelets”**: independent render units.
- **Client-side incremental hydration**: browser assembles final page progressively.
- **Resource dependency management**: topological sorting to enforce CSS/JS order (`DependencySort.php`).
- **Resource registry abstraction**: resources described declaratively (`registered_resource/*.json`) rather than hardcoded.
- **Latency simulation endpoints**: `static/js/delayJS*.php`, `static/css/*.php` suggest intentional delayed loading for demo/testing.

##### BigPipe streams
- HTML `<head>` + base CSS (e.g., `bigPipe.css`)
- `<script src="static/js/BigPipe.js">`
- placeholders `

`
- flush
##### For each pagelet
- server renders pagelet HTML
- server streams a script/payload “pagelet arrived”
##### Browser BigPipe.js
- inserts HTML into placeholder
- requests missing CSS/JS (maybe delayed endpoints)

##### Dependency visualization (optional)
- Nodes: `jquery-js`, `blue-style-css`, `delayJSBlue-js`, etc. from `registered_resource/*.json`
- Edges: “depends on”
- “Sorted by DependencySort.php”

## Diagram (Mermaid)

```mermaid
flowchart TD
  %% =========================
  %% External Actor
  %% =========================
  U["Browser (User Agent)"]:::external

  %% =========================
  %% Client-side (Browser)
  %% =========================
  subgraph "Browser (Client Runtime)"
    direction TB
    DOM["HTML Parser + DOM"]:::client
    BPJS["BigPipe Client Runtime (BigPipe.js)"]:::client
    RL["Resource Loader (CSS/JS Injection + Cache/Dedupe)"]:::client
    NOTE1["Note: Inserts pagelet HTML into placeholders (e.g., <div id='pagelet_X'>)"]:::note
  end

  %% =========================
  %% Server-side (PHP)
  %% =========================
  subgraph "Web Server / PHP Runtime"
    direction TB
    EX["Application Entrypoint (example.php)"]:::server
    BPS["BigPipe Orchestrator (stream shell + pagelets)"]:::server
    PGL["Pagelet Abstraction (id/container + render + deps)"]:::server
    RDC["ResourceDataCache (load + cache resource metadata)"]:::server
    DS["DependencySort (toposort resources/pagelets)"]:::server
    NOTE2["Note: Streaming HTML response (shell first, then pagelet payload chunks)"]:::note
  end

  %% =========================
  %% Resource registry (metadata)
  %% =========================
  subgraph "Resource Registry (Metadata)"
    direction TB
    REGDIR["registered_resource/ (resource definitions)"]:::meta
    REGJSON1["jquery-js.json, *-style-css.json, delayJS*-js.json ..."]:::metaDoc
  end

  %% =========================
  %% Static delivery (assets + simulated latency endpoints)
  %% =========================
  subgraph "Static Asset Delivery"
    direction TB

    subgraph "JavaScript"
      direction TB
      AS_BPJS["static/js/BigPipe.js"]:::asset
      AS_JQ["static/js/jquery-3.1.0.js"]:::asset
      AS_DLY_B["static/js/delayJSBlue.php"]:::asset
      AS_DLY_G["static/js/delayJSGreen.php"]:::asset
      AS_DLY_R["static/js/delayJSRed.php"]:::asset
    end

    subgraph "CSS"
      direction TB
      AS_BASE_CSS["bigPipe.css"]:::asset
      AS_TEST_CSS["static/css/test.css"]:::asset
      AS_BLUE["static/css/blue.php"]:::asset
      AS_GREEN["static/css/green.php"]:::asset
      AS_RED["static/css/red.php"]:::asset
    end
  end

  %% =========================
  %% Core runtime flow
  %% =========================
  U -->|"navigates"| DOM
  DOM -->|"GET /example.php"| EX

  EX -.->|"constructs+registers"| BPS
  BPS -.->|"creates/uses"| PGL
  BPS -.->|"loads registry"| RDC
  RDC -.->|"reads"| REGDIR
  REGDIR -->|"contains"| REGJSON1
  BPS -.->|"sort deps"| DS

  %% Streaming response: shell then pagelets (same response stream)
  BPS ==>|"stream: shell HTML + placeholders + <script src=BigPipe.js> (flush)"| DOM
  BPS ==>|"stream: pagelet payload chunks (script callbacks / data)"| BPJS

  %% Client incremental assembly
  BPJS -->|"insert pagelet HTML"| DOM
  BPJS -->|"request missing deps (respect order)"| RL

  %% Asset requests (separate from streamed pagelets)
  RL -->|"GET BigPipe runtime"| AS_BPJS
  RL -->|"GET base CSS"| AS_BASE_CSS
  RL -->|"GET shared lib"| AS_JQ
  RL -->|"GET CSS (may be delayed)"| AS_BLUE
  RL -->|"GET CSS (may be delayed)"| AS_GREEN
  RL -->|"GET CSS (may be delayed)"| AS_RED
  RL -->|"GET extra CSS"| AS_TEST_CSS
  RL -->|"GET delayed JS (demo)"| AS_DLY_B
  RL -->|"GET delayed JS (demo)"| AS_DLY_G
  RL -->|"GET delayed JS (demo)"| AS_DLY_R

  %% Resource decisions guided by registry + sorting (conceptual)
  RDC -.->|"resource metadata"| BPS
  DS -.->|"ordered deps"| BPS
  REGDIR -.->|"declares deps/types/uris"| RDC

  %% Notes wiring
  NOTE2 -.->|"applies to"| BPS
  NOTE1 -.->|"explains"| BPJS

  %% =========================
  %% Click events (mapped components)
  %% =========================
  click EX "https://github.com/livingflame/bigpipe/blob/master/example.php"
  click BPS "https://github.com/livingflame/bigpipe/blob/master/BigPipe.php"
  click PGL "https://github.com/livingflame/bigpipe/blob/master/Pagelet.php"
  click DS "https://github.com/livingflame/bigpipe/blob/master/DependencySort.php"
  click RDC "https://github.com/livingflame/bigpipe/blob/master/ResourceDataCache.php"
  click REGDIR "https://github.com/livingflame/bigpipe/tree/master/registered_resource/"
  click AS_BPJS "https://github.com/livingflame/bigpipe/blob/master/static/js/BigPipe.js"
  click AS_JQ "https://github.com/livingflame/bigpipe/blob/master/static/js/jquery-3.1.0.js"
  click AS_BASE_CSS "https://github.com/livingflame/bigpipe/blob/master/bigPipe.css"
  click AS_BLUE "https://github.com/livingflame/bigpipe/blob/master/static/css/blue.php"
  click AS_GREEN "https://github.com/livingflame/bigpipe/blob/master/static/css/green.php"
  click AS_RED "https://github.com/livingflame/bigpipe/blob/master/static/css/red.php"
  click AS_TEST_CSS "https://github.com/livingflame/bigpipe/blob/master/static/css/test.css"
  click AS_DLY_B "https://github.com/livingflame/bigpipe/blob/master/static/js/delayJSBlue.php"
  click AS_DLY_G "https://github.com/livingflame/bigpipe/blob/master/static/js/delayJSGreen.php"
  click AS_DLY_R "https://github.com/livingflame/bigpipe/blob/master/static/js/delayJSRed.php"

  %% =========================
  %% Styles
  %% =========================
  classDef external fill:#F5F5F5,stroke:#616161,stroke-width:1.5px,color:#111
  classDef client fill:#E8F5E9,stroke:#2E7D32,stroke-width:1.5px,color:#0B2E13
  classDef server fill:#E3F2FD,stroke:#1565C0,stroke-width:1.5px,color:#0A2A4A
  classDef asset fill:#FFF3E0,stroke:#EF6C00,stroke-width:1.5px,color:#4A2A00
  classDef meta fill:#ECEFF1,stroke:#546E7A,stroke-width:1.5px,color:#102027
  classDef metaDoc fill:#F3E5F5,stroke:#6A1B9A,stroke-width:1.5px,color:#2A0A3D
  classDef note fill:#FFFDE7,stroke:#9E9D24,stroke-width:1.5px,color:#2E2E00,stroke-dasharray: 4 3
```
