How livingflame/bigpipe works
This repo is a PHP implementation/demo of the “BigPipe” pattern (popularized by Facebook): the server decomposes a page into “pagelets”, streams an initial HTML “shell” quickly, then pipelines pagelet rendering and delivery to the browser, where a client-side BigPipe runtime inserts pagelet HTML and loads required JS/CSS dependencies.
Key evidence:
• BigPipe.php, Pagelet.php: core server-side BigPipe + pagelet abstractions
• DependencySort.php: indicates dependency ordering/toposort (likely for resources or pagelets)
• ResourceDataCache.php + registered_resource/*.json: resource registry + caching of resource metadata
• static/js/BigPipe.js: browser-side runtime that receives pagelet payloads and injects them into the DOM
• example.php: entrypoint/demo page wiring everything together
• static/css/*, static/js/*: served assets and “delayed” JS/CSS examples
Server-side (PHP)
• Application entrypoint / demo
• example.php (request handler that constructs pagelets, registers resources, triggers streaming)
• BigPipe orchestrator
• BigPipe.php (main controller: manages pagelets, emits initial HTML, streams pagelet payloads)
• Pagelet model
• Pagelet.php (represents a chunk: id/container, render callback/template, resource deps)
• Dependency / ordering
• DependencySort.php (sorts dependencies—commonly used to ensure correct JS/CSS load order)
• Resource metadata + caching
• ResourceDataCache.php (loads/parses resource registry JSON and caches it)
• registered_resource/*.json (resource definitions; likely include type=css/js, uri, deps)
Client-side (Browser)
• BigPipe runtime
• static/js/BigPipe.js (receives streamed pagelet data, inserts HTML, triggers resource loads)
• 3rd-party lib
• static/js/jquery-3.1.0.js
• Static assets
• static/css/*.php and static/js/delayJS*.php (PHP endpoints that output CSS/JS; used to simulate delay)
• bigPipe.css, static/css/test.css etc.
Shows the runtime blocks: Browser, Web Server (PHP), BigPipe library, Resource Registry, Static asset endpoints.
Shows *streaming + pipelining*: initial HTML flush, then multiple pagelet payloads delivered over time, browser inserts them, loads dependencies.
*(Optional but valuable)* Dependency graph / flow
Shows how registered_resource/*.json and DependencySort.php govern final resource load order.
Browser BigPipe.js → DOM: insert pagelet HTML into container(s)
8. Browser BigPipe.js → Resource loader: request needed CSS/JS (from static/... endpoints or files), respecting dependencies
Resource-specific interactions
• BigPipe decides which resources a pagelet needs (based on JSON registry + pagelet declaration).
• Browser loads missing resources; subsequent pagelets can reuse already-loaded resources (resource caching / dedupe concept—worth drawing as a note).
Architectural patterns/design principles to highlight (specific to this project)
• Streaming HTML response: server flushes partial output (shell first, pagelets later).
• Page composition via “pagelets”: independent render units.
• Client-side incremental hydration: browser assembles final page progressively.
• Resource dependency management: topological sorting to enforce CSS/JS order (DependencySort.php).
• Resource registry abstraction: resources described declaratively (registered_resource/*.json) rather than hardcoded.
• Latency simulation endpoints: static/js/delayJS*.php, static/css/*.php suggest intentional delayed loading for demo/testing.
BigPipe streams
• HTML <head> + base CSS (e.g., bigPipe.css)
• <script src="static/js/BigPipe.js">
• placeholders
• flush
For each pagelet
• server renders pagelet HTML
• server streams a script/payload “pagelet arrived”
Browser BigPipe.js
• inserts HTML into placeholder
• requests missing CSS/JS (maybe delayed endpoints)
Dependency visualization (optional)
• Nodes: jquery-js, blue-style-css, delayJSBlue-js, etc. from registered_resource/*.json
• Edges: “depends on”
• “Sorted by DependencySort.php”