How facebook/proxygen works
This project is Proxygen – Facebook’s open-source C++ HTTP library that provides the building blocks for HTTP servers, proxies, and client implementations. Its purpose is to offer a modern, performant, and flexible HTTP framework supporting multiple protocols (HTTP/1.1, HTTP/2, HTTP/3, and even SPDY variants). The architecture is built around the core abstractions of session, codec, transaction, and handler. The higher-level HTTP server APIs then simplify creating complete web servers using these underlying abstractions.
Main Components
• External Dependencies and Build Infrastructure:
• The “build” directory and “build/fbcode_builder” infrastructure shows there is an elaborate dependency management system where third-party libraries (like folly, mvfst, wangle, fizz, and others) are specified under manifests.
• Core Library Components (located within proxygen/lib):
• HTTPCodec: Converts bytes from the network into higher-level HTTP messages (parsing and serialization).
• HTTPSession: Manages the connection and maps transaction identifiers to HTTPTransaction objects.
• HTTPTransaction: Represents a single request/response pair and links with a handler.
• HTTPHandler: Implements the application-specific logic and interacts with the transaction to generate egress.
• Additional supporting modules such as connection pool, session management, and utils.
• High-Level HTTP Server Framework (from proxygen/httpserver):
• HTTPServer: Wraps the lower-level abstractions to provide a simple interface to run an HTTP server.
• RequestHandlerFactory and RequestHandler: Provide user-level entry points for handling HTTP requests.
• QUIC and HTTP/3 Support:
• Integration with Facebook’s mvfst library is critical. Include a component for “mvfst/QUIC Transport” that links to the HTTP/3 functionality in proxygen.
• Testing and Documentation:
Component Relationships and Data Flow
• Data incoming from the network is first handled by a lower-level network layer (the socket API, not directly depicted in the repo but understood via asynchronous transport in proxygen/lib/transport).
• HTTPCodec (within HTTPSession) parses incoming bytes into HTTP messages, injecting a transaction identifier.
• The HTTPSession manages these parsed messages by handing them over to distinct HTTPTransaction instances.
• Each HTTPTransaction forwards calls to its associated HTTPHandler (user-defined business logic) to act on requests and responses.
• Outgoing responses traverse back from the handler to the transaction and then pass through the codec to be serialized for sending.
• For HTTP/3, the mvfst component acts as the QUIC transport provider feeding data in and out similarly, with additional handling for protocol specifics.
• The high-level HTTPServer wraps these flows, receiving a RequestHandlerFactory which spawns a RequestHandler per connection/request.
Architectural Patterns and Design Principles
• Layered Abstraction: The separation between low-level protocols (session, codec, transaction) and higher-level components (HTTPServer, RequestHandler) is crucial.
• Modularity: The clear directory separation (lib for core abstractions, httpserver for server-specific APIs, external for dependencies) shows modular design.
• Reusability and Extensibility: By encapsulating common HTTP behaviors into shared abstractions, the library ensures that both client and server use similar patterns, making it extensible.
• Asynchronous Processing: Look into modules like transport and session management which indicate asynchronous event handling.
Technologies, Frameworks, and Libraries
• C++ and CMake – the build is managed via CMakeLists.txt and fbcode_builder scripts.
• Third-party libraries: folly (Facebook’s core C++ library), mvfst (QUIC library for HTTP/3), and others as listed in the manifests.
• Testing frameworks and continuous integration workflows (.github/workflows).