How encode/httpcore works
Project type and purpose
• HTTP Core is a low-level, language-native HTTP client library (an open-source tool), providing only request/response mechanics, connection pooling, proxy support, and HTTP/1.1 & HTTP/2 implementations. It exposes both synchronous and asynchronous APIs to serve as a foundational building block for higher-level HTTP clients.
File structure
• Top-level directories:
• httpcore/ ← main package
• docs/ ← user documentation
• scripts/ ← helper scripts (build, test, lint, etc.)
• tests/ ← test suites (sync, async, benchmark)
• Inside httpcore/:
• _api.py ← public-facing convenience functions
• _async/ ← async interfaces & implementations
• _sync/ ← sync interfaces & implementations
• _backends/ ← pluggable network-backend abstractions
• _models.py ← shared data structures (Request, Response)
• _exceptions.py ← error definitions
• _ssl.py, _utils.py ← supporting utilities
• _synchronization.py ← concurrency primitives
• _trace.py ← instrumentation/tracing hooks
• Configuration files:
• pyproject.toml, requirements.txt, mkdocs.yml
• .github/workflows/*.yml
README for additional insights
• Emphasizes the split between “networking code” and “client logic.”
• Supports optional extras: anyio, trio, h2, socksio.
• Targets Python 3.8+, uses SEMVER, intended as a backend for higher‐level clients (e.g., HTTPX).
1) Public API Layer
– httpcore.request() and ConnectionPool constructors
2) Interface Layer
– Sync Interface (httpcore._sync)
– Async Interface (httpcore._async)
3) Connection Pool
– Manages connection lifecycles, pools per origin
4) Protocol Layer
– HTTP/1.1 handler (h11)
– HTTP/2 handler (h2)
5) Proxy Layer
– HTTP proxy, SOCKS proxy implementations
6) Backend Abstraction
– anyio, asyncio, trio, sync backends
7) External Dependencies
– h11, h2, socksio, certifi
8) Network I/O
– OS sockets, TLS handshake
9) Remote Services
– Internet host, optional Proxy servers
– Client code → Public API Layer → selects Sync vs Async → Connection Pool
– Connection Pool → Protocol Layer → Proxy Layer (if configured) → Backend
– Backend → OS socket calls → Remote Service (HTTP server or Proxy)
– Responses flow back along the same path in reverse
c. Architectural patterns
– Layered pattern: clear separation of concerns (API, pooling, protocol, transport)
– Adapter pattern: pluggable backends (asyncio/trio/anyio/sync)
– Factory/injection: optional extras loaded dynamically for HTTP2 or SOCKS
d. Relevant technologies & frameworks
– Python’s async/await, anyio, asyncio, trio
– h11 for HTTP/1.1
– h2 for HTTP/2
– socksio for SOCKS proxy
– certifi for TLS trust store
Essential elements
• Component labels matching actual package paths (e.g., httpcore._async.connection_pool).
• Notes on optional paths (HTTP2 vs HTTP1.1, proxy vs direct).