How parallel-web/parallel-sdk-python works
Project Type and Purpose
• This is a Python SDK (“parallel-web”) that provides both synchronous and asynchronous clients for accessing the Parallel REST API.
• It’s a library/tool, not a standalone web application. Users import Parallel or AsyncParallel, call its methods, and under the hood HTTP requests are made via httpx, responses are parsed into typed Pydantic models.
File‐Structure Analysis
• Top‐Level / Project Configuration
• Packaging & metadata: pyproject.toml, README.md, LICENSE, CHANGELOG.md, api.md
• CI/CD: .github/workflows (linting, tests, publish to PyPI)
• Dev containers & IDE: .devcontainer/, .vscode/
• Scripts & automation: bin/, scripts/ (bootstrap, lint, test, release helpers)
• Source Code Layout (src/parallel/)
• _base_client.py, _client.py – core sync/async client implementations
• _resource.py – generic resource handling (e.g., task_run, task_group)
• _response.py – APIResponse & AsyncAPIResponse abstractions
• _models.py & types/ – Pydantic definitions for all request params and responses
• _utils/ – helpers (logging, datetime parsing, streaming, reflection)
• resources/ & types/ – namespaced API endpoints (stable vs beta)
• Tests (tests/) mirror resource modules to validate client behavior, request-building, response parsing, streaming, retries, timeouts
README Insights
• Emphasizes both sync and async clients powered by httpx
• Strong typing via Pydantic
• Support for raw/streaming responses, retries, timeouts, logging
• Extensibility: undocumented endpoints via low-level client methods
Main Components & Interactions (
A. User Layer
• Your Python code imports Parallel or AsyncParallel → initiates Client instance
B. Client Layer
• Sync/Async client classes (_client.py)
• Configuration (api_key, base_url, max_retries, timeout)
C. HTTP Layer
• httpx.Client / httpx.AsyncClient (DefaultHttpxClient)
• Retry & backoff logic (configured in client)
D. APIResource Layer
• Resource proxy objects (client.task_run, client.beta.task_group, etc.)
• Methods: create(), result(), list(), events()
E. Serialization Layer
• Body serialization: Pydantic models from types/
• Querystring builder (_qs.py)
• Header injection (API key, beta flags)
F. Network Boundary
• Outgoing HTTP requests to Parallel REST API
G. Deserialization Layer
• _response.py: wrap raw httpx.Response, parse JSON
• JSON → Pydantic response models (_models.py, types/)
• Optional streaming or raw access
H. Utility Layer
• Logging, compatibility shims, datetime parsing, reflection, transforms
Architectural Patterns and Principles
• Layered / Onion architecture: User → Client → HTTP → APIResource → Models
• Adapter pattern: httpx adapted into a generic API client interface
• Factory/Proxy: resource objects proxy calls to client methods
• Separation of concerns:
• Transport (httpx) vs serialization (Pydantic) vs business API (task_run, task_group)
• Sync vs Async share most code via base classes and compatibility modules
Technologies and Frameworks
• httpx (sync & async HTTP transport)
• Pydantic (typed request/response models)
• Stainless (code generation)
• nox (test automation)
• GitHub Actions (CI/CD)
• Docker dev container
Diagramming Recommendations
• From User → Client → HTTP → API → HTTP response → Deserializer → User
• Side‐flows for retry/backoff, streaming, raw responses
• Blue for user‐facing components (Parallel, AsyncParallel)
• Green for transport (httpx layer)
• Yellow for serialization (Pydantic models in types/)
• Gray for utilities (logging, compat)
• Clear labels: methods (create, result), data flows (JSON request/response), configuration knobs (timeout, retries)