How jpmorganchase/py-adapter works
Project Type and Purpose
• This is an open-source Python library (packaged on PyPI) whose purpose is “round-trip serialization/deserialization of any Python object to/from any serialization format (Avro, JSON, CSV, etc.)”.
• Core feature: a plugin architecture under src/py_adapter/plugin/ that allows adding new formats without touching core logic.
File Structure
Top-level layout:
• src/py_adapter/ – the library code
– _schema.py – internal schema handling
– plugin/ – built-in adapters: _avro.py, _json.py, _csv.py
– __init__.py, py.typed
• tests/ – pytest test suites and sample Avro schemas (.avsc)
• docs/ – Sphinx documentation
• Configuration files: pyproject.toml, tox.ini, GitHub Actions workflows
• CI/CD & packaging under .github/workflows/
README
• Confirms Python packaging (pip install py-adapter)
• Emphasizes plugin extensibility (“any serialization format”)
• Points to developer workflow with Tox and pre-commit
Main Components
a. Core Adapter Engine (py_adapter)
b. Plugin Interface / Registry – the mechanism by which plugins register themselves with the core
c. Built-in Plugins
• Avro Adapter (_avro.py)
• JSON Adapter (_json.py)
• CSV Adapter (_csv.py)
d. External Format Libraries
• Apache Avro library (for schema parsing, binary encoding)
• Python json stdlib
• Python csv stdlib
e. User Code – Python objects being serialized/deserialized
g. Documentation System (Sphinx)
Relationships & Data Flows
• User passes a Python object to Core Adapter Engine.
• Core Engine invokes Plugin Registry to select target format plugin.
• Plugin calls into respective External Format Library to perform conversion to bytes/text.
• Deserialization flows in reverse order.
• Tests exercise the Core → Plugin → Library chain; docs generate from code and doc sources.
Architectural Patterns & Principles
• Plugin Pattern: loosely-coupled adapters, discoverable via a registry.
• Separation of Concerns: core orchestration vs. format-specific logic.
• Single Responsibility: each plugin handles one format only.
• Open/Closed Principle: new formats can be added without modifying core code.
• Annotations: point out critical files (e.g., _schema.py for schema introspection).