How datacontract/datacontract-cli works
Purpose and Entry Points
Python package and CLI for authoring, validating, testing, comparing, importing, and exporting data contracts, with native Open Data Contract Standard (ODCS) support. Primary CLI dispatch is datacontract/cli.py; command implementations are split into datacontract/command_test.py, datacontract/command_import.py, and datacontract/command_export.py. The reusable Python API centers on datacontract/data_contract.py; an optional HTTP/API runtime is represented by datacontract/api.py.
Canonical Contract Model and Validation
Contracts are YAML/JSON documents resolved into the internal contract representation and validated against bundled versioned schemas. datacontract/data_contract.py is the orchestration boundary for loading and operating on a contract. Schema definitions and initialization templates live in datacontract/schemas/datacontract-1.2.1.schema.json and datacontract/schemas/odcs-3.2.0.schema.json. Linting resolves references, variables, resources, URLs, and schema constraints before execution through datacontract/lint/resolve.py and datacontract/lint/schema.py.
Test Execution Pipeline
test converts contract schema, service-level, and quality declarations into executable checks, runs them against a selected server, and emits a Run result. Check construction and shared semantics are concentrated in datacontract/engines/checks/create_checks.py, while orchestration is in datacontract/engines/data_contract_test.py. Output is deliberately separated from execution, supporting console, CI, JSON, and JUnit consumers via datacontract/output/ci_output.py and datacontract/output/test_results_writer.py.
Data-Source Engine Boundary
Engine adapters isolate contract-level checks from physical systems and type/connection differences. Ibis is the broad execution layer, with connection selection in datacontract/engines/ibis/connections/connect.py and query/check execution in datacontract/engines/ibis/ibis_check_execute.py. Specialized paths cover SAP HANA (datacontract/engines/hana/hana_quality_check.py), JSON Schema validation (datacontract/engines/fastjsonschema/check_jsonschema.py), and file/server preconditions under datacontract/engines/datacontract/. Supported infrastructure includes SQL warehouses/databases, DuckDB/local files, S3/GCS/Azure object storage, Iceberg, Kafka, and API endpoints; optional Python extras supply connectors.
Format Translation Layer
Importers ingest source schemas or artifacts into the canonical contract form, selected through datacontract/imports/importer_factory.py; representative adapters include datacontract/imports/sql_importer.py and datacontract/imports/bigquery_importer.py. Exporters project the canonical model to downstream formats through datacontract/export/exporter_factory.py, including SQL (datacontract/export/sql_exporter.py), JSON Schema, dbt, Avro, HTML, and Excel. This factory-based boundary is the main extensibility point for formats.
Change Governance and Integrations
Contract versions are normalized, diffed, and evaluated for backward compatibility through datacontract/changelog/changelog.py and datacontract/breaking/detector.py; CLI commands expose changelog and breaking gates for CI. dbt synchronization/test mapping is implemented under datacontract/integration/dbt_sync.py. Catalog and publish capabilities are separately surfaced by datacontract/catalog/catalog.py and datacontract/command_publish.py, keeping remote governance workflows outside core validation.
Delivery Runtimes
The package targets Python 3.10–3.12 and is distributable as a CLI/library or Docker image (Dockerfile). CI, release automation, and static documentation delivery are GitHub Actions-based (.github/workflows/ci.yaml, .github/workflows/release.yaml, .github/workflows/azure-static-web-apps-docs.yml). Documentation is a separate Docusaurus/Node runtime rooted at docs/docusaurus.config.ts, while the packaged local editor UI is static assets under datacontract/editor_assets/index.html.
Main entry points: data_contract.py DataContract API · command_test.py Test/import/export commands · data_contract_test.py Test orchestrator · ibis_check_execute.py Engine adapters
Components 22
Entry points
- CLI dispatcher
datacontract/cli.py - Test/import/export commands
datacontract/command_test.py - Optional HTTP API
datacontract/api.py
Contract core
- DataContract API · Loads and operates on canonical contracts.
datacontract/data_contract.py - Contract documents · Author-provided data contracts.
- Lint resolver
datacontract/lint/resolve.py - Schema validation
datacontract/lint/schema.py - Versioned schemas
datacontract/schemas/datacontract-1.2.1.schema.json
Test execution
- Test orchestrator · Produces a Run result from a contract and target.
datacontract/engines/data_contract_test.py - Check construction
datacontract/engines/checks/create_checks.py - Engine adapters · Isolates contract checks from source-specific connections and types.
datacontract/engines/ibis/ibis_check_execute.py - Ibis connections
datacontract/engines/ibis/connections/connect.py - Specialized validators
datacontract/engines/hana/hana_quality_check.py - Data sources · Connector extras support databases, object stores, Kafka, Iceberg, and endpoints.
- Result outputs
datacontract/output/test_results_writer.py
Translation & governance
- Importer factory
datacontract/imports/importer_factory.py - Exporter factory
datacontract/export/exporter_factory.py - Change governance
datacontract/changelog/changelog.py - Catalog & publish
datacontract/catalog/catalog.py - dbt integration
datacontract/integration/dbt_sync.py
Delivery runtimes
- Python package & Docker
Dockerfile - Docs & local editor
docs/docusaurus.config.ts
Connections 27
- CLI dispatcher → Test/import/export commands: dispatches
- CLI dispatcher → DataContract API: operates on
- Optional HTTP API → DataContract API: uses
- Test/import/export commands → Test orchestrator: test
- Test/import/export commands → Importer factory: import
- Test/import/export commands → Exporter factory: export
- Test/import/export commands → Change governance: changelog / breaking gates
- Contract documents → DataContract API: loads into
- DataContract API → Lint resolver: lints
- Lint resolver → Schema validation: resolved contract
- Schema validation → Versioned schemas: validates against
- DataContract API → Test orchestrator: executes tests
- Test orchestrator → Check construction: builds checks
- Check construction → Engine adapters: executable checks
- Engine adapters → Ibis connections: selects connection
- Ibis connections → Data sources: connects to
- Engine adapters → Data sources: queries and checks
- Test orchestrator → Specialized validators: routes specialized checks
- Specialized validators → Data sources: validates
- Test orchestrator → Result outputs: emits Run results
- Importer factory → DataContract API: creates canonical form
- DataContract API → Exporter factory: projects model
- DataContract API → Change governance: compares versions
- DataContract API → dbt integration: synchronizes
- DataContract API → Catalog & publish: publishes
- Python package & Docker → CLI dispatcher: ships
- Docs & local editor → Python package & Docker: documents packaged tooling