How fivetran/great_expectations works
GX Core is a data-quality framework: teams define Expectations, connect data sources, load batches, and evaluate data against those expectations to produce validation results that can be shared as documentation. The README establishes those product capabilities; sampled code supports the batch-management and execution-engine layers, cloud datasource integrations, and query-comparison expectation behavior. The supplied tree is selective, so some larger orchestration and result-rendering components are represented only at a documented level, without inferred call links.
Main entry points: azure_blob_storage_data_connector.py Azure connector · execution_engine.py Execution engine · pandas_azure_blob_storage_datasource.py Pandas Azure · spark_azure_blob_storage_datasource.py Spark Azure
Components 18
Data Sources
- Pandas Azure
great_expectations/datasource/fluent/pandas_azure_blob_storage_datasource.py - Spark Azure
great_expectations/datasource/fluent/spark_azure_blob_storage_datasource.py - Spark GCS
great_expectations/datasource/fluent/spark_google_cloud_storage_datasource.py - Azure connector
great_expectations/datasource/fluent/data_connector/azure_blob_storage_data_connector.py - GCS connector
great_expectations/datasource/fluent/data_connector/google_cloud_storage_data_connector.py
Validation Runtime
- Batch manager
great_expectations/core/batch_manager.py - Execution engine
great_expectations/execution_engine/execution_engine.py - SQL engine
great_expectations/execution_engine/sqlalchemy_execution_engine.py - Spark engine
great_expectations/execution_engine/sparkdf_execution_engine.py
Quality Rules
- Expectations
great_expectations/expectations/core - Query comparison
great_expectations/expectations/core/expect_query_results_to_match_comparison.py
Validation Outcomes
- Validation
- Validation results
- Result documentation
Other components
- Data team
- Data stores
- Azure Blob Storage
- Google Cloud Storage
Connections 16
- Data team → Pandas Azure: configures source
- Data team → Spark Azure: configures source
- Data team → Spark GCS: configures source
- Pandas Azure → Azure connector: builds connector
- Spark Azure → Azure connector: builds connector
- Spark GCS → GCS connector: builds connector
- Azure connector → Azure Blob Storage: discovers files
- GCS connector → Google Cloud Storage: discovers files
- Data team → Expectations: defines rules
- Data team → Validation: requests checks
- Batch manager → Execution engine: manages batch data
- SQL engine → Execution engine: implements
- Spark engine → Execution engine: implements
- Query comparison → Validation: checks query results
- Validation → Validation results: produces
- Validation results → Result documentation: documents outcomes