How snowflakedb/snowflake-connector-python works
Purpose and entry points
This is the Snowflake Python connector: a Python 3.10+ DB-API 2.0 driver for connecting to Snowflake, executing SQL, transferring staged files, and materializing results. Public synchronous entry points are snowflake.connector and DB-API connection creation in src/snowflake/connector/__init__.py and src/snowflake/connector/dbapi.py; the primary lifecycle implementation is src/snowflake/connector/connection.py. An async API is independently implemented under src/snowflake/connector/aio/__init__.py.
Connection, session, and query control
SnowflakeConnection establishes a Snowflake session, owns session parameters/state, coordinates authentication and HTTP operations, and creates cursors. Cursors submit SQL, handle bindings and multi-statement execution, expose fetch APIs, and manage query cancellation/status boundaries.
Paths: src/snowflake/connector/connection.py, src/snowflake/connector/cursor.py, src/snowflake/connector/session_manager.py.
Authentication boundary
Authentication is strategy/plugin based: the connection delegates credentials and login-token acquisition to plugins for password/MFA, key pair/JWT, OAuth variants, PAT, external browser/Okta, and workload identity. Token caching and browser callback handling are adjacent concerns.
Paths: src/snowflake/connector/auth/by_plugin.py, src/snowflake/connector/auth/keypair.py, src/snowflake/connector/auth/workload_identity.py.
Network and transport security
The network layer sends Snowflake REST requests, applies retry/backoff, proxy configuration, redirects, TLS hardening, and session renewal. Certificate validation includes OCSP/CRL support; a vendored Requests/urllib3 stack supplies core HTTP transport behavior.
Paths: src/snowflake/connector/network.py, src/snowflake/connector/ocsp_snowflake.py, src/snowflake/connector/vendored/requests/sessions.py.
Results and Python type conversion
Query responses become result batches/result sets, then cursor fetches. The converter layer maps Snowflake wire values to Python types and supports alternate conversion modes. Arrow context and bundled C++/Cython nanoarrow iterators form the performance-critical Arrow decoding path, including Pandas-oriented consumption.
Paths: src/snowflake/connector/result_set.py, src/snowflake/connector/converter.py, src/snowflake/connector/nanoarrow_cpp/ArrowIterator/nanoarrow_arrow_iterator.pyx.
File staging and cloud storage
PUT/GET and bind-upload flows pass through a file-transfer orchestration layer that compresses/encrypts files and selects storage clients. Implementations target S3, Azure Blob Storage, GCS, and local storage; these are an external-data-plane boundary distinct from Snowflake query REST traffic.
Paths: src/snowflake/connector/file_transfer_agent.py, src/snowflake/connector/storage_client.py, src/snowflake/connector/s3_storage_client.py.
Async parity and observability
The aio package mirrors core connection, cursor, auth, network, results, telemetry, and cloud-storage flows using async implementations rather than wrapping the synchronous path. Telemetry is emitted from connector activity and can be disabled through connection/session settings; out-of-band telemetry is separated explicitly.
Paths: src/snowflake/connector/aio/_connection.py, src/snowflake/connector/aio/_file_transfer_agent.py, src/snowflake/connector/telemetry.py.
Runtime, native artifacts, and delivery
Primary runtime is Python, packaged through PEP 517/setuptools (pyproject.toml, setup.py). Native pieces include nanoarrow C++/Cython sources and platform-specific Rust sf_mini_core binaries used by the driver. CI builds wheels across Linux, macOS, and Windows, including Dockerized and FIPS/Lambda-oriented environments.
Paths: pyproject.toml, src/snowflake/connector/minicore/, ci/build_docker.sh.
Main entry points: connection.py SnowflakeConnection · network.py Network layer · cursor.py Cursor · by_plugin.py Auth plugins
Components 24
Synchronous connector
- DB-API public API · Synchronous connection entry points.
src/snowflake/connector/__init__.py - SnowflakeConnection · Owns session state, authentication, HTTP operations, and cursors.
src/snowflake/connector/connection.py - Session manager
src/snowflake/connector/session_manager.py - Cursor · Submits SQL and exposes fetch, status, and cancellation APIs.
src/snowflake/connector/cursor.py - Auth plugins · Credentials and login tokens for password, OAuth, key pair, and identity flows.
src/snowflake/connector/auth/by_plugin.py - Key pair JWT
src/snowflake/connector/auth/keypair.py - Workload identity
src/snowflake/connector/auth/workload_identity.py - Network layer · Handles retries, proxies, redirects, TLS, and session renewal.
src/snowflake/connector/network.py - OCSP and CRL validation
src/snowflake/connector/ocsp_snowflake.py - Vendored HTTP stack
src/snowflake/connector/vendored/requests/sessions.py - Telemetry · Emits separately controllable connector activity telemetry.
src/snowflake/connector/telemetry.py
Async connector
- Async connection API · Independent async implementation mirroring core connector flows.
src/snowflake/connector/aio/_connection.py - Async file transfer
src/snowflake/connector/aio/_file_transfer_agent.py
Results and staging
- Result sets and batches
src/snowflake/connector/result_set.py - Python converter · Maps Snowflake values to Python types.
src/snowflake/connector/converter.py - Nanoarrow iterator · Performance-critical Arrow and Pandas result decoding.
src/snowflake/connector/nanoarrow_cpp/ArrowIterator/nanoarrow_arrow_iterator.pyx - File transfer agent · Coordinates PUT, GET, bind upload, compression, and encryption.
src/snowflake/connector/file_transfer_agent.py - Storage clients · Selects S3, Azure, GCS, or local storage implementations.
src/snowflake/connector/storage_client.py - S3 storage client
src/snowflake/connector/s3_storage_client.py
Build and native runtime
- PEP 517 packaging
pyproject.toml - sf_mini_core binaries · Platform-specific native artifacts used by the driver.
- Cross-platform wheel CI · Builds Linux, macOS, and Windows wheels, including specialized environments.
ci/build_docker.sh
Other components
- Snowflake service
- Cloud storage
Connections 24
- DB-API public API → SnowflakeConnection: creates
- SnowflakeConnection → Session manager: manages session
- SnowflakeConnection → Cursor: creates
- SnowflakeConnection → Auth plugins: authenticates through
- Auth plugins → Key pair JWT: uses
- Auth plugins → Workload identity: uses
- SnowflakeConnection → Network layer: uses
- Network layer → OCSP and CRL validation: validates certificates
- Network layer → Vendored HTTP stack: sends via
- Network layer → Snowflake service: REST requests
- Cursor → Network layer: executes SQL
- Cursor → Result sets and batches: materializes responses
- Result sets and batches → Python converter: converts values
- Result sets and batches → Nanoarrow iterator: Arrow decoding
- Cursor → File transfer agent: PUT/GET and bind upload
- File transfer agent → Storage clients: selects backend
- Storage clients → S3 storage client: implements
- Storage clients → Cloud storage: transfers files
- SnowflakeConnection → Telemetry: emits activity
- Async connection API → Snowflake service: async REST flows
- Async connection API → Async file transfer: staging operations
- Async file transfer → Cloud storage: async transfers
- PEP 517 packaging → sf_mini_core binaries: packages
- Cross-platform wheel CI → PEP 517 packaging: builds wheels