How pola-rs/polars works
Polars is a Rust analytical query engine exposed through DataFrame, SQL, and Python APIs. A typical workflow builds a lazy query from data, expresses transformations, optimizes a shared plan, executes it in parallel, and returns a DataFrame; SQL queries enter that same optimizer and engine. The repository also includes Arrow interchange, streaming execution, disk spilling, and optional GPU and distributed engines. The map combines README-documented capabilities with sampled paths and source-index references; unsampled implementations and their internal call relationships are not inferred.
Main entry points: context.rs SQL context · frame.py Python DataFrame API · mod.rs Lazy query API · subquery.rs SQL subquery rewrites
Components 24
User APIs
- Python DataFrame API
py-polars/src/polars/dataframe/frame.py - Python expressions
py-polars/src/polars/expr/datetime.py - Python type definitions
py-polars/src/polars/_typing.py - Python utilities
py-polars/src/polars/_utils/various.py - SQL context
crates/polars-sql/src/context.rs - SQL expression conversion
crates/polars-sql/src/sql_expr.rs - SQL functions
crates/polars-sql/src/functions.rs - SQL subquery rewrites
crates/polars-sql/src/subquery.rs - Subquery pull-up
crates/polars-sql/src/subquery_pull_up.rs - SQL AST analysis
crates/polars-sql/src/sql_visitors.rs - Dynamic AST traversal
crates/polars-sql/src/dyn_visit.rs - DataFrame result
crates/polars-core/src/query_result.rs
Query Planning
- Lazy query API
crates/polars-lazy/src/frame/mod.rs - Query optimizer
crates/polars-plan/src/plans/optimizer/projection_pushdown/mod.rs
Execution Runtime
- Vectorized compute
crates/polars-compute/src/lib.rs - Streaming execution
crates/polars-stream/src/nodes/io_sinks/components/morsel_resize_pipeline.rs - Memory and spilling
crates/polars-ooc/src/memory_manager.rs
Data and Interchange
- Arrow columnar data
crates/polars-arrow/src/lib.rs - Data I/O
crates/polars-io/src/csv/read/schema_inference.rs
Optional Engines
- Python Rust binding
py-polars/src/polars/_plr.py - Engine configuration
py-polars/src/polars/lazyframe/engine_config.py - Remote engine
py-polars/src/polars/lazyframe/engine_remote.py - GPU engine
py-polars/src/polars/lazyframe/engine.py
Other components
- Query author
Connections 25
- Query author → Python DataFrame API: builds queries
README.md - Query author → SQL context: writes SQL
README.md - Python DataFrame API → Python expressions: composes expressions
README.md - Python DataFrame API → Lazy query API: collects lazy query
README.md - SQL context → SQL expression conversion: converts expressions
crates/polars-sql/src/context.rs - SQL context → SQL functions: handles functions
crates/polars-sql/src/context.rs - SQL context → SQL subquery rewrites: rewrites subqueries
crates/polars-sql/src/context.rs - SQL subquery rewrites → Subquery pull-up: uses pull-up rewrites
crates/polars-sql/src/subquery.rs - SQL context → Lazy query API: plans SQL
crates/polars-sql/src/context.rs - SQL expression conversion → SQL functions: resolves functions
crates/polars-sql/src/sql_expr.rs - SQL context → SQL AST analysis: analyzes SQL AST
crates/polars-sql/src/context.rs - SQL AST analysis → Dynamic AST traversal: traverses AST
crates/polars-sql/src/sql_visitors.rs - SQL subquery rewrites → Dynamic AST traversal: visits subqueries
crates/polars-sql/src/subquery.rs - Lazy query API → Query optimizer: optimizes plan
README.md - Query optimizer → Vectorized compute: executes plan
README.md - Vectorized compute → DataFrame result: returns result
README.md - Data I/O → Arrow columnar data: uses columnar data
README.md - Arrow columnar data → Python Rust binding: shares data
README.md - Python Rust binding → Python DataFrame API: exposes Rust API
py-polars/src/polars/_plr.py - Query optimizer → Streaming execution: executes streaming
README.md - Vectorized compute → Memory and spilling: spills when needed
README.md - Lazy query API → Remote engine: can execute remotely
README.md - Lazy query API → GPU engine: can use GPU
README.md - Python DataFrame API → Engine configuration: configures engine
py-polars/src/polars/lazyframe/engine_config.py - Engine configuration → Python Rust binding: sets engine options
py-polars/src/polars/lazyframe/engine_config.py