How apache/arrow-go works
Apache Arrow Go provides columnar in-memory data structures and computation, with readers and writers for interchange formats, Parquet support, and Flight/FlightSQL transport. A typical workflow creates Arrow arrays or record batches, processes them with compute kernels, then serializes or transfers them. The graph includes the major format and service branches documented or represented in the tree. Some subsystem links are based on README capability statements or SOURCE INDEX references; unsampled implementation details are not assumed.
Main entry points: numericbuilder.gen.go Arrays & Batches · column_readers.go Arrow-Parquet Bridge · arithmetic.go Compute API · client.go Flight RPC
Components 22
Arrow Core
- Arrow Types
arrow/datatype_fixedwidth.go - Arrays & Batches
arrow/array/numericbuilder.gen.go - Memory Management
arrow/memory/allocator.go - Bitmap Utilities
arrow/bitutil/bitmaps.go - Scalar Values
arrow/scalar/scalar.go - Variant Extension
arrow/extensions/variant.go
Compute
- Compute API
arrow/compute/arithmetic.go - Compute Kernels
arrow/compute/internal/kernels/vector_selection.go
Data Formats
- CSV I/O
arrow/csv/reader.go - Arrow IPC
arrow/ipc/reader.go - C Data Interface
arrow/cdata/cdata.go - Avro Reader
arrow/avro/reader.go - Avro Converter
arrow/avro/avro2parquet/main.go
Parquet
- Parquet Schema
parquet/schema/reflection.go - Parquet Pages
parquet/file/page_reader.go - Parquet Statistics
parquet/metadata/statistics.go - Arrow-Parquet Bridge
parquet/pqarrow/column_readers.go - Writer Properties
parquet/writer_properties.go
Flight Services
- Flight RPC
arrow/flight/client.go - FlightSQL Client
arrow/flight/flightsql/client.go - FlightSQL Server
arrow/flight/flightsql/server.go
Other components
- Go Application
Connections 21
- Go Application → Arrays & Batches: creates data
README.md - Arrays & Batches → Arrow Types: uses types
arrow/array/numericbuilder.gen.go - Arrays & Batches → Memory Management: allocates buffers
arrow/array/numericbuilder.gen.go - Arrays & Batches → Bitmap Utilities: uses bitmaps
arrow/array/numericbuilder.gen.go - Go Application → Compute API: invokes operations
README.md - Compute API → Compute Kernels: dispatches kernels
arrow/compute/arithmetic.go - Compute API → Scalar Values: uses values
arrow/compute/arithmetic.go - Go Application → CSV I/O: reads or writes
README.md - Go Application → Arrow IPC: streams batches
README.md - Arrow IPC → Memory Management: allocates buffers
- Go Application → C Data Interface: exchanges data
README.md - Avro Converter → Avro Reader: reads records
arrow/avro/avro2parquet/main.go - Avro Converter → Arrow-Parquet Bridge: writes batches
arrow/avro/avro2parquet/main.go - Arrow-Parquet Bridge → Writer Properties: configures writer
- Arrow-Parquet Bridge → Parquet Schema: maps schemas
parquet/pqarrow/column_readers.go - Arrow-Parquet Bridge → Parquet Pages: reads pages
parquet/pqarrow/column_readers.go - Parquet Pages → Parquet Statistics: reads statistics
parquet/file/page_reader.go - Go Application → Flight RPC: transfers batches
README.md - FlightSQL Client → Flight RPC: uses RPC client
arrow/flight/flightsql/client.go - FlightSQL Server → Flight RPC: routes RPC
arrow/flight/flightsql/server.go - Go Application → Variant Extension: uses extension
arrow/extensions/variant.go