How openucx/ucx works
UCX is a high-performance communication framework that presents abstract communication operations and selects available transport and hardware capabilities beneath them. Its principal workflow is an application using UCP to establish workers and endpoints, negotiate communication, and perform messaging or memory operations over UCT transports. The README documents UCP, UCT, UCS, and UCM roles and supported transport families; source excerpts are limited, so most cross-layer links below cite that documentation rather than implying inspected call paths.
Main entry points: ucp.h UCP API · ucp_ep.c Endpoints · uct.h UCT primitives · proto_select.c Protocol selection
Components 21
Protocol layer
- UCP API
src/ucp/api/ucp.h - Context setup
src/ucp/core/ucp_context.c - Worker progress
src/ucp/core/ucp_worker.c - Endpoints
src/ucp/core/ucp_ep.c - Connection wireup
src/ucp/wireup/wireup.c - Tag matching
src/ucp/tag/tag_send.c - Streams
src/ucp/stream/stream_send.c - Active messages
src/ucp/core/ucp_am.c - RMA and atomics
src/ucp/rma/rma_send.c - Protocol selection
src/ucp/proto/proto_select.c - Datatype handling
src/ucp/dt/dt.c - Memory types
src/ucp/core/ucp_mm.c
Transport layer
- UCT primitives
src/uct/api/uct.h - Network transports
src/uct/ib/base/ib_iface.c - Shared memory
src/uct/sm/base/sm_iface.c - GPU transports
src/uct/cuda/cuda_copy/cuda_copy_iface.c
Common services
- Utilities and data structures
src/ucs/sys/init.c
Memory hooks
- Memory event hooks
src/ucm/malloc/malloc_hook.c
Other components
- Application
- Performance tests
src/tools/perf/perftest.c - Remote peer
Connections 20
- Application → UCP API: uses primitives
README.md - UCP API → Context setup: initializes
README.md - Context setup → Worker progress: creates workers
- Worker progress → Endpoints: manages endpoints
- Endpoints → Connection wireup: negotiates connection
README.md - Application → Tag matching: requests messaging
README.md - Application → Streams: uses streams
README.md - Application → Active messages: uses messaging
README.md - Application → RMA and atomics: requests operations
README.md - Endpoints → Protocol selection: selects protocol
- Protocol selection → Datatype handling: handles data
- Protocol selection → Memory types: handles memory types
README.md - UCP API → UCT primitives: uses transports
README.md - UCT primitives → Network transports: offers network transport
README.md - UCT primitives → Shared memory: offers local transport
README.md - UCT primitives → GPU transports: supports GPU transfers
README.md - UCP API → Utilities and data structures: uses common services
- Memory event hooks → Memory types: tracks allocation events
README.md - Application → Performance tests: runs benchmarks
README.md - Endpoints → Remote peer: communicates with