How youpen-y/jiajia works
Project Type and Purpose
• JIAJIA3.0 is a distributed shared memory library aimed at allowing applications to allocate and share memory across multiple machines.
• Its static library interface offers functions (jia_init, jia_alloc, jia_lock, jia_unlock, jia_barrier, jia_exit) that abstract away the underlying complexities of memory distribution and inter-node communication.
• The project supports using either an RDMA-based protocol (via libibverbs and librdmacm) or fallback networking (e.g., UDP-based) for communication. It is also designed to be adaptable (e.g., 64-bit support, event-driven IO on Linux).
Analyzing the File Structure
• Top-Level Directories:
– apps: Contains several sample or benchmark applications (ep, hello, is, jacobi, lu, mm, mpitest, pi, sor, tsp, water) that use the JIA API. They include platform-specific Makefiles for Linux and Solaris, emphasizing that deployment is cross-platform.
– lib: Appears to provide core library code with its own Makefiles and sample tests.
– src: Implements the core system functionality. Subdirectories like comm, mem, rdma, msg, syn, init, load, and udp indicate a modular design separating concerns such as communication, memory management, synchronization, and various protocol implementations.
– doc: Contains manual documentation (PDF/PS) that’s helpful for understanding further detailed design aspects.
• Other supporting files (e.g., run.sh, killpid.sh, sort.py) and configuration files point to standardized build and deployment scripts.
Key Components
a. API Layer:
– Represent the static library interface (e.g., jia.h) that provides functions such as jia_init(), jia_exit(), jia_alloc(), jia_lock(), jia_unlock(), and jia_barrier().
– This is the entry point for user applications (shown in the apps directory).
b. Communication Subsystem:
– Include separate modules for different communication strategies:
• RDMA Communication: Represented by components in src/rdma (rdma_client.c, rdma_comm.c, etc.), which use libibverbs and librdmacm.
• UDP Communication: Represented by modules in src/udp (udp_client.c, udp_comm.c, etc.) that provide alternative network communication.
c. Memory Management:
– A dedicated module from src/mem (mem.c, memserver.c, memsyn.c, memuser.c) that implements the distributed memory allocation (jia_alloc) and possibly memory pooling.
– This component interacts with the API layer for memory requests and with the communication subsystem when synchronizing distributed memory state.
d. Synchronization & Locking:
– The synchronization layer including src/syn (syn.c, synserver.c, synuser.c) which manages locks (jia_lock, jia_unlock) and barriers (jia_barrier).
– This layer ensures consistency across nodes.
e. Initialization and Configuration:
– The initialization module in src/init (init.c) is invoked via jia_init() and sets up internal data structures and network connections.
– Configuration/settings management found in src/setting (setting.c) and related include files allows runtime and compile-time configuration.
f. Supporting Utilities:
– Additional modules such as src/tools, src/load, src/stat, and common include files (src/include) provide various helper functions and shared definitions used across the system.
Architectural Relationships and Interactions
• User applications (apps) call the API functions provided by JIAJIA3.0.
• The API (static library interface) delegates calls to the appropriate modules:
– For memory allocation, it calls into the Memory Management module.
– For synchronization, it uses the Synchronization layer.
– For communication and data consistency, it interacts with both the RDMA and UDP modules depending on availability and configuration.
• The Communication subsystem is central to enabling distributed operations – it manages both high-performance (RDMA) and fallback (UDP) protocols.
• Initialization sets up network connections, loads configuration from files (e.g., the .jiahosts file), and prepares internal states, which all subsequent calls depend on.