How xpn-arcos/xpn works
Project type and purpose
• XPN (“Expand Ad‑Hoc Parallel File System”) is an open‑source, MPI‑based parallel file system and I/O middleware.
• It provides a user‑level, POSIX‑compatible file interface via LD_PRELOAD (xpn_bypass), plus optional FUSE and MPI‑IO connectors, to a set of MPI‑based server processes that stripe and store file data on per‑node local storage.
File structure
• Top‑level modules:
• src/xpn_server: the MPI‑based server implementation
• src/xpn_client + include/xpn_client: the client libraries (bypass, policy, core) and MPI socket or message‑queue stubs
• src/base + include/base: foundational utilities (networking, threading, profiling) shared by client & server
• src/bypass + include/bypass: LD_PRELOAD library that intercepts POSIX calls
• src/connector_fuse, src/connector_romio: optional adapters (FUSE mount, MPI‑IO integration)
• src/utils: user utilities (xpn‑ls, xpn‑cp, xpn_expand, etc.)
• src/mq_server: standalone message queue server support
• admire/io‑scheduler: experimental scheduling plugin
• scripts/execute, scripts/compile, scripts/spack: deployment & build automation
• Config & build:
• xpn.conf (example in info/xpn.conf.example) plus hostfile for MPI
• Autotools / CMake build infrastructure (configure.ac, Makefile.am, CMakeLists.txt)
• Docker, Spack recipe for packaging/deployment
README for additional insights
• Architecture: three‑step execution (1. generate xpn.conf, 2. start servers, 3. launch client).
• Communication: clients (LD_PRELOAD or mpiexec) talk to servers via TCP sockets or MPI messaging.
• Deployment: supports MPICH and OpenMPI, cluster launch via mpiexec or prte.
Elements
• Labels:
• “Configuration Manager” (mk_conf.sh, xpn.sh)
• “MPI Servers” (xpn_server)
• “NODE_DIR” (local filesystem)
• “LD_PRELOAD Bypass” (xpn_bypass.so)
• “User App or MPI App”
• “FUSE Connector”
• “Shared WORK_DIR (hostfile, xpn.conf)”
• “External S3 Service (optional)”
• Config flow (script → xpn.conf)
• Control flow (launcher → servers → client → stop)
• Data flow (client bypass → server → local disk)
• Side‑path to S3 via stage_cloud
• Orange for optional connectors
Final advice
• Be explicit about which processes run where (per‑node MPI server vs. client process)
• Both control (start/stop) and data (read/write) flows
• Don’t omit supporting libraries (base, bypass) that glue the system together
• You show pluggability: how new connectors or scheduling plugins fit in