How saezlab/annnet works
1. Top-level shape
annnet is a Python library organized around one central graph model in annnet/core, with three major extension surfaces around it: runtime graph-library adapters in annnet/adapters, file/storage I/O in annnet/io, and native algorithms in annnet/algorithms. Utilities, plotting, and backend-agnostic helpers live in annnet/_support and annnet/utils.
2. Core domain model
The heart of the repo is the Graph object in annnet/core/graph.py. The core design is a sparse, incidence-matrix-based graph representation augmented with annotated tabular state. That means structure and metadata are intentionally separated: topology lives in matrix/record structures, while vertex/edge/layer/slice annotations are stored as dataframe-backed tables.
3. State decomposition inside core
The core package is split into focused subsystems: _Matrix for structural incidence data, _Layers and _Slices for multilayer and slice semantics, _Annotation for metadata, _History for audit/undo-style state tracking, _Views for derived projections, _Ops for graph operations, and _records / _common-style helpers for internal record handling. This separation is a key architectural boundary: the graph object orchestrates these pieces rather than embedding everything in one class.
4. Managers and views
Graph mutation is not just direct object editing; the model uses manager-like components and view objects to expose filtered or derived perspectives over the underlying state. The documentation and tests indicate that indexing, slice selection, active views, and derived caches are first-class concepts. For an architecture diagram, it is important to show that “views” sit on top of the core data rather than duplicating it.
5. Backend accessors as a projection layer
G.nx, G.ig, and G.gt are graph-owned backend accessors under annnet/core/backend_accessors. They provide lazy projection of an AnnNet graph into NetworkX, igraph, or graph-tool graphs, and can also dispatch algorithms directly against those libraries. This is a distinct boundary: AnnNet remains the source of truth, while backend graphs are cached projections that must be refreshed after mutations.
6. Explicit adapters for interoperability
The annnet/adapters package contains the conversion layer between AnnNet and external graph ecosystems, including NetworkX, igraph, graph-tool, and PyG. These adapters are used both by the backend accessors and by explicit import/export APIs. The repo keeps conversion logic separate from the core model so that interoperability does not leak third-party assumptions into the graph internals.
7. File I/O and format modules
annnet/io is a family of format-specific readers/writers, not a single monolithic persistence layer. It includes GraphML, GEXF, SIF, SBML, CX2, JSON/NDJSON, CSV/TSV, Excel, Parquet, DataFrame exchange, and the native .annnet format. The I/O layer is where lossy-vs-lossless conversion policy lives, often via manifests that preserve information when a target format cannot encode all AnnNet semantics.
8. Native persistence layout
The .annnet format is the repo’s lossless on-disk representation. The README and I/O modules show a split layout: manifest.json for metadata and versioning, Zarr for sparse structure, Parquet for tabular annotations and structural indices, and JSON for unstructured or auxiliary metadata. This makes storage a composition of technologies rather than a single database format.
9. Algorithm layer
annnet/algorithms currently centers on traversal and similar graph-native routines. These algorithms are written against the AnnNet data model rather than against NetworkX or igraph, which keeps them usable even when no external backend is installed. In the diagram, this layer should point to core graph structures, not to adapters.
10. Shared support layer
The _support package contains infrastructure that multiple subsystems reuse: dataframe backend abstraction, optional component handling, lazy exports, serialization helpers, plotting backend integration, metadata utilities, and graph-record helpers. This layer is what lets AnnNet stay backend-agnostic for tables while still supporting multiple optional ecosystems.
11. Tabular backend strategy
AnnNet’s dataframe story is intentionally abstracted through Narwhals, with optional support for Polars, pandas, and PyArrow-backed objects. The architecture implication is that annotation tables are not tied to one dataframe implementation; instead, the core model depends on a compatibility layer that normalizes dataframe operations across engines.
12. Optional runtime dependencies
The repo is designed so that the core runtime stays small: NumPy, SciPy, and Narwhals are the essential dependencies, while NetworkX, igraph, graph-tool, plotting libraries, and richer I/O dependencies are optional extras. That modular dependency boundary matters architecturally because it keeps the graph core usable in minimal environments and only activates extra subsystems when installed.
13. Testing, docs, and validation as architectural guardrails
The test suite is broad and mirrors the subsystem boundaries: backend parity, IO round-trips, slice/history correctness, interoperability, and edge cases all have dedicated tests. Docs are also structured by architecture topics, which suggests the public API is intentionally aligned with the internal decomposition. For a diagram, this implies the repo is organized around preserving semantic consistency across conversions and storage, not just around feature modules.
14. Primary data flow
The main flow is: user code mutates or queries Graph in core; views/managers derive projections; optional backend accessors convert to external graph objects for algorithms; IO modules serialize either to external interchange formats or to the native .annnet store; and manifests/caches preserve fidelity and speed across round-trips. The cleanest diagram should show core as the hub, with adapters and I/O as outward-facing projection/persistence layers.
Main entry points: graph.py Graph · optional_components.py Optional deps · nx_accessor.py NX Accessor · ig_accessor.py IG Accessor
Components 28
Core
Source-of-truth graph model and internal state subsystems
- Graph · Central AnnNet graph object and orchestration boundary
annnet/core/graph.py - Matrix · Sparse incidence and structural state
annnet/core/_Matrix.py - Layers · Layer semantics and layer-aware structure
annnet/core/_Layers.py - Slices · Slice semantics and selection state
annnet/core/_Slices.py - Annotations · Vertex, edge, layer, and slice annotation tables
annnet/core/_Annotation.py - History · Mutation history and audit-style tracking
annnet/core/_History.py - Views · Filtered and derived graph views
annnet/core/_Views.py - Ops · Core graph operations and mutation helpers
annnet/core/_Ops.py - Records · Internal record handling and common graph state helpers
annnet/core/_records.py - NX Accessor · Lazy NetworkX projection and backend dispatch
annnet/core/backend_accessors/nx_accessor.py - IG Accessor · Lazy igraph projection and backend dispatch
annnet/core/backend_accessors/ig_accessor.py - GT Accessor · Lazy graph-tool projection and backend dispatch
annnet/core/backend_accessors/gt_accessor.py
Adapters
Projections to external graph ecosystems
- NetworkX · Conversion layer for NetworkX interoperability
annnet/adapters/networkx_adapter.py - igraph · Conversion layer for igraph interoperability
annnet/adapters/igraph_adapter.py - graph-tool · Conversion layer for graph-tool interoperability
annnet/adapters/graphtool_adapter.py - PyG · Conversion layer for PyTorch Geometric interoperability
annnet/adapters/pyg_adapter.py
I/O
Import/export and native persistence formats
- .annnet · Lossless native persistence format with manifest and columnar storage
annnet/io/annnet_format.py - GraphML/GEXF · Graph interchange readers and writers
annnet/io/graphml.py - JSON · JSON and NDJSON serialization
annnet/io/json_format.py - Parquet · Columnar persistence for annotations and indices
annnet/io/parquet.py - CSV/TSV · Delimited text import/export
annnet/io/csv_format.py - SBML · SBML import/export and COBRA variant support
annnet/io/sbml.py - SIF · Simple interaction format support
annnet/io/sif.py
Support
Shared backend-agnostic utilities
- Dataframes · Narwhals-based dataframe backend abstraction
annnet/_support/dataframe_backend.py - Metadata · Metadata helpers and serialization support
annnet/_support/metadata.py - Optional deps · Optional component gating and lazy imports
annnet/_support/optional_components.py - Plotting · Backend-aware plotting integration
annnet/utils/plotting.py
Algorithms
Graph-native routines built on the core model
- Traversal · Graph-native traversal routines
annnet/algorithms/traversal.py
Connections 31
- Graph → Matrix: stores topology · Graph orchestrates sparse structural state
- Graph → Layers: tracks layers · Layer semantics are managed as separate state
- Graph → Slices: tracks slices · Slice semantics are managed as separate state
- Graph → Annotations: stores metadata · Annotations live in dataframe-backed tables
- Graph → History: records changes · Mutation history is tracked separately from structure
- Graph → Views: exposes views · Derived and filtered projections sit on top of core state
- Graph → Ops: mutates via · Graph operations coordinate state transitions
- Graph → Records: uses records · Internal record helpers support core state handling
- Graph → NX Accessor: projected by · Graph-owned accessors lazily build backend graphs
- Graph → IG Accessor: projected by · Graph-owned accessors lazily build backend graphs
- Graph → GT Accessor: projected by · Graph-owned accessors lazily build backend graphs
- NX Accessor → NetworkX: converts to · NetworkX adapter performs interoperability conversion
- IG Accessor → igraph: converts to · igraph adapter performs interoperability conversion
- GT Accessor → graph-tool: converts to · graph-tool adapter performs interoperability conversion
- Graph → PyG: exports to · PyG interoperability is handled outside the core model
- Graph → .annnet: persists as · Native lossless storage format
- Graph → GraphML/GEXF: imports/exports · External interchange formats are handled in I/O
- Graph → JSON: imports/exports · JSON formats are handled in I/O
- Graph → Parquet: stores tables · Columnar storage supports annotations and indices
- Graph → CSV/TSV: imports/exports · Delimited formats are handled in I/O
- Graph → SBML: imports/exports · SBML family is handled in I/O
- Graph → SIF: imports/exports · SIF support is handled in I/O
- Dataframes → Annotations: backing tables · Table operations are backend-abstracted through Narwhals
- Metadata → .annnet: supports · Metadata helpers participate in persistence and manifests
- Optional deps → NX Accessor: gates · Optional dependency handling controls backend availability
- Optional deps → IG Accessor: gates · Optional dependency handling controls backend availability
- Optional deps → GT Accessor: gates · Optional dependency handling controls backend availability
- Optional deps → Plotting: gates · Optional dependency handling controls plotting integrations
- Plotting → Graph: visualizes · Plotting utilities render graph state
- Traversal → Graph: operates on · Algorithms target the AnnNet graph model directly
- Traversal → Views: uses projections · Algorithms can work on derived views and selections