# complexluise/bib2graph architecture

> Architecture diagram of the GitHub repository complexluise/bib2graph, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/complexluise/bib2graph
- Repository: https://github.com/complexluise/bib2graph

## Overview

1. **What this repo is**
   `bib2graph` is a local Python package, not a server app. Its main boundary is a research workspace on disk: you seed a corpus, curate it, persist it, and then project it into bibliometric networks. The same code is exposed both as a CLI (`b2g`) and as a Python library.

2. **Primary data model**
   The central artifact is the **corpus**: a collection of papers/works with stable identity and provenance. Everything else hangs off it: curation state, snapshots, network projections, and exports. The repo is organized around turning raw bibliographic inputs into a reproducible corpus.

3. **Input adapters**
   There are multiple sources under `src/bib2graph/sources`: OpenAlex, BibTeX, and an equation-based spec. OpenAlex is the backbone for retrieval; BibTeX is an alternate seed path. Inputs are normalized into the common corpus model before later stages run.

4. **Ingestion and preprocessing**
   The preprocessing layer (`preprocessors/`) handles normalization, deduplication, and thesaurus-like vocabulary handling before the corpus is stored or expanded. This is where source-specific variation is reduced so later steps can operate on a canonical representation.

5. **Expansion / foraging loop**
   The `foraging/` subsystem is the iterative “grow the corpus” engine. It explores citations and candidate papers, ranks them using deterministic structural signals, and feeds the next round of selection. The README explicitly says this is **not** LLM-based; the expansion logic is bibliometric and reproducible.

6. **Human curation boundary**
   Curation is a first-class subsystem (`service/curate.py`, CLI commands like `accept`, `reject`, `curate`, `validate`, `restore`). This is a human-in-the-loop boundary: the tool records decisions, supports PRISMA-style filtering, and keeps the curation trail versionable rather than hiding it inside the pipeline.

7. **Persistence and workspace**
   Persistence is optional but important: `stores/duckdb.py` and `backends/duckdb.py` make DuckDB the durable local store, while `memory.py` variants support ephemeral/test operation. `workspace.py` and `service/store.py` suggest a workspace-centric design where each investigation keeps its own state, snapshots, and outputs.

8. **Network projection subsystem**
   Once the corpus is ready, `networks/` builds the bibliometric projections. The repo supports five main graph families: bibliographic coupling, co-citation, co-authorship, institutional collaboration, and keyword co-occurrence. The network layer also includes analysis, projection specs, clustering, and decoration of nodes/edges with metrics.

9. **Export boundary**
   Exporters in `exporters/` convert the internal network representation to external analysis formats, notably GraphML and CSV. This is the handoff point to downstream tools such as Gephi, Cytoscape, or networkx.

10. **Service layer as workflow owner**
    The `service/` package is the orchestration layer that owns the canonical flow across ingestion, resolution, curation, snapshotting, and reading. This keeps the CLI thin: commands delegate to services rather than implementing business logic themselves.

11. **CLI surface**
    The CLI under `src/bib2graph/cli/commands` maps directly to lifecycle steps: `init`, `seed`, `chain`, `build`, `read`, `export`, `inspect`, `status`, `snapshot`, `resolve`, `monitor`, `filter`, `enrich`, and more. It supports `--json`, which makes the tool scriptable and easy to drive from agents or automation.

12. **Determinism and reproducibility**
    Reproducibility is a core architectural constraint. The repo includes hashing, snapshots, identity/merge rules, and network specs so the same input state yields the same corpus and networks. That is why the system emphasizes content identity, snapshot history, and deterministic projection rules.

13. **External dependency boundary**
    The only major external service in the runtime architecture is OpenAlex. Everything else is local: files, DuckDB, and exports. This makes the system reproducible offline after seed data is captured, while still allowing live enrichment when needed.

14. **Docs and operational wrappers**
    The `docs/` tree contains ADRs, workflow notes, and feature specs that document the architecture’s evolution. The `.claude/` and `src/bib2graph/skill/` assets package an agent-oriented operational wrapper around the same CLI workflow, but they do not change the core runtime architecture.

15. **How to draw the diagram**
    A good diagram would show: external bibliographic sources → source adapters → preprocessing/dedup → corpus store/workspace → curation/validation loop → foraging/expansion loop → network projection → exporters. Around that, place the CLI as the user-facing entrypoint and the service layer as the orchestration boundary connecting every stage.

Main entry points: [`src/bib2graph/cli/commands`](https://github.com/complexluise/bib2graph/tree/dev/src/bib2graph/cli/commands) (CLI commands), [`src/bib2graph/corpus.py`](https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/corpus.py) (Corpus), [`src/bib2graph/networks/projectors.py`](https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/networks/projectors.py) (Projectors), [`src/bib2graph/service/store.py`](https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/service/store.py) (Store service)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ui["User surface"]
  node_cli_commands["CLI commands<br/>entrypoint"]
  node_skill["Skill wrapper<br/>agent ops<br/>[SKILL.md]"]
end

subgraph group_workspace["Workspace core"]
  node_workspace["Workspace<br/>state root<br/>[workspace.py]"]
  node_corpus[("Corpus<br/>domain model<br/>[corpus.py]")]
  node_schemas["Schemas<br/>[schemas.py]"]
  node_service_store["Store service<br/>orchestration<br/>[store.py]"]
  node_service_resolve["Resolve service<br/>orchestration<br/>[resolve.py]"]
  node_stores[("Stores<br/>persistence")]
  node_backends["Backends<br/>runtime backend"]
end

subgraph group_ingest["Ingestion"]
  node_sources["Sources<br/>adapters"]
  node_preprocessors["Preprocess<br/>pipeline<br/>[pipeline.py]"]
  node_forager["Forager<br/>expansion loop<br/>[forager.py]"]
  node_scent["Scent<br/>ranking<br/>[scent.py]"]
  node_enrichers["Enrichers<br/>augmentation"]
end

subgraph group_curation["Curation"]
  node_service_curate["Curate service<br/>workflow<br/>[curate.py]"]
  node_service_snapshot["Snapshot service<br/>workflow<br/>[snapshot.py]"]
  node_service_reads["Read service<br/>workflow<br/>[reads.py]"]
  node_filter_prisma["PRISMA filter<br/>filtering<br/>[prisma.py]"]
end

subgraph group_networks["Networks"]
  node_networks_facade["Networks facade<br/>projection<br/>[facade.py]"]
  node_projectors["Projectors<br/>projection<br/>[projectors.py]"]
  node_analyzer["Analyzer<br/>metrics<br/>[analyzer.py]"]
  node_clusters["Clusters<br/>community<br/>[clusters.py]"]
  node_decorate["Decorate<br/>annotation<br/>[decorate.py]"]
  node_exporters["Exporters"]
end

node_cli_commands -->|"orchestrates"| node_service_store
node_cli_commands -->|"curates"| node_service_curate
node_cli_commands -->|"resolves"| node_service_resolve
node_cli_commands -->|"snapshots"| node_service_snapshot
node_cli_commands -->|"reads"| node_service_reads
node_cli_commands -->|"builds"| node_forager
node_cli_commands -->|"projects"| node_networks_facade
node_skill -.->|"wraps"| node_cli_commands
node_sources -->|"normalizes"| node_preprocessors
node_preprocessors -->|"writes"| node_corpus
node_sources -->|"ingests"| node_service_store
node_forager -->|"scores"| node_scent
node_forager -->|"expands from"| node_sources
node_enrichers -->|"augments"| node_corpus
node_service_store -->|"persists in"| node_workspace
node_service_store -->|"uses"| node_stores
node_stores -->|"backed by"| node_backends
node_service_resolve -->|"merges"| node_corpus
node_service_curate -->|"screens with"| node_filter_prisma
node_service_curate -->|"updates"| node_corpus
node_service_snapshot -->|"captures"| node_corpus
node_service_reads -->|"reads"| node_corpus
node_networks_facade -->|"builds"| node_projectors
node_projectors -->|"analyzes"| node_analyzer
node_projectors -->|"clusters"| node_clusters
node_projectors -->|"decorates"| node_decorate
node_networks_facade -->|"exports to"| node_exporters
node_corpus -.->|"conforms to"| node_schemas
node_service_reads -.->|"prepares"| node_exporters

click node_cli_commands "https://github.com/complexluise/bib2graph/tree/dev/src/bib2graph/cli/commands"
click node_skill "https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/skill/SKILL.md"
click node_workspace "https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/workspace.py"
click node_corpus "https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/corpus.py"
click node_schemas "https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/schemas.py"
click node_service_store "https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/service/store.py"
click node_service_resolve "https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/service/resolve.py"
click node_service_curate "https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/service/curate.py"
click node_service_snapshot "https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/service/snapshot.py"
click node_service_reads "https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/service/reads.py"
click node_filter_prisma "https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/filters/prisma.py"
click node_sources "https://github.com/complexluise/bib2graph/tree/dev/src/bib2graph/sources"
click node_preprocessors "https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/preprocessors/pipeline.py"
click node_forager "https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/foraging/forager.py"
click node_scent "https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/foraging/scent.py"
click node_enrichers "https://github.com/complexluise/bib2graph/tree/dev/src/bib2graph/enrichers"
click node_networks_facade "https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/networks/facade.py"
click node_projectors "https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/networks/projectors.py"
click node_analyzer "https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/networks/analyzer.py"
click node_clusters "https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/networks/clusters.py"
click node_decorate "https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/networks/decorate.py"
click node_exporters "https://github.com/complexluise/bib2graph/tree/dev/src/bib2graph/exporters"
click node_stores "https://github.com/complexluise/bib2graph/tree/dev/src/bib2graph/stores"
click node_backends "https://github.com/complexluise/bib2graph/tree/dev/src/bib2graph/backends"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli_commands,node_skill toneBlue
class node_workspace,node_corpus,node_schemas,node_service_store,node_service_resolve,node_stores,node_backends toneAmber
class node_sources,node_preprocessors,node_forager,node_scent,node_enrichers toneMint
class node_service_curate,node_service_snapshot,node_service_reads,node_filter_prisma toneRose
class node_networks_facade,node_projectors,node_analyzer,node_clusters,node_decorate,node_exporters toneIndigo
```

## Components

### User surface

CLI entrypoints and agent-oriented wrappers.

- [CLI commands](https://github.com/complexluise/bib2graph/tree/dev/src/bib2graph/cli/commands): entrypoint · Command surface for init, seed, build, curate, read, export, and related lifecycle verbs. · `src/bib2graph/cli/commands`
- [Skill wrapper](https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/skill/SKILL.md): agent ops · Agent-oriented packaging and reference workflow around the CLI. · `src/bib2graph/skill/SKILL.md`

### Workspace core

Corpus model, state, storage, and orchestration services.

- [Workspace](https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/workspace.py): state root · Per-investigation disk workspace holding corpus state, snapshots, and outputs. · `src/bib2graph/workspace.py`
- [Corpus](https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/corpus.py): domain model · Canonical collection of works with identity, provenance, and lifecycle state. · `src/bib2graph/corpus.py`
- [Schemas](https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/schemas.py): schema · Shared data contracts used across ingestion, storage, and projections. · `src/bib2graph/schemas.py`
- [Store service](https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/service/store.py): orchestration · Workspace-aware storage and read/write coordination. · `src/bib2graph/service/store.py`
- [Resolve service](https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/service/resolve.py): orchestration · Identity resolution and merge logic for corpus records. · `src/bib2graph/service/resolve.py`
- [Stores](https://github.com/complexluise/bib2graph/tree/dev/src/bib2graph/stores): persistence · Persistence backends for durable DuckDB storage and memory-mode operation. · `src/bib2graph/stores`
- [Backends](https://github.com/complexluise/bib2graph/tree/dev/src/bib2graph/backends): runtime backend · Backend abstraction layer used by the store implementations. · `src/bib2graph/backends`

### Ingestion

Source adapters and preprocessing before corpus storage.

- [Sources](https://github.com/complexluise/bib2graph/tree/dev/src/bib2graph/sources): adapters · Input adapters for OpenAlex, BibTeX, and declarative equation specs. · `src/bib2graph/sources`
- [Preprocess](https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/preprocessors/pipeline.py): pipeline · Normalization, deduplication, and thesaurus handling before canonical storage. · `src/bib2graph/preprocessors/pipeline.py`
- [Forager](https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/foraging/forager.py): expansion loop · Deterministic corpus growth engine driven by bibliometric signals. · `src/bib2graph/foraging/forager.py`
- [Scent](https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/foraging/scent.py): ranking · Structural scoring and selection signals for iterative expansion. · `src/bib2graph/foraging/scent.py`
- [Enrichers](https://github.com/complexluise/bib2graph/tree/dev/src/bib2graph/enrichers): augmentation · Optional enrichment layer, including OpenAlex-backed enrichment. · `src/bib2graph/enrichers`

### Curation

Human-in-the-loop filtering, validation, and resolution.

- [Curate service](https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/service/curate.py): workflow · Human curation workflow for accept, reject, validate, restore, and traceable decisions. · `src/bib2graph/service/curate.py`
- [Snapshot service](https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/service/snapshot.py): workflow · Snapshotting and reproducibility boundary for stable corpus history. · `src/bib2graph/service/snapshot.py`
- [Read service](https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/service/reads.py): workflow · Read-oriented views over the corpus and its derived outputs. · `src/bib2graph/service/reads.py`
- [PRISMA filter](https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/filters/prisma.py): filtering · Structured screening and exclusion logic used during curation. · `src/bib2graph/filters/prisma.py`

### Networks

Projection, analysis, clustering, decoration, and export.

- [Networks facade](https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/networks/facade.py): projection · High-level entry into bibliometric network building and analysis. · `src/bib2graph/networks/facade.py`
- [Projectors](https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/networks/projectors.py): projection · Builds the main graph families such as coupling, co-citation, co-authorship, institutions, and keywords. · `src/bib2graph/networks/projectors.py`
- [Analyzer](https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/networks/analyzer.py): metrics · Network analysis and metric computation over projected graphs. · `src/bib2graph/networks/analyzer.py`
- [Clusters](https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/networks/clusters.py): community · Grouping and cluster extraction for projected networks. · `src/bib2graph/networks/clusters.py`
- [Decorate](https://github.com/complexluise/bib2graph/blob/dev/src/bib2graph/networks/decorate.py): annotation · Adds metrics and annotations onto network nodes and edges. · `src/bib2graph/networks/decorate.py`
- [Exporters](https://github.com/complexluise/bib2graph/tree/dev/src/bib2graph/exporters): export · CSV and GraphML handoff for external graph tools. · `src/bib2graph/exporters`

## Connections

- CLI commands → Store service: orchestrates · CLI verbs delegate to workspace and storage services.
- CLI commands → Curate service: curates · Curate/accept/reject/validate commands flow through the curation service.
- CLI commands → Resolve service: resolves · Resolve commands invoke identity and merge logic.
- CLI commands → Snapshot service: snapshots · Snapshot-related commands preserve corpus history.
- CLI commands → Read service: reads · Read and inspect commands use the read service.
- CLI commands → Forager: builds · Seed/chain/build commands drive iterative corpus expansion.
- CLI commands → Networks facade: projects · Network commands build and inspect graph projections.
- Skill wrapper → CLI commands: wraps · Agent-oriented workflow wrapper around the same CLI surface.
- Sources → Preprocess: normalizes · Raw bibliographic inputs are canonicalized before storage.
- Preprocess → Corpus: writes · Normalized and deduplicated records enter the corpus model.
- Sources → Store service: ingests · Source adapters hand normalized records into the store boundary.
- Forager → Scent: scores · Expansion uses deterministic structural signals for ranking.
- Forager → Sources: expands from · Iterative growth explores citations and candidate papers through adapters.
- Enrichers → Corpus: augments · Optional enrichment adds external metadata to corpus records.
- Store service → Workspace: persists in · Workspace-scoped state owns storage lifecycle and paths.
- Store service → Stores: uses · Storage service delegates persistence to concrete stores.
- Stores → Backends: backed by · Store implementations rely on backend abstractions such as DuckDB or memory.
- Resolve service → Corpus: merges · Resolution updates identities and merge state in the canonical corpus.
- Curate service → PRISMA filter: screens with · Curation applies PRISMA-style filtering and decision tracking.
- Curate service → Corpus: updates · Human decisions change corpus curation state.
- Snapshot service → Corpus: captures · Snapshots freeze corpus state for reproducibility.
- Read service → Corpus: reads · Read views are derived from the stored corpus.
- Networks facade → Projectors: builds · Facade coordinates the main bibliometric projections.
- Projectors → Analyzer: analyzes · Projected graphs feed metric computation and analysis.
- Projectors → Clusters: clusters · Projected graphs are grouped into communities.
- Projectors → Decorate: decorates · Projection outputs are annotated with derived metrics.
- Networks facade → Exporters: exports to · Network outputs are handed off to CSV and GraphML exporters.
- Corpus → Schemas: conforms to · The domain model is shaped by shared schemas and contracts.
- Read service → Exporters: prepares · Read-side views can be handed to export formats.
