# thesentient-field-initiative/sgi-persistence architecture

> Architecture diagram of the GitHub repository thesentient-field-initiative/sgi-persistence, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thesentient-field-initiative/sgi-persistence
- Repository: https://github.com/thesentient-field-initiative/sgi-persistence

## Overview

1. **What this repository is**
   - This is a Python 3.10+ research codebase for a reproducible empirical program on “organizational persistence.”
   - The architecture is not a product/app stack; it is a pipeline of simulation, metric extraction, geometric formalism, validation, and publication artifacts.

2. **Core execution boundary: `src/` vs `experiments/`**
   - `src/` contains reusable library code: system models, embeddings, metrics, and geometry.
   - `experiments/` contains phase- and study-specific runners that instantiate systems, call the core library, and write result artifacts.
   - This is the main boundary in the repo: library logic is separated from experiment orchestration.

3. **Simulation subsystems: `src/systems/`**
   - The repository models four main system families: distributed coordination, immune signaling, ant colony stigmergy, and institutional trust networks.
   - Each family has its own study module, and the `resistant/` subpackage provides variants used in intervention and stress experiments.
   - These systems are the source of raw organizational dynamics that downstream metrics analyze.

4. **Representation layer: `src/embeddings/`**
   - The embedding modules convert system state into domain-specific organizational representations.
   - These are the inputs to the metric and geometry layers, and they form the bridge from simulated dynamics to measurable structure.
   - The README’s “organizational geometry” claims are grounded here.

5. **Metric layer: `src/metrics/`**
   - `contracts.py`, `registry.py`, and `validators.py` define metric definitions, lookup, and consistency checks.
   - This layer enforces canonical metric naming and traceability, which is important because many docs focus on metric legitimacy and survival.
   - The metric boundary is central to the repo’s reproducibility story: experiments are only meaningful if their observables are contract-compliant.

6. **Geometry layer: `src/geometry/`**
   - `connection_formalism.py` and `discrete_transport.py` are the main architectural core for phase 002B/002C.
   - The subpackages `noncommutative/`, `path_groupoids/`, and `replay_algebra/` are scaffolded future extensions, indicating an evolving formalism around transport and replay equivalence.
   - Architecturally, this layer sits between embeddings/metrics and higher-level experiment interpretation.

7. **Experiment pipelines**
   - `experiments/phase_001`, `phase_002`, `validation`, and `interventions` are separate experiment families with different purposes.
   - Phase folders correspond to the main research progression; validation scripts test robustness, scalability, and failure modes; intervention scripts probe collapse/recovery and coupling behavior.
   - Each experiment family emits JSON results files, creating a clear data-producing boundary.

8. **Data flow and artifacts**
   - Raw outputs live under `data/raw/`, curated canonical datasets under `data/canonical/`, and experiment-specific results under `experiments/**/results/`.
   - Aggregated publication tables are stored in `results/tables/`, while some older or exploratory outputs are preserved in `archive/`.
   - The flow is effectively: simulation/experiment → JSON results → canonicalization/summary tables → manuscript figures and prose.

9. **Validation and reproducibility**
   - `tests/` covers basis invariance, connection formalism, discrete transport, metric consistency, figure generation, and reproducibility.
   - `reproduction/` contains external review and minimal-demo paths, which are separate from the main experimental pipeline and intended to verify claims independently.
   - The repo also includes schema files and external-review checklists, reinforcing that validation is a first-class subsystem rather than an afterthought.

10. **Publication and reporting**
   - `papers/` contains phase-specific manuscript drafts and appendices; `docs/` holds the methodological, theoretical, and audit trail documentation.
   - `scripts/` generate tables, figures, and artifact hashes, connecting experiment outputs to publication-ready assets.
   - `docs/` is especially important for architecture because many conceptual boundaries, failure modes, and claim-language constraints are documented there.

11. **Project lifecycle and release boundaries**
   - `releases/` captures frozen phase outputs, separating canonical milestone states from ongoing development.
   - `archive/` contains deprecated, exploratory, and superseded studies, which should be treated as historical context rather than active architecture.
   - This makes the repository a versioned research program with explicit phase transitions.

12. **Operational tooling**
   - The repository is built around standard Python tooling: `pyproject.toml`, `requirements.txt`, `Makefile`, and GitHub Actions in `.github/workflows/tests.yml`.
   - CI and local automation support the same core boundary: verify code, run experiments/tests, and regenerate figures/tables.
   - External services are minimal; the main operational dependency is the Python execution environment and the repo’s own artifact generation chain.

Main entry points: [`data`](https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/data) (Data artifacts), [`src/geometry`](https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/src/geometry) (Geometry), [`src/metrics`](https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/src/metrics) (Metrics), [`experiments/validation`](https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/experiments/validation) (Validation)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_src["Library core"]
  node_node_systems["System models<br/>simulation family"]
  node_node_distributed["Distributed<br/>system model<br/>[study_001.py]"]
  node_node_immune["Immune<br/>system model"]
  node_node_ant["Ant colony<br/>system model"]
  node_node_institution["Institution<br/>system model"]
  node_node_resistant["Resistant variants<br/>stress variants"]
  node_node_embeddings["Embeddings<br/>representation layer"]
  node_node_metrics["Metrics<br/>contract layer"]
  node_node_geometry{{"Geometry<br/>formalism layer"}}
  node_node_transport["Transport core<br/>geometry core"]
  node_node_replay["Replay extensions<br/>formalism scaffold"]
end

subgraph group_group_experiments["Experiment pipelines"]
  node_node_phase001["Phase 001<br/>experiment runner"]
  node_node_phase002["Phase 002<br/>experiment runner"]
  node_node_validation["Validation<br/>experiment suite"]
  node_node_interventions["Interventions<br/>experiment suite"]
  node_node_data[("Data artifacts<br/>result stores")]
end

subgraph group_group_publication["Publication and reporting"]
  node_node_tables["Tables<br/>publication artifacts"]
  node_node_papers["Papers<br/>manuscripts"]
  node_node_scripts["Generation scripts<br/>build tooling"]
end

subgraph group_group_quality["Validation and reproducibility"]
  node_node_tests["Tests<br/>verification suite"]
  node_node_repro["Reproduction<br/>external review"]
  node_node_schema["Schemas<br/>contract spec"]
  node_node_docs["Docs<br/>methodology and audits"]
end

node_node_systems -->|"state encoding"| node_node_embeddings
node_node_embeddings -->|"observables"| node_node_metrics
node_node_metrics -->|"formal input"| node_node_geometry
node_node_geometry -->|"core method"| node_node_transport
node_node_geometry -.->|"extends"| node_node_replay
node_node_systems -->|"variants"| node_node_resistant
node_node_systems -->|"studied by"| node_node_phase001
node_node_geometry -->|"evaluated by"| node_node_phase002
node_node_metrics -->|"checked by"| node_node_validation
node_node_embeddings -->|"stress tested"| node_node_validation
node_node_systems -->|"perturbed by"| node_node_interventions
node_node_data -->|"summarized as"| node_node_tables
node_node_phase001 -->|"writes"| node_node_data
node_node_phase002 -->|"writes"| node_node_data
node_node_validation -->|"writes"| node_node_data
node_node_interventions -->|"writes"| node_node_data
node_node_data -->|"informs"| node_node_papers
node_node_tables -->|"cited in"| node_node_papers
node_node_scripts -->|"builds"| node_node_tables
node_node_scripts -->|"builds"| node_node_papers
node_node_schema -->|"constrains"| node_node_validation
node_node_docs -->|"specifies"| node_node_metrics
node_node_docs -->|"specifies"| node_node_geometry
node_node_repro -->|"independent check"| node_node_validation
node_node_tests -->|"verifies"| node_node_geometry
node_node_tests -->|"verifies"| node_node_metrics
node_node_tests -->|"verifies"| node_node_scripts
node_node_data -.->|"shared inputs"| node_node_repro

click node_node_systems "https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/src/systems"
click node_node_distributed "https://github.com/thesentient-field-initiative/sgi-persistence/blob/main/src/systems/distributed/study_001.py"
click node_node_immune "https://github.com/thesentient-field-initiative/sgi-persistence/blob/main/src/systems/immune/study_001c_immune.py"
click node_node_ant "https://github.com/thesentient-field-initiative/sgi-persistence/blob/main/src/systems/ant_colony/study_001b_colony.py"
click node_node_institution "https://github.com/thesentient-field-initiative/sgi-persistence/blob/main/src/systems/institution/study_001d_institution.py"
click node_node_resistant "https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/src/systems/resistant"
click node_node_embeddings "https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/src/embeddings"
click node_node_metrics "https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/src/metrics"
click node_node_geometry "https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/src/geometry"
click node_node_transport "https://github.com/thesentient-field-initiative/sgi-persistence/blob/main/src/geometry/connection_formalism.py"
click node_node_replay "https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/src/geometry/replay_algebra"
click node_node_phase001 "https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/experiments/phase_001"
click node_node_phase002 "https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/experiments/phase_002"
click node_node_validation "https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/experiments/validation"
click node_node_interventions "https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/experiments/interventions"
click node_node_data "https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/data"
click node_node_tables "https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/results/tables"
click node_node_papers "https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/papers"
click node_node_scripts "https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/scripts"
click node_node_tests "https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/tests"
click node_node_repro "https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/reproduction"
click node_node_schema "https://github.com/thesentient-field-initiative/sgi-persistence/blob/main/schemas/experiment_result.schema.json"
click node_node_docs "https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/docs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_node_systems,node_node_distributed,node_node_immune,node_node_ant,node_node_institution,node_node_resistant,node_node_embeddings,node_node_metrics,node_node_geometry,node_node_transport,node_node_replay toneBlue
class node_node_phase001,node_node_phase002,node_node_validation,node_node_interventions,node_node_data toneAmber
class node_node_tables,node_node_papers,node_node_scripts toneMint
class node_node_tests,node_node_repro,node_node_schema,node_node_docs toneRose
```

## Components

### Library core

Reusable simulation, representation, metric, and geometry code under src/

- [System models](https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/src/systems): simulation family · Core organizational dynamics across distributed, immune, ant colony, and institutional studies · `src/systems`
- [Distributed](https://github.com/thesentient-field-initiative/sgi-persistence/blob/main/src/systems/distributed/study_001.py): system model · Distributed coordination study model · `src/systems/distributed/study_001.py`
- [Immune](https://github.com/thesentient-field-initiative/sgi-persistence/blob/main/src/systems/immune/study_001c_immune.py): system model · Immune signaling study model · `src/systems/immune/study_001c_immune.py`
- [Ant colony](https://github.com/thesentient-field-initiative/sgi-persistence/blob/main/src/systems/ant_colony/study_001b_colony.py): system model · Stigmergic colony study model · `src/systems/ant_colony/study_001b_colony.py`
- [Institution](https://github.com/thesentient-field-initiative/sgi-persistence/blob/main/src/systems/institution/study_001d_institution.py): system model · Institutional trust network study model · `src/systems/institution/study_001d_institution.py`
- [Resistant variants](https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/src/systems/resistant): stress variants · Adaptive and sparse variants used for intervention and collapse experiments · `src/systems/resistant`
- [Embeddings](https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/src/embeddings): representation layer · Domain-specific organizational representations bridging dynamics to observables · `src/embeddings`
- [Metrics](https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/src/metrics): contract layer · Canonical metric definitions, registry lookup, and validation rules · `src/metrics`
- [Geometry](https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/src/geometry): formalism layer · Connection formalism, discrete transport, and future transport extensions · `src/geometry`
- [Transport core](https://github.com/thesentient-field-initiative/sgi-persistence/blob/main/src/geometry/connection_formalism.py): geometry core · Main transport and connection implementation for phase 002B/002C · `src/geometry/connection_formalism.py`
- [Replay extensions](https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/src/geometry/replay_algebra): formalism scaffold · Noncommutative, path groupoid, and replay algebra extension points · `src/geometry/replay_algebra`

### Experiment pipelines

Phase, validation, and intervention runners that produce JSON outputs

- [Phase 001](https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/experiments/phase_001): experiment runner · Early phase studies and cross-domain entanglement experiments · `experiments/phase_001`
- [Phase 002](https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/experiments/phase_002): experiment runner · Transport-focused phase studies and phase result generation · `experiments/phase_002`
- [Validation](https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/experiments/validation): experiment suite · Robustness, scalability, identifiability, and failure-mode studies · `experiments/validation`
- [Interventions](https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/experiments/interventions): experiment suite · Collapse, coupling, recovery, and resistant-system intervention studies · `experiments/interventions`
- [Data artifacts](https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/data): result stores · Raw, canonical, and experiment result JSON outputs · `data`

### Publication and reporting

Manuscripts, figures, tables, and release artifacts

- [Tables](https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/results/tables): publication artifacts · Aggregated CSV, Markdown, and LaTeX summary tables · `results/tables`
- [Papers](https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/papers): manuscripts · Phase-specific manuscript drafts, results, and appendices · `papers`
- [Generation scripts](https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/scripts): build tooling · Figure, table, and artifact-hash generation utilities · `scripts`

### Validation and reproducibility

Tests, external review paths, schemas, and auditing support

- [Tests](https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/tests): verification suite · Basis invariance, transport, metric, figure, and reproducibility tests · `tests`
- [Reproduction](https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/reproduction): external review · Minimal demo and external review verification paths · `reproduction`
- [Schemas](https://github.com/thesentient-field-initiative/sgi-persistence/blob/main/schemas/experiment_result.schema.json): contract spec · Experiment result schema and validation contract · `schemas/experiment_result.schema.json`
- [Docs](https://github.com/thesentient-field-initiative/sgi-persistence/tree/main/docs): methodology and audits · Methodological, theoretical, audit, and policy documentation · `docs`

## Connections

- System models → Embeddings: state encoding · System dynamics are encoded into organizational representations
- Embeddings → Metrics: observables · Embedded states feed canonical metric extraction
- Metrics → Geometry: formal input · Metric definitions and observables support geometric formalism
- Geometry → Transport core: core method · Connection formalism and transport are implemented here
- Geometry → Replay extensions: extends · Future formal extensions scaffold replay and groupoid semantics
- System models → Resistant variants: variants · Resistant systems provide altered dynamics for stress studies
- System models → Phase 001: studied by · Phase 001 runs source studies and cross-domain experiments
- Geometry → Phase 002: evaluated by · Phase 002 exercises transport-related studies
- Metrics → Validation: checked by · Validation tests metric identifiability, survival, and consistency
- Embeddings → Validation: stress tested · Validation probes representation dependence and reconstruction limits
- System models → Interventions: perturbed by · Intervention studies probe collapse, coupling, and recovery
- Data artifacts → Tables: summarized as · Raw and canonical outputs are aggregated into publication tables
- Phase 001 → Data artifacts: writes · Phase 001 emits JSON result artifacts
- Phase 002 → Data artifacts: writes · Phase 002 emits phase result artifacts
- Validation → Data artifacts: writes · Validation scripts produce result JSONs
- Interventions → Data artifacts: writes · Intervention runners produce result JSONs
- Data artifacts → Papers: informs · Canonical results support manuscript drafting
- Tables → Papers: cited in · Tables feed figures, results, and appendices
- Generation scripts → Tables: builds · Scripts generate tables from experiment outputs
- Generation scripts → Papers: builds · Scripts generate figures and publication assets
- Schemas → Validation: constrains · Schemas define acceptable experiment result structure
- Docs → Metrics: specifies · Documentation defines metric contracts and traceability
- Docs → Geometry: specifies · Documentation captures formalism, theory, and failure modes
- Reproduction → Validation: independent check · External review paths verify claims separately from main experiments
- Tests → Geometry: verifies · Tests cover basis invariance and transport behavior
- Tests → Metrics: verifies · Tests cover metric consistency and reproducibility
- Tests → Generation scripts: verifies · Tests include figure generation and reproducibility checks
- Data artifacts → Reproduction: shared inputs · External review consumes frozen result artifacts and expectations
