How thesentient-field-initiative/sgi-persistence works
1. What this repository is
- This is a Python 3.10+ research codebase for a reproducible empirical program on “organizational persistence.”
- The architecture is not a product/app stack; it is a pipeline of simulation, metric extraction, geometric formalism, validation, and publication artifacts.
2. Core execution boundary: src/ vs experiments/
- src/ contains reusable library code: system models, embeddings, metrics, and geometry.
- experiments/ contains phase- and study-specific runners that instantiate systems, call the core library, and write result artifacts.
- This is the main boundary in the repo: library logic is separated from experiment orchestration.
3. Simulation subsystems: src/systems/
- The repository models four main system families: distributed coordination, immune signaling, ant colony stigmergy, and institutional trust networks.
- Each family has its own study module, and the resistant/ subpackage provides variants used in intervention and stress experiments.
- These systems are the source of raw organizational dynamics that downstream metrics analyze.
4. Representation layer: src/embeddings/
- The embedding modules convert system state into domain-specific organizational representations.
- These are the inputs to the metric and geometry layers, and they form the bridge from simulated dynamics to measurable structure.
- The README’s “organizational geometry” claims are grounded here.
5. Metric layer: src/metrics/
- contracts.py, registry.py, and validators.py define metric definitions, lookup, and consistency checks.
- This layer enforces canonical metric naming and traceability, which is important because many docs focus on metric legitimacy and survival.
- The metric boundary is central to the repo’s reproducibility story: experiments are only meaningful if their observables are contract-compliant.
6. Geometry layer: src/geometry/
- connection_formalism.py and discrete_transport.py are the main architectural core for phase 002B/002C.
- The subpackages noncommutative/, path_groupoids/, and replay_algebra/ are scaffolded future extensions, indicating an evolving formalism around transport and replay equivalence.
- Architecturally, this layer sits between embeddings/metrics and higher-level experiment interpretation.
7. Experiment pipelines
- experiments/phase_001, phase_002, validation, and interventions are separate experiment families with different purposes.
- Phase folders correspond to the main research progression; validation scripts test robustness, scalability, and failure modes; intervention scripts probe collapse/recovery and coupling behavior.
- Each experiment family emits JSON results files, creating a clear data-producing boundary.
8. Data flow and artifacts
- Raw outputs live under data/raw/, curated canonical datasets under data/canonical/, and experiment-specific results under experiments/**/results/.
- Aggregated publication tables are stored in results/tables/, while some older or exploratory outputs are preserved in archive/.
- The flow is effectively: simulation/experiment → JSON results → canonicalization/summary tables → manuscript figures and prose.
9. Validation and reproducibility
- tests/ covers basis invariance, connection formalism, discrete transport, metric consistency, figure generation, and reproducibility.
- reproduction/ contains external review and minimal-demo paths, which are separate from the main experimental pipeline and intended to verify claims independently.
- The repo also includes schema files and external-review checklists, reinforcing that validation is a first-class subsystem rather than an afterthought.
10. Publication and reporting
- papers/ contains phase-specific manuscript drafts and appendices; docs/ holds the methodological, theoretical, and audit trail documentation.
- scripts/ generate tables, figures, and artifact hashes, connecting experiment outputs to publication-ready assets.
- docs/ is especially important for architecture because many conceptual boundaries, failure modes, and claim-language constraints are documented there.
11. Project lifecycle and release boundaries
- releases/ captures frozen phase outputs, separating canonical milestone states from ongoing development.
- archive/ contains deprecated, exploratory, and superseded studies, which should be treated as historical context rather than active architecture.
- This makes the repository a versioned research program with explicit phase transitions.
12. Operational tooling
- The repository is built around standard Python tooling: pyproject.toml, requirements.txt, Makefile, and GitHub Actions in .github/workflows/tests.yml.
- CI and local automation support the same core boundary: verify code, run experiments/tests, and regenerate figures/tables.
- External services are minimal; the main operational dependency is the Python execution environment and the repo’s own artifact generation chain.
Main entry points: data Data artifacts · geometry Geometry · metrics Metrics · validation Validation
Components 23
Library core
Reusable simulation, representation, metric, and geometry code under src/
- System models · Core organizational dynamics across distributed, immune, ant colony, and institutional studies
src/systems - Distributed · Distributed coordination study model
src/systems/distributed/study_001.py - Immune · Immune signaling study model
src/systems/immune/study_001c_immune.py - Ant colony · Stigmergic colony study model
src/systems/ant_colony/study_001b_colony.py - Institution · Institutional trust network study model
src/systems/institution/study_001d_institution.py - Resistant variants · Adaptive and sparse variants used for intervention and collapse experiments
src/systems/resistant - Embeddings · Domain-specific organizational representations bridging dynamics to observables
src/embeddings - Metrics · Canonical metric definitions, registry lookup, and validation rules
src/metrics - Geometry · Connection formalism, discrete transport, and future transport extensions
src/geometry - Transport core · Main transport and connection implementation for phase 002B/002C
src/geometry/connection_formalism.py - Replay extensions · Noncommutative, path groupoid, and replay algebra extension points
src/geometry/replay_algebra
Experiment pipelines
Phase, validation, and intervention runners that produce JSON outputs
- Phase 001 · Early phase studies and cross-domain entanglement experiments
experiments/phase_001 - Phase 002 · Transport-focused phase studies and phase result generation
experiments/phase_002 - Validation · Robustness, scalability, identifiability, and failure-mode studies
experiments/validation - Interventions · Collapse, coupling, recovery, and resistant-system intervention studies
experiments/interventions - Data artifacts · Raw, canonical, and experiment result JSON outputs
data
Publication and reporting
Manuscripts, figures, tables, and release artifacts
- Tables · Aggregated CSV, Markdown, and LaTeX summary tables
results/tables - Papers · Phase-specific manuscript drafts, results, and appendices
papers - Generation scripts · Figure, table, and artifact-hash generation utilities
scripts
Validation and reproducibility
Tests, external review paths, schemas, and auditing support
- Tests · Basis invariance, transport, metric, figure, and reproducibility tests
tests - Reproduction · Minimal demo and external review verification paths
reproduction - Schemas · Experiment result schema and validation contract
schemas/experiment_result.schema.json - Docs · Methodological, theoretical, audit, and policy documentation
docs
Connections 28
- System models → Embeddings: state encoding · System dynamics are encoded into organizational representations
- Embeddings → Metrics: observables · Embedded states feed canonical metric extraction
- Metrics → Geometry: formal input · Metric definitions and observables support geometric formalism
- Geometry → Transport core: core method · Connection formalism and transport are implemented here
- Geometry → Replay extensions: extends · Future formal extensions scaffold replay and groupoid semantics
- System models → Resistant variants: variants · Resistant systems provide altered dynamics for stress studies
- System models → Phase 001: studied by · Phase 001 runs source studies and cross-domain experiments
- Geometry → Phase 002: evaluated by · Phase 002 exercises transport-related studies
- Metrics → Validation: checked by · Validation tests metric identifiability, survival, and consistency
- Embeddings → Validation: stress tested · Validation probes representation dependence and reconstruction limits
- System models → Interventions: perturbed by · Intervention studies probe collapse, coupling, and recovery
- Data artifacts → Tables: summarized as · Raw and canonical outputs are aggregated into publication tables
- Phase 001 → Data artifacts: writes · Phase 001 emits JSON result artifacts
- Phase 002 → Data artifacts: writes · Phase 002 emits phase result artifacts
- Validation → Data artifacts: writes · Validation scripts produce result JSONs
- Interventions → Data artifacts: writes · Intervention runners produce result JSONs
- Data artifacts → Papers: informs · Canonical results support manuscript drafting
- Tables → Papers: cited in · Tables feed figures, results, and appendices
- Generation scripts → Tables: builds · Scripts generate tables from experiment outputs
- Generation scripts → Papers: builds · Scripts generate figures and publication assets
- Schemas → Validation: constrains · Schemas define acceptable experiment result structure
- Docs → Metrics: specifies · Documentation defines metric contracts and traceability
- Docs → Geometry: specifies · Documentation captures formalism, theory, and failure modes
- Reproduction → Validation: independent check · External review paths verify claims separately from main experiments
- Tests → Geometry: verifies · Tests cover basis invariance and transport behavior
- Tests → Metrics: verifies · Tests cover metric consistency and reproducibility
- Tests → Generation scripts: verifies · Tests include figure generation and reproducibility checks
- Data artifacts → Reproduction: shared inputs · External review consumes frozen result artifacts and expectations