# sarancoding/production-reliability-deployment-field-engineering-suite architecture

> Architecture diagram of the GitHub repository sarancoding/production-reliability-deployment-field-engineering-suite, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sarancoding/production-reliability-deployment-field-engineering-suite
- Repository: https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite

## Overview

#### Purpose and Entry Points
Field Engineering Suite is a Python 3.11+ operational platform for isolated customer deployment, SLA monitoring, incident automation, and publication of postmortems/case studies. Primary operator entry points are `Makefile` (`make docker-up`, `make deploy`) and the API route layer under `src/api/routes/`. Runtime configuration begins in `.env.example` and `src/core/config.py`.

#### Lifecycle Architecture
The intended control flow is: provision → monitor → detect → respond → learn → share. Deployment hands environments to health monitoring; SLA breaches become events; incidents consume those events; resolution produces postmortems and material for externally reviewed case studies. The source package is organized around these bounded lifecycle domains:
- Deployment: `src/deploy/__init__.py`
- Health/SLA: `src/health/__init__.py`
- Incident response: `src/incident/__init__.py`
- Case-study publishing: `src/casestudy/__init__.py`

#### API and Core Runtime Boundary
FastAPI/Pydantic/Uvicorn expose operational functions through `src/api/routes/__init__.py`. Cross-cutting runtime concerns are centralized rather than embedded in lifecycle domains: configuration in `src/core/config.py`, standardized failures in `src/core/exceptions.py`, structured logs in `src/core/logging.py`, and distributed trace propagation in `src/core/tracing.py`. Trace IDs are a key boundary-crossing correlation mechanism across deployment, monitoring, and incident workflows.

#### Deployment and Isolation
The deployment domain (`src/deploy/__init__.py`) validates customer/environment inputs, applies infrastructure, gates traffic on health, registers successful customers, and rolls back failed deployments. The README specifies Terraform plus Kubernetes, including per-customer namespaces and network policies; remote Terraform state locking is expected infrastructure behavior. Production application requires a human approval gate, unlike lower environments.

#### Health, SLA, and Event Transport
Health processing in `src/health/__init__.py` aggregates customer metrics, calculates error-budget burn rates, and identifies SLA violations. Fast burn alerts page immediately; slow burn creates planning work. Domain-to-domain communication is mediated by the event bus at `src/events/bus.py`, intended to provide transactional-outbox-style reliable delivery rather than direct health-to-incident coupling. Event contracts belong under `src/events/__init__.py`.

#### Incident Learning and Publication
Incident processing (`src/incident/__init__.py`) classifies SEV1–SEV4, opens and manages mitigation/escalation loops, uses traces and metrics for diagnosis, and generates a blameless postmortem after resolution. Generated artifacts are retained in `postmortems/generated/`; durable improvement work and lessons flow into `tasks/todo.md` and `tasks/lessons.md`. Case-study assembly (`src/casestudy/__init__.py`) gathers approved operational material and requires human review before publication. PDF generation is supported by `scripts/generate_pdfs.py` into `docs/pdfs/`.

#### Material Services and Operations
The full local stack comprises the app, PostgreSQL, Redis, OpenTelemetry/OTLP, Prometheus, and Grafana. PostgreSQL is the system of record; Redis supports caching or coordination; OpenTelemetry links signals; Prometheus/Grafana provide metrics and dashboarding. Reliability validation spans unit, integration, and failure-injection tiers under `tests/unit/`, `tests/integration/`, and `tests/chaos/`, with chaos gates specifically validating rollback, breach-to-incident, postmortem generation, and end-to-end trace propagation.

Main entry points: [`src/deploy/__init__.py`](https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/src/deploy/__init__.py) (Deployment), [`src/health/__init__.py`](https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/src/health/__init__.py) (Health & SLA), [`src/incident/__init__.py`](https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/src/incident/__init__.py) (Incident response), [`src/api/routes/__init__.py`](https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/src/api/routes/__init__.py) (API routes)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Operator & API"]
  node_make["Make targets<br/>operator entry"]
  node_api["API routes<br/>FastAPI boundary<br/>[__init__.py]"]
end

subgraph group_lifecycle["Operational Lifecycle"]
  node_deploy["Deployment<br/>lifecycle domain<br/>[__init__.py]"]
  node_isolation{{"Terraform &amp; Kubernetes<br/>infrastructure"}}
  node_health["Health &amp; SLA<br/>lifecycle domain<br/>[__init__.py]"]
  node_contracts["Event contracts<br/>domain events<br/>[__init__.py]"]
  node_eventbus["Reliable event bus<br/>event transport<br/>[bus.py]"]
  node_incident["Incident response<br/>lifecycle domain<br/>[__init__.py]"]
end

subgraph group_platform["Runtime &amp; Observability"]
  node_config["Runtime config<br/>configuration<br/>[config.py]"]
  node_core["Core telemetry &amp; failures<br/>runtime services<br/>[tracing.py]"]
  node_postgres[("PostgreSQL<br/>system of record")]
  node_redis[("Redis<br/>cache &amp; coordination")]
  node_otel["OpenTelemetry / OTLP<br/>trace transport"]
  node_metrics["Prometheus &amp; Grafana<br/>metrics &amp; dashboards"]
end

subgraph group_learning["Learning &amp; Publication"]
  node_postmortems["Generated postmortems<br/>incident artifacts"]
  node_improvements["Lessons &amp; improvement work<br/>operational backlog<br/>[lessons.md]"]
  node_casestudy["Case-study assembly<br/>publication domain<br/>[__init__.py]"]
  node_pdfs["Published PDFs<br/>publication output"]
end

node_make -->|"deploys"| node_deploy
node_api -->|"operates"| node_deploy
node_api -->|"operates"| node_health
node_api -->|"operates"| node_incident
node_config -->|"configures"| node_api
node_config -->|"configures"| node_deploy
node_deploy -->|"applies infrastructure"| node_isolation
node_deploy -->|"hands off environment"| node_health
node_deploy -->|"registers customer"| node_postgres
node_health -->|"uses metrics"| node_metrics
node_health -->|"emits violations"| node_contracts
node_contracts -->|"defines messages"| node_eventbus
node_eventbus -->|"delivers breach events"| node_incident
node_incident -->|"generates after resolution"| node_postmortems
node_incident -->|"records lessons"| node_improvements
node_postmortems -->|"supplies approved material"| node_casestudy
node_casestudy -->|"publishes after review"| node_pdfs
node_core -.->|"propagates trace IDs"| node_deploy
node_core -.->|"propagates trace IDs"| node_health
node_core -.->|"correlates diagnosis"| node_incident
node_deploy -.->|"exports traces"| node_otel
node_health -.->|"exports signals"| node_otel
node_incident -.->|"queries traces"| node_otel
node_deploy -.->|"coordinates"| node_redis

click node_make "https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/Makefile"
click node_api "https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/src/api/routes/__init__.py"
click node_config "https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/src/core/config.py"
click node_core "https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/src/core/tracing.py"
click node_deploy "https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/src/deploy/__init__.py"
click node_health "https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/src/health/__init__.py"
click node_contracts "https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/src/events/__init__.py"
click node_eventbus "https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/src/events/bus.py"
click node_incident "https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/src/incident/__init__.py"
click node_improvements "https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/tasks/lessons.md"
click node_casestudy "https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/src/casestudy/__init__.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_make,node_api toneBlue
class node_deploy,node_isolation,node_health,node_contracts,node_eventbus,node_incident toneAmber
class node_config,node_core,node_postgres,node_redis,node_otel,node_metrics toneMint
class node_postmortems,node_improvements,node_casestudy,node_pdfs toneRose
```

## Components

### Operator & API

- [Make targets](https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/Makefile): operator entry · Starts local stack and runs deployments. · `Makefile`
- [API routes](https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/src/api/routes/__init__.py): FastAPI boundary · Exposes operational functions. · `src/api/routes/__init__.py`

### Operational Lifecycle

- [Deployment](https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/src/deploy/__init__.py): lifecycle domain · Validates, provisions, health-gates, registers, or rolls back customer environments. · `src/deploy/__init__.py`
- **Terraform & Kubernetes**: infrastructure · Provides isolated customer namespaces, network policies, and locked remote state.
- [Health & SLA](https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/src/health/__init__.py): lifecycle domain · Aggregates metrics, calculates burn rates, and detects SLA violations. · `src/health/__init__.py`
- [Event contracts](https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/src/events/__init__.py): domain events · `src/events/__init__.py`
- [Reliable event bus](https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/src/events/bus.py): event transport · Uses transactional-outbox-style delivery between domains. · `src/events/bus.py`
- [Incident response](https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/src/incident/__init__.py): lifecycle domain · Classifies severity, coordinates mitigation, diagnoses, and creates postmortems. · `src/incident/__init__.py`

### Runtime & Observability

- [Runtime config](https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/src/core/config.py): configuration · `src/core/config.py`
- [Core telemetry & failures](https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/src/core/tracing.py): runtime services · Standardizes logging, exceptions, and trace propagation. · `src/core/tracing.py`
- **PostgreSQL**: system of record
- **Redis**: cache & coordination
- **OpenTelemetry / OTLP**: trace transport · Carries correlation across operational workflows.
- **Prometheus & Grafana**: metrics & dashboards

### Learning & Publication

- **Generated postmortems**: incident artifacts
- [Lessons & improvement work](https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/tasks/lessons.md): operational backlog · `tasks/lessons.md`
- [Case-study assembly](https://github.com/sarancoding/production-reliability-deployment-field-engineering-suite/blob/main/src/casestudy/__init__.py): publication domain · Collects approved material for human-reviewed publication. · `src/casestudy/__init__.py`
- **Published PDFs**: publication output

## Connections

- Make targets → Deployment: deploys
- API routes → Deployment: operates
- API routes → Health & SLA: operates
- API routes → Incident response: operates
- Runtime config → API routes: configures
- Runtime config → Deployment: configures
- Deployment → Terraform & Kubernetes: applies infrastructure
- Deployment → Health & SLA: hands off environment
- Deployment → PostgreSQL: registers customer
- Health & SLA → Prometheus & Grafana: uses metrics
- Health & SLA → Event contracts: emits violations
- Event contracts → Reliable event bus: defines messages
- Reliable event bus → Incident response: delivers breach events
- Incident response → Generated postmortems: generates after resolution
- Incident response → Lessons & improvement work: records lessons
- Generated postmortems → Case-study assembly: supplies approved material
- Case-study assembly → Published PDFs: publishes after review
- Core telemetry & failures → Deployment: propagates trace IDs
- Core telemetry & failures → Health & SLA: propagates trace IDs
- Core telemetry & failures → Incident response: correlates diagnosis
- Deployment → OpenTelemetry / OTLP: exports traces
- Health & SLA → OpenTelemetry / OTLP: exports signals
- Incident response → OpenTelemetry / OTLP: queries traces
- Deployment → Redis: coordinates