# samarth6/failure-forensics-tool architecture

> Architecture diagram of the GitHub repository samarth6/failure-forensics-tool, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/samarth6/failure-forensics-tool
- Repository: https://github.com/samarth6/failure-forensics-tool

## Overview

Failure Forensics Tool runs insurance-claim documents through a four-stage AI pipeline, records each step, diagnoses the earliest independently failing step, and verifies suspected causes by replaying downstream work with a correction. Reviewers can turn confirmed failures into step-level regression cases and inspect runs, evidence, and analytics in the explorer. The diagram follows the documented workflow and sampled references; some API and frontend wiring is represented at subsystem level because their full implementations were not excerpted.

Main entry points: [`ffx/pipeline/runner.py`](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/pipeline/runner.py) (Pipeline runner), [`web/src/App.tsx`](https://github.com/samarth6/failure-forensics-tool/blob/main/web/src/App.tsx) (Trace explorer), [`ffx/analyzer/diagnose.py`](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/analyzer/diagnose.py) (Root-cause diagnosis), [`ffx/api.py`](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/api.py) (Explorer API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_pipeline["Claim pipeline"]
  node_intake["Intake<br/>[steps.py]"]
  node_extraction["Extraction<br/>[steps.py]"]
  node_classification["Classification<br/>[steps.py]"]
  node_summarization["Summarization<br/>[steps.py]"]
  node_runner["Pipeline runner<br/>[runner.py]"]
  node_corpus["Failure corpus<br/>[generate.py]"]
  node_localmodel["Local inference<br/>[ollama_client.py]"]
end

subgraph group_analysis["Forensic analysis"]
  node_signals["Deterministic signals<br/>[signals.py]"]
  node_grounding["Value grounding<br/>[grounding.py]"]
  node_judge["LLM judge<br/>[judge.py]"]
  node_diagnosis["Root-cause diagnosis<br/>[diagnose.py]"]
  node_counterfactual["Counterfactual replay<br/>[counterfactual.py]"]
end

subgraph group_observability["Tracing and storage"]
  node_tracecapture["Trace capture<br/>[tracer.py]"]
  node_tracestore[("Trace store<br/>[store.py]")]
  node_otel["OTel export<br/>[otel.py]"]
end

subgraph group_evaluation["Regression evaluation"]
  node_evalgeneration["Case generation<br/>[generate.py]"]
  node_evalcases[("Eval cases<br/>[cases.py]")]
  node_evalrunner["Regression runner<br/>[runner.py]"]
  node_analytics["Eval analytics<br/>[analytics.py]"]
end

subgraph group_explorer["Review explorer"]
  node_api["Explorer API<br/>[api.py]"]
  node_frontend["Trace explorer<br/>[App.tsx]"]
  node_traceviews["Trace views<br/>[TraceGraph.tsx]"]
  node_flagpanel["Review flags<br/>[FlagPanel.tsx]"]
  node_evalview["Eval views<br/>[EvalsView.tsx]"]
  node_analyticsview["Analytics view<br/>[AnalyticsView.tsx]"]
end

node_claimant["Claim document"]
node_reviewer(("Human reviewer"))
node_cloudjudges{{"Cloud judge services"}}

node_claimant -->|"runs through"| node_runner
node_corpus -->|"supplies documents"| node_runner
node_runner -->|"executes steps"| node_intake
node_runner -->|"executes steps"| node_extraction
node_runner -->|"executes steps"| node_classification
node_runner -->|"executes steps"| node_summarization
node_runner -->|"requests inference"| node_localmodel
node_runner -->|"records execution"| node_tracecapture
node_tracecapture -->|"persists traces"| node_tracestore
node_tracecapture -.->|"exports spans"| node_otel
node_tracestore -->|"provides trace data"| node_diagnosis
node_diagnosis -->|"assesses spans"| node_signals
node_signals -->|"checks evidence"| node_grounding
node_diagnosis -->|"adjudicates ambiguity"| node_judge
node_judge -.->|"queries providers"| node_cloudjudges
node_diagnosis -->|"tests candidates"| node_counterfactual
node_counterfactual -->|"replays downstream"| node_runner
node_reviewer -->|"reviews failures"| node_frontend
node_frontend -->|"requests data"| node_api
node_api -->|"requests diagnosis"| node_diagnosis
node_api -->|"reads and writes"| node_tracestore
node_frontend -->|"renders traces"| node_traceviews
node_frontend -->|"renders review"| node_flagpanel
node_frontend -->|"renders evaluations"| node_evalview
node_frontend -->|"renders analytics"| node_analyticsview
node_reviewer -->|"confirms failure"| node_evalgeneration
node_evalgeneration -->|"saves validated cases"| node_evalcases
node_evalcases -->|"provides cases"| node_evalrunner
node_evalrunner -->|"executes step"| node_runner
node_evalrunner -->|"records results"| node_tracestore
node_analytics -->|"queries run data"| node_tracestore
node_evalview -->|"requests eval data"| node_api
node_analyticsview -->|"requests analytics"| node_api

click node_intake "https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/pipeline/steps.py"
click node_extraction "https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/pipeline/steps.py"
click node_classification "https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/pipeline/steps.py"
click node_summarization "https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/pipeline/steps.py"
click node_runner "https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/pipeline/runner.py"
click node_corpus "https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/corpus/generate.py"
click node_localmodel "https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/llm/ollama_client.py"
click node_tracecapture "https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/tracing/tracer.py"
click node_tracestore "https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/tracing/store.py"
click node_otel "https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/tracing/otel.py"
click node_signals "https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/analyzer/signals.py"
click node_grounding "https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/analyzer/grounding.py"
click node_judge "https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/analyzer/judge.py"
click node_diagnosis "https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/analyzer/diagnose.py"
click node_counterfactual "https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/analyzer/counterfactual.py"
click node_evalgeneration "https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/evals/generate.py"
click node_evalcases "https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/evals/cases.py"
click node_evalrunner "https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/evals/runner.py"
click node_analytics "https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/evals/analytics.py"
click node_api "https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/api.py"
click node_frontend "https://github.com/samarth6/failure-forensics-tool/blob/main/web/src/App.tsx"
click node_traceviews "https://github.com/samarth6/failure-forensics-tool/blob/main/web/src/components/TraceGraph.tsx"
click node_flagpanel "https://github.com/samarth6/failure-forensics-tool/blob/main/web/src/components/FlagPanel.tsx"
click node_evalview "https://github.com/samarth6/failure-forensics-tool/blob/main/web/src/components/EvalsView.tsx"
click node_analyticsview "https://github.com/samarth6/failure-forensics-tool/blob/main/web/src/components/AnalyticsView.tsx"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_intake,node_extraction,node_classification,node_summarization,node_runner,node_corpus,node_localmodel,node_reviewer toneBlue
class node_signals,node_grounding,node_judge,node_diagnosis,node_counterfactual toneAmber
class node_tracecapture,node_tracestore,node_otel toneMint
class node_evalgeneration,node_evalcases,node_evalrunner,node_analytics toneRose
class node_api,node_frontend,node_traceviews,node_flagpanel,node_evalview,node_analyticsview,node_claimant,node_cloudjudges toneIndigo
```

## Components

### Claim pipeline

- [Intake](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/pipeline/steps.py): `ffx/pipeline/steps.py`
- [Extraction](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/pipeline/steps.py): `ffx/pipeline/steps.py`
- [Classification](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/pipeline/steps.py): `ffx/pipeline/steps.py`
- [Summarization](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/pipeline/steps.py): `ffx/pipeline/steps.py`
- [Pipeline runner](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/pipeline/runner.py): `ffx/pipeline/runner.py`
- [Failure corpus](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/corpus/generate.py): `ffx/corpus/generate.py`
- [Local inference](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/llm/ollama_client.py): `ffx/llm/ollama_client.py`

### Forensic analysis

- [Deterministic signals](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/analyzer/signals.py): `ffx/analyzer/signals.py`
- [Value grounding](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/analyzer/grounding.py): `ffx/analyzer/grounding.py`
- [LLM judge](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/analyzer/judge.py): `ffx/analyzer/judge.py`
- [Root-cause diagnosis](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/analyzer/diagnose.py): `ffx/analyzer/diagnose.py`
- [Counterfactual replay](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/analyzer/counterfactual.py): `ffx/analyzer/counterfactual.py`

### Tracing and storage

- [Trace capture](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/tracing/tracer.py): `ffx/tracing/tracer.py`
- [Trace store](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/tracing/store.py): `ffx/tracing/store.py`
- [OTel export](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/tracing/otel.py): `ffx/tracing/otel.py`

### Regression evaluation

- [Case generation](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/evals/generate.py): `ffx/evals/generate.py`
- [Eval cases](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/evals/cases.py): `ffx/evals/cases.py`
- [Regression runner](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/evals/runner.py): `ffx/evals/runner.py`
- [Eval analytics](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/evals/analytics.py): `ffx/evals/analytics.py`

### Review explorer

- [Explorer API](https://github.com/samarth6/failure-forensics-tool/blob/main/ffx/api.py): `ffx/api.py`
- [Trace explorer](https://github.com/samarth6/failure-forensics-tool/blob/main/web/src/App.tsx): `web/src/App.tsx`
- [Trace views](https://github.com/samarth6/failure-forensics-tool/blob/main/web/src/components/TraceGraph.tsx): `web/src/components/TraceGraph.tsx`
- [Review flags](https://github.com/samarth6/failure-forensics-tool/blob/main/web/src/components/FlagPanel.tsx): `web/src/components/FlagPanel.tsx`
- [Eval views](https://github.com/samarth6/failure-forensics-tool/blob/main/web/src/components/EvalsView.tsx): `web/src/components/EvalsView.tsx`
- [Analytics view](https://github.com/samarth6/failure-forensics-tool/blob/main/web/src/components/AnalyticsView.tsx): `web/src/components/AnalyticsView.tsx`

### Other components

- **Claim document**
- **Human reviewer**
- **Cloud judge services**

## Connections

- Claim document → Pipeline runner: runs through (evidence: `README.md`)
- Failure corpus → Pipeline runner: supplies documents (evidence: `README.md`)
- Pipeline runner → Intake: executes steps (evidence: `ffx/pipeline/runner.py`)
- Pipeline runner → Extraction: executes steps (evidence: `ffx/pipeline/runner.py`)
- Pipeline runner → Classification: executes steps (evidence: `ffx/pipeline/runner.py`)
- Pipeline runner → Summarization: executes steps (evidence: `ffx/pipeline/runner.py`)
- Pipeline runner → Local inference: requests inference (evidence: `ffx/pipeline/runner.py`)
- Pipeline runner → Trace capture: records execution (evidence: `ffx/pipeline/runner.py`)
- Trace capture → Trace store: persists traces (evidence: `ffx/tracing/tracer.py`)
- Trace capture → OTel export: exports spans (evidence: `README.md`)
- Trace store → Root-cause diagnosis: provides trace data (evidence: `ffx/analyzer/diagnose.py`)
- Root-cause diagnosis → Deterministic signals: assesses spans (evidence: `ffx/analyzer/diagnose.py`)
- Deterministic signals → Value grounding: checks evidence (evidence: `ffx/analyzer/signals.py`)
- Root-cause diagnosis → LLM judge: adjudicates ambiguity (evidence: `ffx/analyzer/diagnose.py`)
- LLM judge → Cloud judge services: queries providers (evidence: `README.md`)
- Root-cause diagnosis → Counterfactual replay: tests candidates (evidence: `ffx/analyzer/diagnose.py`)
- Counterfactual replay → Pipeline runner: replays downstream (evidence: `ffx/analyzer/counterfactual.py`)
- Human reviewer → Trace explorer: reviews failures (evidence: `README.md`)
- Trace explorer → Explorer API: requests data
- Explorer API → Root-cause diagnosis: requests diagnosis (evidence: `ffx/api.py`)
- Explorer API → Trace store: reads and writes (evidence: `ffx/api.py`)
- Trace explorer → Trace views: renders traces (evidence: `web/src/App.tsx`)
- Trace explorer → Review flags: renders review (evidence: `web/src/App.tsx`)
- Trace explorer → Eval views: renders evaluations (evidence: `web/src/App.tsx`)
- Trace explorer → Analytics view: renders analytics (evidence: `web/src/App.tsx`)
- Human reviewer → Case generation: confirms failure (evidence: `README.md`)
- Case generation → Eval cases: saves validated cases (evidence: `ffx/evals/generate.py`)
- Eval cases → Regression runner: provides cases (evidence: `ffx/evals/runner.py`)
- Regression runner → Pipeline runner: executes step (evidence: `ffx/evals/runner.py`)
- Regression runner → Trace store: records results (evidence: `ffx/evals/runner.py`)
- Eval analytics → Trace store: queries run data (evidence: `ffx/evals/analytics.py`)
- Eval views → Explorer API: requests eval data
- Analytics view → Explorer API: requests analytics