# multimodal-ai-lab/defame architecture

> Architecture diagram of the GitHub repository multimodal-ai-lab/defame, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/multimodal-ai-lab/defame
- Repository: https://github.com/multimodal-ai-lab/defame

## Overview

DEFAME verifies multimodal claims through a dynamic fact-checking workflow: it accepts claim content, extracts and plans around checkable claims, uses tools to gather evidence, then judges and reports a verdict. Its runtime also supports benchmark evaluation and asynchronous API jobs. The README identifies a six-stage pipeline, but the excerpts do not show all stage-level control flow; edges below are limited to relationships visible in cited source or documented in the README.

Main entry points: [`defame/fact_checker.py`](https://github.com/multimodal-ai-lab/defame/blob/main/defame/fact_checker.py) (Fact Checker), [`defame/evidence_retrieval/tools`](https://github.com/multimodal-ai-lab/defame/tree/main/defame/evidence_retrieval/tools) (Evidence Tools), [`defame/evidence_retrieval/integrations/search`](https://github.com/multimodal-ai-lab/defame/tree/main/defame/evidence_retrieval/integrations/search) (Search Integrations), [`defame/common/modeling.py`](https://github.com/multimodal-ai-lab/defame/blob/main/defame/common/modeling.py) (Language Models)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_verification["Claim Verification"]
  node_content["Claim Content<br/>[content.py]"]
  node_checker["Fact Checker<br/>[fact_checker.py]"]
  node_extractor["Claim Extractor<br/>[claim_extractor.py]"]
  node_planner["Action Planner<br/>[planner.py]"]
  node_procedure["Procedure Variants"]
  node_actor["Tool Actor<br/>[actor.py]"]
  node_judge["Verdict Judge<br/>[judge.py]"]
  node_summarizer["Evidence Summarizer<br/>[doc_summarizer.py]"]
  node_report["Fact-check Report<br/>[report.py]"]
end

subgraph group_evidence["Evidence Retrieval"]
  node_toolset["Evidence Tools"]
  node_searcher["Searcher<br/>[searcher.py]"]
  node_search["Search Integrations"]
  node_scraper["Web Scraper<br/>[scraper.py]"]
  node_social["Social Integrations"]
end

subgraph group_models["Models and Prompts"]
  node_model["Language Models<br/>[modeling.py]"]
  node_prompts["Prompt Templates<br/>[prompts.py]"]
end

subgraph group_service["API and Jobs"]
  node_api["HTTP API<br/>[main.py]"]
  node_manager["Job Manager<br/>[job_manager.py]"]
  node_pool["Worker Pool<br/>[pool.py]"]
  node_worker["Fact-check Worker<br/>[worker.py]"]
end

subgraph group_evaluation["Evaluation"]
  node_benchmarks["Benchmark Evaluation"]
end

node_user(("Claim Submitter"))
node_llmservice(("Model Providers"))
node_web(("Web and Search Services"))
node_verdict["Verified Verdict"]

node_user -->|"submits"| node_content
node_checker -->|"extracts claims"| node_extractor
node_checker -->|"plans actions"| node_planner
node_checker -->|"dispatches actions"| node_actor
node_checker -->|"judges claim"| node_judge
node_checker -->|"summarizes evidence"| node_summarizer
node_checker -->|"selects procedure"| node_procedure
node_checker -->|"initializes tools"| node_toolset
node_checker -->|"creates model"| node_model
node_checker -.->|"produces report"| node_report
node_actor -.->|"runs tools"| node_toolset
node_toolset -->|"includes search"| node_searcher
node_searcher -->|"queries platforms"| node_search
node_search -.->|"requests results"| node_web
node_scraper -->|"uses integrations"| node_search
node_toolset -.->|"optional sources"| node_social
node_model -.->|"calls provider"| node_llmservice
node_prompts -.->|"formats prompts"| node_model
node_api -->|"manages jobs"| node_manager
node_manager -->|"queues task"| node_pool
node_pool -->|"dispatches tasks"| node_worker
node_worker -->|"runs fact-checker"| node_checker
node_benchmarks -->|"evaluates system"| node_checker
node_judge -.->|"returns verdict"| node_verdict

click node_content "https://github.com/multimodal-ai-lab/defame/blob/main/defame/common/content.py"
click node_checker "https://github.com/multimodal-ai-lab/defame/blob/main/defame/fact_checker.py"
click node_extractor "https://github.com/multimodal-ai-lab/defame/blob/main/defame/modules/claim_extractor.py"
click node_planner "https://github.com/multimodal-ai-lab/defame/blob/main/defame/modules/planner.py"
click node_procedure "https://github.com/multimodal-ai-lab/defame/tree/main/defame/procedure"
click node_actor "https://github.com/multimodal-ai-lab/defame/blob/main/defame/modules/actor.py"
click node_judge "https://github.com/multimodal-ai-lab/defame/blob/main/defame/modules/judge.py"
click node_summarizer "https://github.com/multimodal-ai-lab/defame/blob/main/defame/modules/doc_summarizer.py"
click node_report "https://github.com/multimodal-ai-lab/defame/blob/main/defame/common/report.py"
click node_toolset "https://github.com/multimodal-ai-lab/defame/tree/main/defame/evidence_retrieval/tools"
click node_searcher "https://github.com/multimodal-ai-lab/defame/blob/main/defame/evidence_retrieval/tools/searcher.py"
click node_search "https://github.com/multimodal-ai-lab/defame/tree/main/defame/evidence_retrieval/integrations/search"
click node_scraper "https://github.com/multimodal-ai-lab/defame/blob/main/defame/evidence_retrieval/scraping/scraper.py"
click node_social "https://github.com/multimodal-ai-lab/defame/tree/main/defame/evidence_retrieval/integrations/social_media"
click node_model "https://github.com/multimodal-ai-lab/defame/blob/main/defame/common/modeling.py"
click node_prompts "https://github.com/multimodal-ai-lab/defame/blob/main/defame/prompts/prompts.py"
click node_api "https://github.com/multimodal-ai-lab/defame/blob/main/defame/helpers/api/main.py"
click node_manager "https://github.com/multimodal-ai-lab/defame/blob/main/defame/helpers/api/job_manager.py"
click node_pool "https://github.com/multimodal-ai-lab/defame/blob/main/defame/helpers/parallelization/pool.py"
click node_worker "https://github.com/multimodal-ai-lab/defame/blob/main/defame/helpers/parallelization/worker.py"
click node_benchmarks "https://github.com/multimodal-ai-lab/defame/tree/main/defame/eval"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_content,node_checker,node_extractor,node_planner,node_procedure,node_actor,node_judge,node_summarizer,node_report toneBlue
class node_toolset,node_searcher,node_search,node_scraper,node_social toneAmber
class node_model,node_prompts toneMint
class node_api,node_manager,node_pool,node_worker toneRose
class node_benchmarks,node_user,node_llmservice,node_web,node_verdict toneIndigo
```

## Components

### Claim Verification

- [Claim Content](https://github.com/multimodal-ai-lab/defame/blob/main/defame/common/content.py): `defame/common/content.py`
- [Fact Checker](https://github.com/multimodal-ai-lab/defame/blob/main/defame/fact_checker.py): `defame/fact_checker.py`
- [Claim Extractor](https://github.com/multimodal-ai-lab/defame/blob/main/defame/modules/claim_extractor.py): `defame/modules/claim_extractor.py`
- [Action Planner](https://github.com/multimodal-ai-lab/defame/blob/main/defame/modules/planner.py): `defame/modules/planner.py`
- [Procedure Variants](https://github.com/multimodal-ai-lab/defame/tree/main/defame/procedure): `defame/procedure`
- [Tool Actor](https://github.com/multimodal-ai-lab/defame/blob/main/defame/modules/actor.py): `defame/modules/actor.py`
- [Verdict Judge](https://github.com/multimodal-ai-lab/defame/blob/main/defame/modules/judge.py): `defame/modules/judge.py`
- [Evidence Summarizer](https://github.com/multimodal-ai-lab/defame/blob/main/defame/modules/doc_summarizer.py): `defame/modules/doc_summarizer.py`
- [Fact-check Report](https://github.com/multimodal-ai-lab/defame/blob/main/defame/common/report.py): `defame/common/report.py`

### Evidence Retrieval

- [Evidence Tools](https://github.com/multimodal-ai-lab/defame/tree/main/defame/evidence_retrieval/tools): `defame/evidence_retrieval/tools`
- [Searcher](https://github.com/multimodal-ai-lab/defame/blob/main/defame/evidence_retrieval/tools/searcher.py): `defame/evidence_retrieval/tools/searcher.py`
- [Search Integrations](https://github.com/multimodal-ai-lab/defame/tree/main/defame/evidence_retrieval/integrations/search): `defame/evidence_retrieval/integrations/search`
- [Web Scraper](https://github.com/multimodal-ai-lab/defame/blob/main/defame/evidence_retrieval/scraping/scraper.py): `defame/evidence_retrieval/scraping/scraper.py`
- [Social Integrations](https://github.com/multimodal-ai-lab/defame/tree/main/defame/evidence_retrieval/integrations/social_media): `defame/evidence_retrieval/integrations/social_media`

### Models and Prompts

- [Language Models](https://github.com/multimodal-ai-lab/defame/blob/main/defame/common/modeling.py): `defame/common/modeling.py`
- [Prompt Templates](https://github.com/multimodal-ai-lab/defame/blob/main/defame/prompts/prompts.py): `defame/prompts/prompts.py`

### API and Jobs

- [HTTP API](https://github.com/multimodal-ai-lab/defame/blob/main/defame/helpers/api/main.py): `defame/helpers/api/main.py`
- [Job Manager](https://github.com/multimodal-ai-lab/defame/blob/main/defame/helpers/api/job_manager.py): `defame/helpers/api/job_manager.py`
- [Worker Pool](https://github.com/multimodal-ai-lab/defame/blob/main/defame/helpers/parallelization/pool.py): `defame/helpers/parallelization/pool.py`
- [Fact-check Worker](https://github.com/multimodal-ai-lab/defame/blob/main/defame/helpers/parallelization/worker.py): `defame/helpers/parallelization/worker.py`

### Evaluation

- [Benchmark Evaluation](https://github.com/multimodal-ai-lab/defame/tree/main/defame/eval): `defame/eval`

### Other components

- **Claim Submitter**
- **Model Providers**
- **Web and Search Services**
- **Verified Verdict**

## Connections

- Claim Submitter → Claim Content: submits
- Fact Checker → Claim Extractor: extracts claims (evidence: `defame/fact_checker.py`)
- Fact Checker → Action Planner: plans actions (evidence: `defame/fact_checker.py`)
- Fact Checker → Tool Actor: dispatches actions (evidence: `defame/fact_checker.py`)
- Fact Checker → Verdict Judge: judges claim (evidence: `defame/fact_checker.py`)
- Fact Checker → Evidence Summarizer: summarizes evidence (evidence: `defame/fact_checker.py`)
- Fact Checker → Procedure Variants: selects procedure (evidence: `defame/fact_checker.py`)
- Fact Checker → Evidence Tools: initializes tools (evidence: `defame/fact_checker.py`)
- Fact Checker → Language Models: creates model (evidence: `defame/fact_checker.py`)
- Fact Checker → Fact-check Report: produces report
- Tool Actor → Evidence Tools: runs tools
- Evidence Tools → Searcher: includes search (evidence: `defame/evidence_retrieval/tools/__init__.py`)
- Searcher → Search Integrations: queries platforms (evidence: `defame/evidence_retrieval/tools/searcher.py`)
- Search Integrations → Web and Search Services: requests results
- Web Scraper → Search Integrations: uses integrations (evidence: `defame/evidence_retrieval/scraping/scraper.py`)
- Evidence Tools → Social Integrations: optional sources
- Language Models → Model Providers: calls provider
- Prompt Templates → Language Models: formats prompts
- HTTP API → Job Manager: manages jobs (evidence: `defame/helpers/api/main.py`)
- Job Manager → Worker Pool: queues task (evidence: `defame/helpers/api/job_manager.py`)
- Worker Pool → Fact-check Worker: dispatches tasks (evidence: `defame/helpers/parallelization/pool.py`)
- Fact-check Worker → Fact Checker: runs fact-checker (evidence: `defame/helpers/parallelization/worker.py`)
- Benchmark Evaluation → Fact Checker: evaluates system (evidence: `defame/eval/evaluate.py`)
- Verdict Judge → Verified Verdict: returns verdict