How multimodal-ai-lab/defame works
DEFAME verifies multimodal claims through a dynamic fact-checking workflow: it accepts claim content, extracts and plans around checkable claims, uses tools to gather evidence, then judges and reports a verdict. Its runtime also supports benchmark evaluation and asynchronous API jobs. The README identifies a six-stage pipeline, but the excerpts do not show all stage-level control flow; edges below are limited to relationships visible in cited source or documented in the README.
Main entry points: fact_checker.py Fact Checker · tools Evidence Tools · search Search Integrations · modeling.py Language Models
Components 25
Claim Verification
- Claim Content
defame/common/content.py - Fact Checker
defame/fact_checker.py - Claim Extractor
defame/modules/claim_extractor.py - Action Planner
defame/modules/planner.py - Procedure Variants
defame/procedure - Tool Actor
defame/modules/actor.py - Verdict Judge
defame/modules/judge.py - Evidence Summarizer
defame/modules/doc_summarizer.py - Fact-check Report
defame/common/report.py
Evidence Retrieval
- Evidence Tools
defame/evidence_retrieval/tools - Searcher
defame/evidence_retrieval/tools/searcher.py - Search Integrations
defame/evidence_retrieval/integrations/search - Web Scraper
defame/evidence_retrieval/scraping/scraper.py - Social Integrations
defame/evidence_retrieval/integrations/social_media
Models and Prompts
- Language Models
defame/common/modeling.py - Prompt Templates
defame/prompts/prompts.py
API and Jobs
- HTTP API
defame/helpers/api/main.py - Job Manager
defame/helpers/api/job_manager.py - Worker Pool
defame/helpers/parallelization/pool.py - Fact-check Worker
defame/helpers/parallelization/worker.py
Evaluation
- Benchmark Evaluation
defame/eval
Other components
- Claim Submitter
- Model Providers
- Web and Search Services
- Verified Verdict
Connections 24
- Claim Submitter → Claim Content: submits
- Fact Checker → Claim Extractor: extracts claims
defame/fact_checker.py - Fact Checker → Action Planner: plans actions
defame/fact_checker.py - Fact Checker → Tool Actor: dispatches actions
defame/fact_checker.py - Fact Checker → Verdict Judge: judges claim
defame/fact_checker.py - Fact Checker → Evidence Summarizer: summarizes evidence
defame/fact_checker.py - Fact Checker → Procedure Variants: selects procedure
defame/fact_checker.py - Fact Checker → Evidence Tools: initializes tools
defame/fact_checker.py - Fact Checker → Language Models: creates model
defame/fact_checker.py - Fact Checker → Fact-check Report: produces report
- Tool Actor → Evidence Tools: runs tools
- Evidence Tools → Searcher: includes search
defame/evidence_retrieval/tools/__init__.py - Searcher → Search Integrations: queries platforms
defame/evidence_retrieval/tools/searcher.py - Search Integrations → Web and Search Services: requests results
- Web Scraper → Search Integrations: uses integrations
defame/evidence_retrieval/scraping/scraper.py - Evidence Tools → Social Integrations: optional sources
- Language Models → Model Providers: calls provider
- Prompt Templates → Language Models: formats prompts
- HTTP API → Job Manager: manages jobs
defame/helpers/api/main.py - Job Manager → Worker Pool: queues task
defame/helpers/api/job_manager.py - Worker Pool → Fact-check Worker: dispatches tasks
defame/helpers/parallelization/pool.py - Fact-check Worker → Fact Checker: runs fact-checker
defame/helpers/parallelization/worker.py - Benchmark Evaluation → Fact Checker: evaluates system
defame/eval/evaluate.py - Verdict Judge → Verified Verdict: returns verdict