How utsavhirani/email-action-extractor works
This project extracts business actions from an Email and, chiefly, evaluates extractor output against hand-labelled cases. The extraction workflow routes through a single entry point to either an offline keyword baseline or an optional OpenRouter LLM, returning validated actions. Separately, the evaluator loads golden cases, matches predicted actions, grades them, and builds a report; it accepts an extractor function rather than importing the extractor. Edges reflect documented or excerpted calls; no direct connection from the sample runner to evaluation is shown.
Main entry points: models.py Email and Actions · extractor.py Extraction Entry · llm_extractor.py LLM Extractor · runner.py Evaluation Runner
Components 14
Entry and Models
- Sample Runner
main.py - Extraction Entry
core/extractor.py - Email and Actions
core/models.py
Action Extraction
- Runtime Settings
core/config.py - Keyword Baseline
core/stub_extractor.py - LLM Extractor
core/llm_extractor.py - Prompt Builder
core/prompts.py
Evaluation
- Golden Cases
evaluator/data/golden.json - Dataset Loader
evaluator/dataset.py - Evaluation Runner
evaluator/runner.py - Action Matching
evaluator/matching.py - Scoring Metrics
evaluator/metrics.py
External Integration
- OpenRouter
Other components
- Caller
Connections 20
- Caller → Extraction Entry: calls
README.md - Sample Runner → Extraction Entry: calls
main.py - Sample Runner → Email and Actions: builds email
main.py - Extraction Entry → Runtime Settings: checks mode
core/extractor.py - Extraction Entry → Keyword Baseline: dispatches to
core/extractor.py - Extraction Entry → LLM Extractor: dispatches to
core/extractor.py - Extraction Entry → Email and Actions: uses types
core/extractor.py - Keyword Baseline → Email and Actions: builds actions
core/stub_extractor.py - LLM Extractor → Runtime Settings: reads settings
core/llm_extractor.py - LLM Extractor → Prompt Builder: builds prompts
core/llm_extractor.py - LLM Extractor → Email and Actions: validates output
core/llm_extractor.py - LLM Extractor → OpenRouter: sends request
core/llm_extractor.py - Prompt Builder → Email and Actions: formats email
core/prompts.py - Golden Cases → Dataset Loader: loaded by
evaluator/dataset.py - Dataset Loader → Email and Actions: validates cases
evaluator/dataset.py - Evaluation Runner → Action Matching: matches actions
evaluator/runner.py - Evaluation Runner → Scoring Metrics: grades and aggregates
evaluator/runner.py - Evaluation Runner → Email and Actions: uses types
evaluator/runner.py - Scoring Metrics → Action Matching: scores matches
evaluator/metrics.py - Scoring Metrics → Email and Actions: grades fields
evaluator/metrics.py