How eulando/defending-against-cross-channel-social-engineering-through-agentic-frameworks works
This repository supports two research workflows: generating fictional, coordinated multi-channel attack scenarios with Gemini, and evaluating message-level classifiers over a folder of text files. Generation produces scenario documents; the separate training script cleans inputs, invokes Claude and/or GPT-4o, aggregates per-message results, and writes CSV summaries. The README documents the generator workflow; the sampled classifier source shows its cleaning, model calls, and CSV/reporting stages. The checked-in scenario examples are data, but no sampled code establishes that either workflow reads them.
Main entry points: model-training.py Classification runner · generate_scenarios.py Scenario generator
Components 16
Scenario generation
- Scenario specification
generate_scenarios.py - Scenario generator
generate_scenarios.py - Scenario documents
generate_scenarios.py
Classification pipeline
- Message inputs
- Input cleaning
model-training.py - Classification runner
model-training.py - Claude classifier
model-training.py - GPT-4o classifier
model-training.py
Evaluation results
- Scenario aggregation
model-training.py - CSV formatting
model-training.py - Results CSV
model-training.py - Evaluation summary
model-training.py
Other components
- Researcher
- Gemini API
- Anthropic API
- OpenAI API
Connections 17
- Researcher → Scenario generator: requests scenarios
README.md - Scenario generator → Scenario specification: uses prompt
generate_scenarios.py - Scenario generator → Gemini API: generates content
generate_scenarios.py - Gemini API → Scenario generator: returns content
generate_scenarios.py - Scenario generator → Scenario documents: writes documents
generate_scenarios.py - Researcher → Classification runner: runs analysis
model-training.py - Classification runner → Input cleaning: cleans folder
model-training.py - Classification runner → Message inputs: loads text files
model-training.py - Classification runner → Claude classifier: classifies messages
model-training.py - Claude classifier → Anthropic API: requests verdict
model-training.py - Classification runner → GPT-4o classifier: classifies messages
model-training.py - GPT-4o classifier → OpenAI API: requests verdict
model-training.py - Classification runner → Scenario aggregation: collects results
model-training.py - Scenario aggregation → CSV formatting: groups results
model-training.py - CSV formatting → Results CSV: formats rows
model-training.py - Classification runner → Results CSV: saves CSV
model-training.py - Classification runner → Evaluation summary: prints report
model-training.py