How denispeskoff/2020_acl_diplomacy works
Project type and purpose
• This is a research codebase (ACL 2020) for “It Takes Two to Lie: One to Lie, and One to Listen.”
• Its goal is to train and evaluate neural models (via allennlp) to detect lies in the board‐game Diplomacy dialogue dataset.
File structure
• Top‐level items:
• configs/ – allennlp JSONNET configs for different model variants (e.g., bert+context, lstm) split into actual_lie vs. suspected_lie setups
• data/ – train/validation/test message files in JSONL
• moves/ – full game state files (unused by the core paper models)
• diplomacy/ – Python package with two subpackages:
– models/ – implementations of classifier architectures and baselines
– readers/ – data loaders for game state and message streams
• utils/ – preprocessing script to convert single‐message data into the format expected by LSTM models
• run_models.sh – wrapper to launch multiple experiments
• setup.py, requirements.txt – packaging and dependencies
• Signals: an ML‐centric pipeline built on allennlp, with clear separation of configuration, data reading, modeling, and experiment orchestration.
Extract insights from README
• Uses conda + pip install -e .
• Trains via:
allennlp train -f --include-package diplomacy -s logdir configs/...jsonnet
• Lists two major pipelines: a “single‐message” LSTM (requires preprocessing via utils) and multi‐message/context‐based classifiers.
a. Main components
• External tools: Conda/Python runtime, allennlp framework
• Configuration layer: configs/{actual_lie,suspected_lie}/*.jsonnet
• Data layer:
– Raw message data (data/*.jsonl)
– Full game JSON files (moves/*.json)
• Preprocessing: utils/singlemessage_format.py → formatted JSONL for LSTM
• Readers: diplomacy/readers/{message_reader, game_reader}
• Model implementations: diplomacy/models/*.py
• Experiment orchestrator: run_models.sh & allennlp CLI
• Output: logdir/ (metrics, model checkpoints)
b. Relationships & data flow
1. configs load hyperparameters & dataset paths
2. allennlp CLI invokes diplomacy.readers to parse data
3. Data passes to diplomacy.models classes (embedding → encoding → classifier head)
4. Training/Evaluation emits metrics and model artifacts to logdir
c. Architectural patterns & design principles
• Modular ML pipeline: clear separation of config, data I/O, model code, and experiment scripts
• Plug‐and‐play model variants via config files
• Reusable readers abstracting message vs. game state ingestion
d. Key technologies
• allennlp for training loop and configuration management
• PyTorch (via allennlp) for model implementations
• JSON/JSONL for data interchange
• Jsonnet for parameterized configs
Detailed capture
• Of the eight model files under diplomacy/models is represented as a variant node connected to the shared reader.
• That configs select which model class to instantiate.