How priyanshu-profile/genteel-negotiator works
Project type and purpose
• This is a research‐oriented machine‐learning system for polite negotiation dialogue, combining a mixture‐of‐experts architecture with reinforcement learning to train a negotiation agent (the “GENTEEL-NEGOTIATOR”).
• It provides data‐construction pipelines, a training loop (main.py), a specialized reward module (rewards_main.py), and built‐in evaluation metrics (BLEU, CIDEr, ROUGE, Word2Vec etc.).
File structure
Top level:
• GENTEEL-NEGOTIATOR/ – core code
• Sample_Data/ – example datasets
• README.md, GENTEEL-NEGOTIATOR_Appendix.pdf
Inside GENTEEL-NEGOTIATOR/:
• data/ – data ingestion & preprocessing
– ConstructDataset/, ConstructGraph/, ConstructSimulatorData/
– IND/ and NegoChat/ raw samples
• src/ – model components & utilities
– dataloader.py, model.py, classifiers.py, rewards.py, utils.py
• metrics/ – evaluation suite
– metric_utils.py, metrics.py, pycocoevalcap/ (BLEU, CIDEr, METEOR, ROUGE), word2vec/
• rewards/ – reward computation data loader & calculators
• main.py – orchestrates training & evaluation
• rewards_main.py – orchestrates reward‐based simulations
README
• Confirms: “LLM-Enhanced Mixture-of-Expert-based Reinforcement Learning”
• Sample data provided for IND and NeGoChat
• Paper reference to politeness markers & negotiation dialogue
a. Main components:
1. Data Ingestion & Preprocessing
• Raw data (IND, NeGoChat)
• ConstructDataset, ConstructGraph, ConstructSimulatorData modules
2. DataLoader & Feature Extraction
• src/dataloader.py, src/utils.py
3. Model Architecture
• Mixture-of-Experts policy network (src/model.py)
• Classifiers for politeness or other dialogue acts (src/classifiers.py)
4. Reinforcement Learning Engine
• Training loop in main.py
• Interaction with simulated environment (uses ConstructSimulatorData)
5. Reward Modules
• Custom rewards (rewards/construct_dataset.py, rewards/rewards_dataloader.py)
• rewards_main.py to kickoff reward‐driven simulations
6. Evaluation Suite
• In‐training metrics (metrics/metric_utils.py, metrics/metrics.py)
• Standard NLP metrics (pycocoevalcap, word2vec)
b. Relationships & interactions:
• Raw Data → Data Construction (graph & dataset constructors) → DataLoader
• DataLoader → Model (Mixture-of-Experts + LLM embeddings)
• Model ⇄ Environment (policy generates utterances → environment returns state/reward)
• Environment uses Reward Modules to compute reward signals
• Training Loop updates Model parameters using rewards
• Final model outputs → Evaluation Suite → Metric reports
c. Architectural patterns & design principles:
• Pipeline pattern for data processing
• Modular separation: data / model / reward / metrics
• Reinforcement Learning loop decoupled from data constructors
• Mixture-of-Experts layer for modular policy components
d. Key technologies & frameworks:
• Python (all modules)
• PyTorch or TensorFlow (implied in src/model.py)
• pycocoevalcap for standard NLP metrics
• Word2Vec/Glove conversion scripts