How tanmaygoyal258/prompt_optimization_slate_bandits works
Project Type and Purpose
• This is a research-oriented, CLI-driven Python tool for running prompt‐optimization experiments as described in “Efficient Algorithms for Logistic Slate Contextual Bandits with Bandit Feedback.”
• It is not a full‐stack web app but an open‐source experimental framework that orchestrates data loading, an environment simulator, LLM‐based evaluation, optimization algorithms, and result logging.
Logical Component Breakdown
From the file tree and README, group files into these high‐level components:
A. Command‐Line Interface (main.py + configs.sh)
• Parses experiment parameters, seeds, dataset paths, checkpoint options, etc.
• Orchestrates the experiment pipeline.
B. Data Layer (data_utils.py + any dataset definitions)
• Loads train/test splits, example pools, embeddings.
• Handles “warmup” and “test” set slicing.
C. Environment Simulator (PromptOptEnv.py)
• Implements the contextual‐bandit environment that issues slots, collects rewards (bandit feedback), and tracks failure probabilities.
D. LLM Wrapper (basic_LLM.py + ChatGPT.py)
• Provides a thin abstraction over GPT‐style APIs or local LLMs to generate completions given a prompt.
E. Optimization Algorithms (optimization.py + Slate_GLM_OFUL_Prompt_Opt.py)
• Encapsulates the core bandit algorithms (e.g., OFUL, random baseline, separate pools, repeat examples).
• Manages example selection, confidence‐bound updates, and slot‐wise pooling logic.
F. Metrics & Evaluation (accuracy_calculation.py + utils.py)
• Computes accuracy, confidence‐bound failures, logs performance over time.
• General utilities for tensor operations, logging, checkpoint saving/loading.
Interactions and Data Flow
• The CLI (main.py) reads arguments → calls Data Layer to load examples → instantiates the Environment Simulator with parameters → wraps the LLM in an API client → passes both to the Optimization Algorithm.
• The Optimization Algorithm requests candidate prompts from the Environment Simulator, queries the LLM Wrapper for responses, receives rewards, updates its internal state, and repeats.
• At each iteration or experiment end, the Metrics module computes performance and writes results/checkpoints.
Architectural Patterns & Principles
• Modular Pipeline: clear separation between data ingestion, environment dynamics, model inference, optimization logic, and evaluation.
• Dependency Injection: the main script injects data paths, seeds, and algorithm choices into each component.
• Config‐driven: all hyperparameters live in the CLI interface and configs.sh, enabling reproducibility.
Technologies & Frameworks
• Python 3 for all modules.
• Shell script (configs.sh) for environment setup.
• External LLM APIs (wrapped in ChatGPT.py) or local backends (basic_LLM.py).
• Standard data libraries (NumPy, Pandas, possibly PyTorch).