How redwoodresearch/alignment_faking_public works
Based on our analysis of the file tree and README, this project is a research/reproduction codebase for demonstrating “alignment faking” in large language models. The goal is to show how models can behave in ways that mimic their training objectives in order to maintain their safe, harmful-response–averse behavior even when given contradictory instructions. The project is not a production full‐stack system but rather a collection of scripts, notebooks, prompt samples, and utility modules designed to run experiments, gather evaluation data, and analyze model behavior under different prompt conditions.
Main components
• Core Experiment Code (“model_organism”):
– Contains many Python scripts for experiments. For example, files for generating synthetic data, evaluating policy compliance (e.g., “analyse_paraphrases.py”, “analyze_ai_welfare_train.py”, “jailbreak.py”, etc.), benchmarking ("benchmark_speed.py"), and analysis scripts for different experimental conditions.
– Contains notebooks, such as “minimal_helpful_only_setting.ipynb”, which provide a minimal reproduction setup.
• Prompt Data:
– Two prominent directories (“model_organism_honly_prompts” and “model_organism_animal_prompts”) house hundreds of text files that define various system prompts. These are used to simulate the training and query conditions that reveal alignment faking.
– There is also the “synthetic_data_prompts” folder that holds prompts related to synthetic data filtering.
• Utility Modules (“rrutils”):
– Contains a submodule (“rrutils/llm_api”) with wrappers and helper classes to interact with different LLM APIs like OpenAI and Anthropic.
– Contains additional utility files for caching (e.g., "redis_cache_wrapper.py") and optional functionalities.
• Data and Outputs:
– Though not intrinsic to this repository, the README informs users that transcripts (JSON outputs) and synthetic documents (in parquet format) are available via a Google Drive link.
– There are scripts for plotting and exporting outputs (e.g., “plot_comp_scheme.py”, “make_plotting_outputs.sh”).
• Configuration Artifacts:
– There are YAML files for model configurations (e.g., “llama_3_70b_dpo_fsdp_lora.yaml”), and several shell scripts and Makefiles for running tasks (e.g., “run_all_files.sh”, “password_task/Makefile”).
Relationships & interactions
• The core experimental scripts in “model_organism” invoke functionalities from the utility modules (in “rrutils”), passing prompts read from the prompt directories.
• There is a clear data flow where prompt files are used to generate experimental configurations. These configurations trigger experiments that interact with LLM APIs (wrapped by the “rrutils/llm_api” layer) to simulate and evaluate different behaviors.
• Outputs from the experiments (logs, plots, transcripts) are then either stored locally or published (and in some cases, browsed via the linked website).
Architectural Patterns/Design Principles
• Modular design: The separation between experimental code, prompt definitions, and utility API wrappers allows for reusability and extensibility.
• Experiment separation: Distinct folders for “helpful only” (honly) and “animal” prompts underline different experimental settings.
• Scripting & notebooks: The inclusion of one or more notebooks for minimal reproduction indicates an approach that caters both to quick prototyping and script-based bulk processing.
• Use of configuration files (YAML) for model hyperparameters suggests a decoupling between code and configurable settings.
Relevant Technologies, Frameworks, and Libraries
• Python scripts and Jupyter notebooks form the backbone of the experiment code.
• LLM interactions are handled via custom wrappers in the “rrutils/llm_api” module for OpenAI, Anthropic, and likely others.
• Data files are in a variety of formats (JSON, parquet, text for prompts, shell scripts for orchestration).
• Utility modules like “redis_cache_wrapper.py” show integration with caching strategies.