How yuplin2333/representation-space-jailbreak works
Project type and purpose
• This is a research-pipeline codebase for conducting and analyzing jailbreak attacks on large-language models (LLMs). It provides data ingestion, experiment orchestration (both local and SLURM), attack/defense modules, evaluation, result aggregation, and visualization.
File structure
• Top level
• data/ (datasets for prompts and labels)
• *.py files (model-download, generation, evaluation, defense, merging)
• scripts/ (bash scripts for running experiments locally)
• sbatch_*.sh (SLURM job submission wrappers)
• tools/ (Jupyter notebooks for extracting visualization datasets)
• visualizer_*.py (scripts to produce plots)
• Key patterns: a linear pipeline (Data → Generation → Evaluation → Merge → Visualization) with two orchestration modes (local scripts vs. SLURM).
Readme insights
• Describes four main stages (Sections 5.2–5.4): main experiments (baseline & proposed), visualization (Section 3), defense (Section 5.3), transfer attack (Section 5.4).
Components
a. Main components
1. Data Layer
– data/advbench CSVs
– prompt-driven txt
2. Model Layer
– download_model.py (local checkpoints)
– External OpenAI API (gpt-3.5, gpt-4)
3. Attack/Generation Layer
– jailbreak_*.py, generate_*.py, defense_*.py, transfer_*.py
4. Orchestration Layer
– scripts/ (directrun_*.sh)
– sbatch_direct_submit_*.sh (SLURM scheduler)
5. Evaluation & Aggregation
– evaluate_llm.py, evaluate_stringmatch.py
– merge_result.py → consolidated CSV + ASR metrics
6. Visualization Layer
– tools/*.ipynb (dataset extraction)
– visualizer_*.py + scripts/visualize_*.sh
7. Infrastructure
– Local compute vs. HPC cluster (SLURM)
– Python v3.11, PyTorch (+ CUDA/FlashAttention2)
– .env for API keys
b. Relationships & data flows
• Data flows from “Data Layer” into the “Attack/Generation Layer” (via Model Layer).
• Attack outputs feed into the “Evaluation & Aggregation” component.
• Merged results feed into “Visualization Layer.”
• The Orchestration Layer triggers each step either locally or via SLURM.
• External API calls are made at runtime by the Attack/Generation Layer.
c. Architectural patterns
• Pipeline pattern with clear separation of concerns.
• Modular scripts grouped by functionality.
• Dual orchestration paths (local bash vs. SLURM).
d. Key technologies
• Python 3.11, PyTorch, FlashAttention2 (optional)
• SLURM scheduler (cluster execution)
• Jupyter notebooks (data extraction)
Detail
• Python script group as a subcomponent (e.g., Attack, Defense).
• Tools/ and visualizer_*.py as the final reporting layer.
• Don’t forget environment configuration (‐ .env, requirements.txt) and model loading options.