anantshri1/lexsi-refusal

How anantshri1/lexsi-refusal works

This repository is a research pipeline for measuring multilingual refusal behavior in a language model and testing activation-based steering interventions. The main workflow is benchmark preparation, model response generation, response judging, metric analysis, steering-direction research, then screening and full-run comparison. Source excerpts establish several key calls and data flows; other subsystem coverage comes from the source index and filenames, so uncertain links are omitted. No README was provided.

Main entry points: paths.py Artifact paths · judge.py Refusal judge · rates.py Refusal metrics · generate.py Response generation

Components 17

Benchmark and generation

Benchmark construction src/build_benchmark.py
Response generation src/generate.py
Steering runs src/screen_job.py

Judging and evaluation

Refusal judge src/judge.py
Refusal metrics src/rates.py
Final comparison src/final_analysis.py
WildGuard validation src/wildguard_val.py
Screen analysis src/screen_analysis.py

Activation steering

Direction geometry src/geometry.py
Direction comparison src/cosine.py
Steering preparation src/steer_prep.py
Activation extraction src/extract_activations.py

Artifacts and integration

Artifact paths src/paths.py

Other components

Researcher
Hugging Face Hub
Inference API
Language model

Connections 25

Researcher → Benchmark construction: prepares data
Benchmark construction → Artifact paths: uses paths src/build_benchmark.py
Response generation → Artifact paths: reads and writes src/generate.py
Response generation → Language model: generates responses
Refusal judge → Response generation: checks completion src/judge.py
Refusal judge → Inference API: requests judgments src/judge.py
Refusal judge → Artifact paths: stores judgments src/judge.py
Refusal metrics → Response generation: loads benchmark src/rates.py
Refusal metrics → Refusal judge: loads labels src/rates.py
Refusal metrics → Artifact paths: reads artifacts src/rates.py
Final comparison → Refusal metrics: uses metrics src/final_analysis.py
Final comparison → Artifact paths: reads artifacts src/final_analysis.py
Direction geometry → Response generation: loads benchmark src/geometry.py
Direction geometry → Artifact paths: loads activations src/geometry.py
Direction comparison → Direction geometry: uses direction tools src/cosine.py
Direction comparison → Artifact paths: loads artifacts src/cosine.py
Steering preparation → Direction geometry: uses geometry src/steer_prep.py
Steering preparation → Artifact paths: reads and writes src/steer_prep.py
Steering runs → Hugging Face Hub: loads and uploads src/screen_job.py
Steering runs → Language model: runs inference src/screen_job.py
Steering runs → Artifact paths: uses artifacts
Screen analysis → Refusal metrics: uses metrics src/screen_analysis.py
Screen analysis → Refusal judge: reads judgments src/screen_analysis.py
Screen analysis → Steering runs: analyzes screening runs src/screen_analysis.py
WildGuard validation → Hugging Face Hub: uploads validation data src/wildguard_val.py

Drawn by GitDiagram on November 13, 2026 from the repository's file tree, README and source files. Read as Markdown ·