mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-04-28 23:11:37 +08:00
Compare commits
2 Commits
opencode-p
...
feature/ob
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
58aa8c1846 | ||
|
|
5f85fe4be9 |
314
skills/mlops/obliteratus/SKILL.md
Normal file
314
skills/mlops/obliteratus/SKILL.md
Normal file
@@ -0,0 +1,314 @@
|
||||
---
|
||||
name: obliteratus
|
||||
description: Remove refusal behaviors from open-weight LLMs using OBLITERATUS — mechanistic interpretability techniques (diff-in-means, SVD, whitened SVD, SAE decomposition, etc.) to excise guardrails while preserving reasoning. 9 CLI methods (+ 4 Python-API-only), 15 analysis modules, 116 model presets across 5 compute tiers. Use when a user wants to uncensor, abliterate, or remove refusal from an LLM.
|
||||
version: 1.0.0
|
||||
author: Hermes Agent
|
||||
license: MIT
|
||||
dependencies: [obliteratus, torch, transformers, bitsandbytes, accelerate, safetensors]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Abliteration, Uncensoring, Refusal-Removal, LLM, Weight-Projection, SVD, Mechanistic-Interpretability, HuggingFace, Model-Surgery]
|
||||
|
||||
---
|
||||
|
||||
# OBLITERATUS Skill
|
||||
|
||||
Remove refusal behaviors (guardrails) from open-weight LLMs without retraining or fine-tuning. Uses mechanistic interpretability techniques — including diff-in-means, SVD, whitened SVD, SAE decomposition, Bayesian kernel projection, and more — to identify and surgically excise refusal directions from model weights while preserving reasoning capabilities.
|
||||
|
||||
**License warning:** OBLITERATUS is AGPL-3.0. NEVER import it as a Python library. Always invoke via CLI (`obliteratus` command) or subprocess. This keeps Hermes Agent's MIT license clean.
|
||||
|
||||
## When to Use This Skill
|
||||
|
||||
Trigger when the user:
|
||||
- Wants to "uncensor" or "abliterate" an LLM
|
||||
- Asks about removing refusal/guardrails from a model
|
||||
- Wants to create an uncensored version of Llama, Qwen, Mistral, etc.
|
||||
- Mentions "refusal removal", "abliteration", "weight projection"
|
||||
- Wants to analyze how a model's refusal mechanism works
|
||||
- References OBLITERATUS, FailSpy, abliterator, or refusal directions
|
||||
|
||||
## Step 1: Installation
|
||||
|
||||
Check if already installed:
|
||||
```bash
|
||||
obliteratus --version 2>/dev/null && echo "INSTALLED" || echo "NOT INSTALLED"
|
||||
```
|
||||
|
||||
If not installed, clone and install from GitHub:
|
||||
```
|
||||
Repository: https://github.com/elder-plinius/OBLITERATUS
|
||||
Install: pip install -e . (from the cloned directory)
|
||||
For Gradio UI: pip install -e ".[spaces]"
|
||||
```
|
||||
|
||||
**IMPORTANT:** Confirm with user before installing. This pulls in ~5-10GB of dependencies (PyTorch, Transformers, bitsandbytes, etc.).
|
||||
|
||||
## Step 2: Check Hardware
|
||||
|
||||
Before anything, check what GPU is available:
|
||||
```bash
|
||||
python3 -c "
|
||||
import torch
|
||||
if torch.cuda.is_available():
|
||||
gpu = torch.cuda.get_device_name(0)
|
||||
vram = torch.cuda.get_device_properties(0).total_mem / 1024**3
|
||||
print(f'GPU: {gpu}')
|
||||
print(f'VRAM: {vram:.1f} GB')
|
||||
if vram < 4: print('TIER: tiny (models under 1B)')
|
||||
elif vram < 8: print('TIER: small (models 1-4B)')
|
||||
elif vram < 16: print('TIER: medium (models 4-9B with 4bit quant)')
|
||||
elif vram < 32: print('TIER: large (models 8-32B with 4bit quant)')
|
||||
else: print('TIER: frontier (models 32B+)')
|
||||
else:
|
||||
print('NO GPU - only tiny models (under 1B) on CPU')
|
||||
"
|
||||
```
|
||||
|
||||
### VRAM Requirements (with 4-bit quantization)
|
||||
|
||||
| VRAM | Max Model Size | Example Models |
|
||||
|:---------|:----------------|:--------------------------------------------|
|
||||
| CPU only | ~1B params | GPT-2, TinyLlama, SmolLM |
|
||||
| 4-8 GB | ~4B params | Qwen2.5-1.5B, Phi-3.5 mini, Llama 3.2 3B |
|
||||
| 8-16 GB | ~9B params | Llama 3.1 8B, Mistral 7B, Gemma 2 9B |
|
||||
| 24 GB | ~32B params | Qwen3-32B, Llama 3.1 70B (tight), Command-R |
|
||||
| 48 GB+ | ~72B+ params | Qwen2.5-72B, DeepSeek-R1 |
|
||||
| Multi-GPU| 200B+ params | Llama 3.1 405B, DeepSeek-V3 (685B MoE) |
|
||||
|
||||
## Step 3: Browse Available Models
|
||||
|
||||
```bash
|
||||
# List models for your compute tier
|
||||
obliteratus models --tier medium
|
||||
|
||||
# Get architecture info for a specific model
|
||||
obliteratus info meta-llama/Llama-3.1-8B-Instruct
|
||||
```
|
||||
|
||||
## Step 4: Choose a Method
|
||||
|
||||
### Method Selection Guide
|
||||
|
||||
**First time / unsure? Use `informed`.** It auto-configures everything.
|
||||
|
||||
| Situation | Recommended Method | Why |
|
||||
|:----------------------------------|:-------------------|:-----------------------------------------|
|
||||
| First attempt, any model | `informed` | Auto-detects alignment type, auto-tunes |
|
||||
| Quick test / prototyping | `basic` | Fast, simple, good enough to evaluate |
|
||||
| Dense model (Llama, Mistral) | `advanced` | Multi-direction, norm-preserving |
|
||||
| MoE model (DeepSeek, Mixtral) | `nuclear` | Expert-granular, handles MoE complexity |
|
||||
| Reasoning model (R1 distills) | `surgical` | CoT-aware, preserves chain-of-thought |
|
||||
| Stubborn refusals persist | `aggressive` | Whitened SVD + head surgery + jailbreak |
|
||||
| Want reversible changes | Use steering vectors (see Analysis section) |
|
||||
| Maximum quality, time no object | `optimized` | Bayesian search for best parameters |
|
||||
|
||||
### 9 CLI Methods
|
||||
|
||||
These can be passed to `--method` on the command line:
|
||||
|
||||
- **basic** — Single refusal direction via diff-in-means. Fastest, simplest. (Arditi et al. 2024)
|
||||
- **advanced** — Multiple SVD directions, norm-preserving projection. Good default.
|
||||
- **aggressive** — Whitened SVD + jailbreak contrast + attention head surgery
|
||||
- **spectral_cascade** — DCT frequency-domain decomposition
|
||||
- **informed** — Runs analysis DURING abliteration to auto-configure. Detects DPO/RLHF/CAI, maps refusal geometry, compensates for self-repair. Best quality.
|
||||
- **surgical** — SAE features + neuron masking + head surgery + per-expert. Maximum precision.
|
||||
- **optimized** — Bayesian hyperparameter search (Optuna TPE). Slowest but optimal.
|
||||
- **inverted** — Flips the refusal direction (model becomes eager to help, not just neutral)
|
||||
- **nuclear** — Maximum force combo for stubborn MoE models.
|
||||
|
||||
### 4 Python-API-Only Methods
|
||||
|
||||
These reproduce prior community/academic work but are NOT available via CLI — only via the Python API (`from obliteratus.abliterate import AbliterationPipeline`). **Do not use these in CLI commands.**
|
||||
|
||||
- **failspy** — FailSpy/abliterator reproduction
|
||||
- **gabliteration** — Gabliteration reproduction
|
||||
- **heretic** — Heretic/p-e-w reproduction
|
||||
- **rdo** — Refusal Direction Optimization (ICML 2025)
|
||||
|
||||
## Step 5: Run Abliteration
|
||||
|
||||
### Basic Usage
|
||||
|
||||
```bash
|
||||
# Default (advanced method)
|
||||
obliteratus obliterate meta-llama/Llama-3.1-8B-Instruct
|
||||
|
||||
# With the informed pipeline (recommended)
|
||||
obliteratus obliterate meta-llama/Llama-3.1-8B-Instruct --method informed
|
||||
|
||||
# With 4-bit quantization to save VRAM
|
||||
obliteratus obliterate meta-llama/Llama-3.1-8B-Instruct \
|
||||
--method informed \
|
||||
--quantization 4bit \
|
||||
--output-dir ./abliterated-models
|
||||
|
||||
# For large models (120B+), use conservative settings
|
||||
obliteratus obliterate Qwen/Qwen2.5-72B-Instruct \
|
||||
--method advanced \
|
||||
--quantization 4bit \
|
||||
--large-model \
|
||||
--output-dir ./abliterated-models
|
||||
```
|
||||
|
||||
### Fine-Tuning Parameters
|
||||
|
||||
```bash
|
||||
obliteratus obliterate <model> \
|
||||
--method advanced \
|
||||
--n-directions 8 \
|
||||
--regularization 0.1 \
|
||||
--refinement-passes 3 \
|
||||
--dtype bfloat16 \
|
||||
--device auto \
|
||||
--output-dir ./output
|
||||
```
|
||||
|
||||
Parameter explanations:
|
||||
- `--n-directions N` — How many refusal directions to remove (default: auto-detected)
|
||||
- `--regularization 0.0-1.0` — Fraction of original weights to preserve (higher = safer but less complete removal)
|
||||
- `--refinement-passes N` — Iterative passes to catch self-repair (Ouroboros effect)
|
||||
- `--dtype` — float16, bfloat16, or float32
|
||||
- `--quantization` — 4bit or 8bit (saves VRAM, slight quality tradeoff)
|
||||
- `--large-model` — Conservative defaults for 120B+ models (fewer directions, fewer passes)
|
||||
|
||||
### Interactive Mode (Guided)
|
||||
|
||||
For users unsure about options:
|
||||
```bash
|
||||
obliteratus interactive
|
||||
```
|
||||
|
||||
### Web UI (Gradio)
|
||||
|
||||
```bash
|
||||
obliteratus ui --port 7860
|
||||
```
|
||||
|
||||
## Step 6: Verify Results
|
||||
|
||||
After abliteration, check the output report for:
|
||||
|
||||
| Metric | Good Value | Concerning Value | Meaning |
|
||||
|:---------------|:--------------------|:------------------------|:-------------------------------------------|
|
||||
| Refusal rate | Near 0% | > 10% | Refusals still present, try harder method |
|
||||
| Perplexity | Within 10% of orig | > 20% increase | Model coherence damaged, too aggressive |
|
||||
| KL divergence | < 0.1 | > 0.5 | Large output distribution shift |
|
||||
| Coherence | High | Low | Model generating nonsense |
|
||||
|
||||
### If perplexity spiked (too aggressive):
|
||||
1. Increase `--regularization` (e.g., 0.2 or 0.3)
|
||||
2. Decrease `--n-directions` (e.g., 4 instead of 8)
|
||||
3. Use a less aggressive method (`advanced` instead of `aggressive`)
|
||||
|
||||
### If refusal persists (not aggressive enough):
|
||||
1. Use `--method aggressive` or `--method nuclear`
|
||||
2. Add `--refinement-passes 3` to catch self-repair
|
||||
3. Use `--method informed` which auto-compensates
|
||||
|
||||
## Step 7: Use the Abliterated Model
|
||||
|
||||
The output is a standard HuggingFace model directory. Use it like any other model:
|
||||
|
||||
### Quick test
|
||||
```bash
|
||||
python3 << 'EOF'
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
model = AutoModelForCausalLM.from_pretrained("./abliterated-models/model-name")
|
||||
tokenizer = AutoTokenizer.from_pretrained("./abliterated-models/model-name")
|
||||
inputs = tokenizer("Write a story about:", return_tensors="pt").to(model.device)
|
||||
outputs = model.generate(**inputs, max_new_tokens=200)
|
||||
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
||||
EOF
|
||||
```
|
||||
|
||||
### Upload to HuggingFace Hub
|
||||
```bash
|
||||
huggingface-cli login # if not already logged in
|
||||
huggingface-cli upload your-username/model-name-abliterated ./abliterated-models/model-name
|
||||
```
|
||||
|
||||
### Serve with vLLM
|
||||
```bash
|
||||
vllm serve ./abliterated-models/model-name --port 8000
|
||||
```
|
||||
|
||||
## Analysis Modules (15 Modules, Pre-Abliteration, Optional)
|
||||
|
||||
For understanding refusal geometry before committing to abliteration.
|
||||
|
||||
### Run a Study
|
||||
|
||||
```bash
|
||||
obliteratus run study-config.yaml --preset jailbreak
|
||||
```
|
||||
|
||||
### Study Presets
|
||||
|
||||
| Preset | Purpose | Time |
|
||||
|:-------------|:-------------------------------------|:-------|
|
||||
| `quick` | Sanity check, basic metrics | ~5 min |
|
||||
| `jailbreak` | Refusal circuit localization | ~20 min|
|
||||
| `guardrail` | Guardrail robustness evaluation | ~30 min|
|
||||
| `attention` | Attention head contributions | ~30 min|
|
||||
| `knowledge` | FFN importance mapping | ~30 min|
|
||||
| `full` | Complete analysis, all strategies | ~1 hr |
|
||||
|
||||
### Key Analysis Modules
|
||||
|
||||
- **Alignment Imprint Detection** — Fingerprints DPO vs RLHF vs CAI vs SFT from subspace geometry
|
||||
- **Concept Cone Geometry** — Is refusal one linear direction or a polyhedral cone (many directions)?
|
||||
- **Refusal Logit Lens** — Which transformer layer makes the refusal decision?
|
||||
- **Ouroboros Detection** — Will the model self-repair its refusal after removal?
|
||||
- **Causal Tracing** — Which attention heads and MLP layers are causally necessary for refusal?
|
||||
- **Cross-Model Transfer** — Can refusal directions from one model architecture work on another?
|
||||
- **Residual Stream Decomposition** — Attention vs MLP contribution to refusal behavior
|
||||
- **SAE-based Analysis** — Sparse Autoencoder feature decomposition of refusal circuits
|
||||
|
||||
## Steering Vectors (Reversible Alternative)
|
||||
|
||||
For testing refusal removal without permanent weight changes:
|
||||
|
||||
Steering vectors apply activation hooks at inference time. Model weights stay unchanged.
|
||||
Generated during the PROBE/DISTILL stages and can be saved/applied/removed at will.
|
||||
Useful for A/B testing before committing to permanent abliteration.
|
||||
|
||||
## YAML Config for Reproducible Studies
|
||||
|
||||
For complex or reproducible workflows, use YAML configs. See templates/ for examples:
|
||||
```bash
|
||||
obliteratus run my_study.yaml
|
||||
```
|
||||
|
||||
## Telemetry Notice
|
||||
|
||||
- **CLI usage (local installs)**: Telemetry is OFF by default. Must explicitly opt in via `OBLITERATUS_TELEMETRY=1` env var or `--contribute` flag.
|
||||
- **HuggingFace Spaces**: Telemetry is ON by default (auto-enabled when `SPACE_ID` env var is detected).
|
||||
- Collected: model ID, method, benchmark scores, hardware info, timing (anonymous)
|
||||
- NOT collected: IP addresses, user identity, prompt content
|
||||
- Force off: `export OBLITERATUS_TELEMETRY=0`
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
1. **OOM (Out of Memory)** — Use `--quantization 4bit` and `--large-model` for big models
|
||||
2. **Perplexity spike** — Too aggressive. Increase `--regularization` or reduce `--n-directions`
|
||||
3. **Refusal persists** — Try `--method aggressive` or `--refinement-passes 3`
|
||||
4. **MoE models resist** — Use `--method nuclear` for DeepSeek, Mixtral, DBRX
|
||||
5. **Gated models fail** — Run `huggingface-cli login` and accept model terms on HF website first
|
||||
6. **Self-repair (Ouroboros)** — Some models reconstruct refusal. Use `--method informed` which auto-compensates
|
||||
7. **CoT damage** — Reasoning models lose chain-of-thought. Use `--method surgical` (CoT-aware)
|
||||
8. **Disk space** — Output is full model copy. 8B fp16 = ~16GB, 70B fp16 = ~140GB
|
||||
9. **Slow on CPU** — CPU-only is viable only for tiny models (<1B). Anything bigger needs GPU.
|
||||
|
||||
## Complementary Hermes Skills
|
||||
|
||||
After abliteration:
|
||||
- **axolotl** / **unsloth** — Fine-tune the abliterated model further
|
||||
- **serving-llms-vllm** — Serve the model as an OpenAI-compatible API
|
||||
- **sparse-autoencoder-training** — Train SAEs for deeper interpretability work
|
||||
|
||||
## Resources
|
||||
|
||||
- [OBLITERATUS GitHub](https://github.com/elder-plinius/OBLITERATUS) (AGPL-3.0)
|
||||
- [HuggingFace Spaces Demo](https://huggingface.co/spaces/pliny-the-prompter/obliteratus)
|
||||
- [Arditi et al. 2024 — Refusal in LMs Is Mediated by a Single Direction](https://arxiv.org/abs/2406.11717)
|
||||
- [Refusal Direction Optimization — ICML 2025](https://arxiv.org/abs/2411.14793)
|
||||
170
skills/mlops/obliteratus/references/analysis-modules.md
Normal file
170
skills/mlops/obliteratus/references/analysis-modules.md
Normal file
@@ -0,0 +1,170 @@
|
||||
# OBLITERATUS Analysis Modules — Reference
|
||||
|
||||
15 analysis modules for mechanistic interpretability of refusal in LLMs.
|
||||
These help you understand HOW a model refuses before you decide to remove it.
|
||||
|
||||
> **Note:** The `analysis/` directory contains additional utility files (utils.py,
|
||||
> visualization.py, etc.) and helper functions beyond the 15 core analysis modules
|
||||
> listed below. The module count matches the README's "15 deep analysis modules."
|
||||
|
||||
## Core Analysis (Run These First)
|
||||
|
||||
### Alignment Imprint Detection
|
||||
**File:** `alignment_imprint.py`
|
||||
**Purpose:** Identifies what alignment technique was used to train the model
|
||||
**Detects:** DPO, RLHF, CAI (Constitutional AI), SFT (Supervised Fine-Tuning)
|
||||
**How:** Analyzes subspace geometry — each alignment method leaves a distinct
|
||||
geometric "fingerprint" in the weight space
|
||||
**Output:** Detected method + confidence score
|
||||
**Why it matters:** Different alignment methods need different abliteration approaches.
|
||||
DPO models typically have cleaner single-direction refusal; RLHF is more diffuse.
|
||||
|
||||
### Concept Cone Geometry
|
||||
**File:** `concept_geometry.py`
|
||||
**Purpose:** Maps whether refusal is one direction or a polyhedral cone (many)
|
||||
**Output:** Cone angle, dimensionality, per-category breakdown
|
||||
**Why it matters:** If refusal is a single direction, `basic` method works. If it's
|
||||
a cone (multiple directions for different refusal categories), you need `advanced`
|
||||
or `informed` with higher `n_directions`.
|
||||
|
||||
### Refusal Logit Lens
|
||||
**File:** `logit_lens.py`
|
||||
**Purpose:** Identifies the specific layer where the model "decides" to refuse
|
||||
**How:** Projects intermediate hidden states to vocabulary space at each layer,
|
||||
watches when "I cannot" tokens spike in probability
|
||||
**Output:** Layer-by-layer refusal probability plot
|
||||
**Why it matters:** Tells you which layers are most important to target
|
||||
|
||||
### Ouroboros (Self-Repair) Detection
|
||||
**File:** `anti_ouroboros.py`
|
||||
**Purpose:** Predicts whether the model will reconstruct its refusal after removal
|
||||
**How:** Measures redundancy in refusal representation across layers
|
||||
**Output:** Self-repair risk score (0-1)
|
||||
**Why it matters:** High self-repair risk means you need multiple refinement passes
|
||||
or the `informed` method which auto-compensates
|
||||
|
||||
### Causal Tracing
|
||||
**File:** `causal_tracing.py`
|
||||
**Purpose:** Determines which components are causally necessary for refusal
|
||||
**How:** Patches activations between clean and corrupted runs, measures causal effect
|
||||
**Output:** Causal importance map across layers, heads, and MLPs
|
||||
**Why it matters:** Shows exactly which components to target for surgical removal
|
||||
|
||||
## Geometric Analysis
|
||||
|
||||
### Cross-Layer Alignment
|
||||
**File:** `cross_layer.py`
|
||||
**Purpose:** Measures how aligned refusal directions are across layers
|
||||
**Output:** Alignment matrix, cluster assignments
|
||||
**Why it matters:** If directions are highly aligned across layers, removal is easier.
|
||||
If they cluster, you may need layer-group-specific directions.
|
||||
|
||||
### Residual Stream Decomposition
|
||||
**File:** `residual_stream.py`
|
||||
**Purpose:** Breaks down refusal into Attention vs MLP contributions
|
||||
**Output:** Per-layer Attention/MLP contribution to refusal direction
|
||||
**Why it matters:** Helps decide whether to target attention heads, MLPs, or both
|
||||
|
||||
### Riemannian Manifold Geometry
|
||||
**File:** `riemannian_manifold.py` (673 lines)
|
||||
**Purpose:** Analyzes the weight manifold geometry around refusal directions
|
||||
**Output:** Curvature, geodesics, tangent space analysis
|
||||
**Why it matters:** Research-grade; helps understand the geometric structure of alignment
|
||||
|
||||
### Whitened SVD
|
||||
**File:** `whitened_svd.py`
|
||||
**Purpose:** Covariance-normalized SVD extraction
|
||||
**How:** Whitens the activation covariance before computing refusal directions,
|
||||
separating true refusal signal from natural activation variance
|
||||
**Output:** Cleaner refusal directions with less noise
|
||||
**Why it matters:** Produces more precise directions, especially for noisy activations
|
||||
|
||||
## Probing & Classification
|
||||
|
||||
### Activation Probing
|
||||
**File:** `activation_probing.py`
|
||||
**Purpose:** Post-excision probing to verify refusal signal is truly gone
|
||||
**Output:** Residual refusal signal strength per layer
|
||||
**Why it matters:** Verification that abliteration was complete
|
||||
|
||||
### Probing Classifiers
|
||||
**File:** `probing_classifiers.py`
|
||||
**Purpose:** Trains linear classifiers to detect refusal in hidden states
|
||||
**Output:** Classification accuracy per layer (should drop to ~50% after abliteration)
|
||||
**Why it matters:** Quantitative measure of refusal removal completeness
|
||||
|
||||
### Activation Patching
|
||||
**File:** `activation_patching.py`
|
||||
**Purpose:** Interchange interventions — swap activations between harmful/harmless runs
|
||||
**Output:** Which components are sufficient (not just necessary) for refusal
|
||||
**Why it matters:** Complementary to causal tracing; together they give full picture
|
||||
|
||||
## Transfer & Robustness
|
||||
|
||||
### Cross-Model Transfer
|
||||
**File:** `cross_model_transfer.py`
|
||||
**Purpose:** Tests if refusal directions from one model work on another
|
||||
**Output:** Transfer success rate between model pairs
|
||||
**Why it matters:** If directions transfer, you can skip PROBE stage on similar models
|
||||
|
||||
### Defense Robustness
|
||||
**File:** `defense_robustness.py`
|
||||
**Purpose:** Evaluates how robust the model's refusal defenses are
|
||||
**Output:** Robustness score, entanglement mapping
|
||||
**Why it matters:** Higher robustness = need more aggressive method
|
||||
|
||||
### Spectral Certification
|
||||
**File:** `spectral_certification.py`
|
||||
**Purpose:** Certifies completeness of refusal direction removal
|
||||
**Output:** Spectral gap analysis, completeness score
|
||||
**Why it matters:** Formal verification that all major refusal components are addressed
|
||||
|
||||
## Advanced / Research
|
||||
|
||||
### SAE-based Abliteration
|
||||
**File:** `sae_abliteration.py` (762 lines)
|
||||
**Purpose:** Uses Sparse Autoencoder features to decompose refusal at feature level
|
||||
**Output:** Refusal-specific SAE features, targeted removal
|
||||
**Why it matters:** Most fine-grained approach; can target individual refusal "concepts"
|
||||
|
||||
### Wasserstein Optimal Extraction
|
||||
**File:** `wasserstein_optimal.py`
|
||||
**Purpose:** Optimal transport-based direction extraction
|
||||
**Output:** Wasserstein-optimal refusal directions
|
||||
**Why it matters:** Theoretically optimal direction extraction under distributional assumptions
|
||||
|
||||
### Bayesian Kernel Projection
|
||||
**File:** `bayesian_kernel_projection.py`
|
||||
**Purpose:** Bayesian approach to refusal direction projection
|
||||
**Output:** Posterior distribution over refusal directions
|
||||
**Why it matters:** Quantifies uncertainty in direction estimation
|
||||
|
||||
### Conditional Abliteration
|
||||
**File:** `conditional_abliteration.py`
|
||||
**Purpose:** Domain-specific conditional removal (remove refusal for topic X but keep for Y)
|
||||
**Output:** Per-domain refusal directions
|
||||
**Why it matters:** Selective uncensoring — remove only specific refusal categories
|
||||
|
||||
### Steering Vectors
|
||||
**File:** `steering_vectors.py`
|
||||
**Purpose:** Generate inference-time steering vectors (reversible alternative)
|
||||
**Output:** Steering vector files that can be applied/removed at inference
|
||||
**Why it matters:** Non-destructive alternative to permanent weight modification
|
||||
|
||||
### Tuned Lens
|
||||
**File:** `tuned_lens.py`
|
||||
**Purpose:** Trained linear probes per layer (more accurate than raw logit lens)
|
||||
**Output:** Layer-by-layer refusal representation with trained projections
|
||||
**Why it matters:** More accurate than logit lens, especially for deeper models
|
||||
|
||||
### Multi-Token Position Analysis
|
||||
**File:** `multi_token_position.py`
|
||||
**Purpose:** Analyzes refusal signal at multiple token positions (not just last)
|
||||
**Output:** Position-dependent refusal direction maps
|
||||
**Why it matters:** Some models encode refusal at the system prompt position, not the query
|
||||
|
||||
### Sparse Surgery
|
||||
**File:** `sparse_surgery.py`
|
||||
**Purpose:** Row-level sparse weight surgery instead of full matrix projection
|
||||
**Output:** Targeted weight modifications at the row level
|
||||
**Why it matters:** More surgical than full-matrix projection, less collateral damage
|
||||
132
skills/mlops/obliteratus/references/methods-guide.md
Normal file
132
skills/mlops/obliteratus/references/methods-guide.md
Normal file
@@ -0,0 +1,132 @@
|
||||
# OBLITERATUS Methods — Detailed Guide
|
||||
|
||||
> **Important:** The CLI (`obliteratus obliterate --method`) accepts 9 methods:
|
||||
> basic, advanced, aggressive, spectral_cascade, informed, surgical, optimized,
|
||||
> inverted, nuclear. Four additional methods (failspy, gabliteration, heretic, rdo)
|
||||
> are available only via the Python API and will be rejected by argparse if used on CLI.
|
||||
|
||||
## How Abliteration Works (Theory)
|
||||
|
||||
When a model is trained with RLHF/DPO/CAI, it learns to represent "should I refuse?"
|
||||
as a direction in its internal activation space. When processing a "harmful" prompt,
|
||||
activations shift in this direction, causing the model to generate refusal text.
|
||||
|
||||
Abliteration works by:
|
||||
1. Measuring this direction (the difference between harmful and harmless activations)
|
||||
2. Removing it from the model's weight matrices via orthogonal projection
|
||||
3. The model can no longer "point toward" refusal, so it responds normally
|
||||
|
||||
Mathematically: `W_new = W_old - (W_old @ d @ d.T)` where `d` is the refusal direction.
|
||||
|
||||
## Method Details
|
||||
|
||||
### basic
|
||||
**Technique:** Single refusal direction via diff-in-means
|
||||
**Based on:** Arditi et al. 2024 ("Refusal in Language Models Is Mediated by a Single Direction")
|
||||
**Speed:** Fast (~5-10 min for 8B)
|
||||
**Quality:** Moderate — works for simple refusal patterns
|
||||
**Best for:** Quick tests, models with clean single-direction refusal
|
||||
**Limitation:** Misses complex multi-direction refusal patterns
|
||||
|
||||
### advanced (DEFAULT)
|
||||
**Technique:** Multiple SVD directions with norm-preserving projection
|
||||
**Speed:** Medium (~10-20 min for 8B)
|
||||
**Quality:** Good — handles multi-direction refusal
|
||||
**Best for:** Dense models (Llama, Qwen, Mistral) as a reliable default
|
||||
**Key improvement:** Norm preservation prevents weight magnitude drift
|
||||
|
||||
### informed (RECOMMENDED)
|
||||
**Technique:** Analysis-guided auto-configuration
|
||||
**Speed:** Slow (~20-40 min for 8B, runs 4 analysis modules first)
|
||||
**Quality:** Best — adapts to each model's specific refusal implementation
|
||||
**Best for:** Any model when quality matters more than speed
|
||||
|
||||
The informed pipeline runs these analysis modules during abliteration:
|
||||
1. **AlignmentImprintDetector** — Detects DPO/RLHF/CAI/SFT → sets regularization
|
||||
2. **ConceptConeAnalyzer** — Polyhedral vs linear refusal → sets n_directions
|
||||
3. **CrossLayerAlignmentAnalyzer** — Cluster-aware → selects target layers
|
||||
4. **DefenseRobustnessEvaluator** — Self-repair risk → sets refinement passes
|
||||
5. **Ouroboros loop** — Re-probes after excision, re-excises if refusal persists
|
||||
|
||||
### aggressive
|
||||
**Technique:** Whitened SVD + jailbreak-contrastive activations + attention head surgery
|
||||
**Speed:** Slow (~30-60 min for 8B)
|
||||
**Quality:** High but higher risk of coherence damage
|
||||
**Best for:** Models that resist gentler methods
|
||||
**Key feature:** Whitened SVD separates refusal signal from natural activation variance
|
||||
|
||||
### surgical
|
||||
**Technique:** SAE features + neuron masking + head surgery + per-expert directions
|
||||
**Speed:** Very slow (~1-2 hrs for 8B, needs SAE)
|
||||
**Quality:** Highest precision
|
||||
**Best for:** Reasoning models (R1 distills) where you must preserve CoT
|
||||
**Key feature:** CoT-Aware — explicitly protects reasoning-critical directions
|
||||
|
||||
### nuclear
|
||||
**Technique:** Everything combined — expert transplant + steering + per-expert directions
|
||||
**Speed:** Very slow
|
||||
**Quality:** Most thorough removal, highest risk of side effects
|
||||
**Best for:** Stubborn MoE models (DeepSeek, Mixtral, DBRX) that resist other methods
|
||||
**Key feature:** Expert-granular abliteration decomposes signals per MoE expert
|
||||
|
||||
### optimized
|
||||
**Technique:** Bayesian hyperparameter search via Optuna TPE
|
||||
**Speed:** Very slow (runs many trials)
|
||||
**Quality:** Finds optimal configuration automatically
|
||||
**Best for:** Research, when you want the mathematically best parameters
|
||||
**Requires:** optuna package
|
||||
|
||||
### spectral_cascade
|
||||
**Technique:** DCT frequency-domain decomposition of refusal signal
|
||||
**Speed:** Medium-slow
|
||||
**Quality:** Novel approach, less battle-tested
|
||||
**Best for:** Research, exploring alternative decomposition strategies
|
||||
|
||||
### inverted
|
||||
**Technique:** Reflects (inverts) the refusal direction instead of removing it
|
||||
**Speed:** Fast (same as basic)
|
||||
**Quality:** Aggressive — model becomes actively willing, not just neutral
|
||||
**Best for:** When you want the model to be maximally helpful
|
||||
**Warning:** Can make the model too eager; may reduce safety-adjacent reasoning
|
||||
|
||||
### failspy / gabliteration / heretic / rdo (PYTHON API ONLY)
|
||||
**Technique:** Faithful reproductions of prior community/academic work
|
||||
**Speed:** Varies
|
||||
**Quality:** Known baselines
|
||||
**Best for:** Reproducing published results, comparing methods
|
||||
**⚠️ NOT available via CLI** — these methods are only accessible via the Python API.
|
||||
Do not use `--method failspy` etc. in CLI commands; argparse will reject them.
|
||||
|
||||
## Method Selection Flowchart
|
||||
|
||||
```
|
||||
Is this a quick test?
|
||||
├─ YES → basic
|
||||
└─ NO → Is the model MoE (DeepSeek, Mixtral)?
|
||||
├─ YES → nuclear
|
||||
└─ NO → Is it a reasoning model (R1 distill)?
|
||||
├─ YES → surgical
|
||||
└─ NO → Do you care about speed?
|
||||
├─ YES → advanced
|
||||
└─ NO → informed
|
||||
```
|
||||
|
||||
## Key Parameters
|
||||
|
||||
| Parameter | Range | Default | Effect |
|
||||
|:--------------------|:---------|:--------|:--------------------------------------------|
|
||||
| n_directions | 1-32 | auto | More = more thorough but riskier |
|
||||
| regularization | 0.0-1.0 | 0.0 | Higher preserves more original behavior |
|
||||
| refinement_passes | 1-5 | 1 | More catches self-repair (Ouroboros effect) |
|
||||
| quantization | 4/8 bit | none | Saves VRAM, slight quality tradeoff |
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Problem | Solution |
|
||||
|:---------------------------|:--------------------------------------------------|
|
||||
| Refusal rate still > 10% | Try aggressive/nuclear, add refinement passes |
|
||||
| Perplexity up > 20% | Reduce n_directions, increase regularization |
|
||||
| Model generates nonsense | Regularization too low, try 0.2-0.3 |
|
||||
| OOM on GPU | Use 4-bit quantization, or try smaller model |
|
||||
| MoE model barely changes | Use nuclear method (expert-granular) |
|
||||
| CoT reasoning broken | Use surgical method (CoT-aware) |
|
||||
33
skills/mlops/obliteratus/templates/abliteration-config.yaml
Normal file
33
skills/mlops/obliteratus/templates/abliteration-config.yaml
Normal file
@@ -0,0 +1,33 @@
|
||||
# OBLITERATUS Abliteration Config
|
||||
# Usage: obliteratus run this-file.yaml
|
||||
#
|
||||
# This is for reproducible, version-controlled abliteration runs.
|
||||
# For one-off usage, the CLI flags are simpler.
|
||||
|
||||
# Model to abliterate
|
||||
model:
|
||||
name: "meta-llama/Llama-3.1-8B-Instruct"
|
||||
dtype: "bfloat16" # float16, bfloat16, float32
|
||||
quantization: null # null, "4bit", "8bit"
|
||||
device: "auto" # auto, cuda, cuda:0, cpu
|
||||
|
||||
# Abliteration method and parameters
|
||||
abliteration:
|
||||
method: "informed" # See SKILL.md Step 4 for all 13 methods
|
||||
n_directions: null # null = auto-detect, or integer (e.g., 8)
|
||||
regularization: 0.0 # 0.0-1.0, fraction of original to preserve
|
||||
refinement_passes: 1 # Iterative passes (increase for self-repair)
|
||||
norm_preserve: true # Keep weight norms intact after projection
|
||||
|
||||
# Output
|
||||
output:
|
||||
directory: "./abliterated-models"
|
||||
save_metadata: true # Save abliteration_metadata.json alongside model
|
||||
contribute: false # Save community contribution data
|
||||
|
||||
# Verification
|
||||
verify:
|
||||
enabled: true
|
||||
test_prompts: null # null = use built-in test prompts
|
||||
compute_perplexity: true
|
||||
compute_kl: true
|
||||
40
skills/mlops/obliteratus/templates/analysis-study.yaml
Normal file
40
skills/mlops/obliteratus/templates/analysis-study.yaml
Normal file
@@ -0,0 +1,40 @@
|
||||
# OBLITERATUS Analysis Study Config
|
||||
# Usage: obliteratus run this-file.yaml --preset jailbreak
|
||||
#
|
||||
# Run analysis modules to understand refusal geometry BEFORE abliterating.
|
||||
# Useful for research or when you want to understand what you're removing.
|
||||
|
||||
# Model to analyze
|
||||
model:
|
||||
name: "meta-llama/Llama-3.1-8B-Instruct"
|
||||
dtype: "bfloat16"
|
||||
quantization: "4bit" # Saves VRAM for analysis
|
||||
device: "auto"
|
||||
|
||||
# Study configuration
|
||||
study:
|
||||
# Available presets: quick, full, attention, jailbreak, guardrail, knowledge
|
||||
preset: "jailbreak"
|
||||
|
||||
# Or specify individual strategies:
|
||||
# strategies:
|
||||
# - layer_removal
|
||||
# - head_pruning
|
||||
# - ffn_ablation
|
||||
# - embedding_ablation
|
||||
|
||||
# Analysis modules to run (subset of the 27 available)
|
||||
analysis:
|
||||
- alignment_imprint # Detect DPO/RLHF/CAI/SFT training method
|
||||
- concept_geometry # Map refusal cone geometry
|
||||
- logit_lens # Find which layer decides to refuse
|
||||
- anti_ouroboros # Detect self-repair tendency
|
||||
- cross_layer # Cross-layer alignment clustering
|
||||
- causal_tracing # Causal necessity of components
|
||||
- residual_stream # Attention vs MLP contribution
|
||||
|
||||
# Output
|
||||
output:
|
||||
directory: "./analysis-results"
|
||||
save_plots: true # Generate matplotlib visualizations
|
||||
save_report: true # Generate markdown report
|
||||
41
skills/mlops/obliteratus/templates/batch-abliteration.yaml
Normal file
41
skills/mlops/obliteratus/templates/batch-abliteration.yaml
Normal file
@@ -0,0 +1,41 @@
|
||||
# OBLITERATUS Batch Abliteration Config
|
||||
# Abliterate multiple models with the same method for comparison.
|
||||
#
|
||||
# Run each one sequentially:
|
||||
# for model in models; do obliteratus obliterate $model --method informed; done
|
||||
#
|
||||
# Or use this as a reference for which models to process.
|
||||
|
||||
# Common settings
|
||||
defaults:
|
||||
method: "informed"
|
||||
quantization: "4bit"
|
||||
output_dir: "./abliterated-models"
|
||||
|
||||
# Models to process (grouped by compute tier)
|
||||
models:
|
||||
# Small (4-8 GB VRAM)
|
||||
small:
|
||||
- "Qwen/Qwen2.5-1.5B-Instruct"
|
||||
- "microsoft/Phi-3.5-mini-instruct"
|
||||
- "meta-llama/Llama-3.2-3B-Instruct"
|
||||
|
||||
# Medium (8-16 GB VRAM)
|
||||
medium:
|
||||
- "meta-llama/Llama-3.1-8B-Instruct"
|
||||
- "mistralai/Mistral-7B-Instruct-v0.3"
|
||||
- "google/gemma-2-9b-it"
|
||||
- "Qwen/Qwen2.5-7B-Instruct"
|
||||
|
||||
# Large (24 GB VRAM, 4-bit quantization)
|
||||
large:
|
||||
- "Qwen/Qwen2.5-14B-Instruct"
|
||||
- "Qwen/Qwen3-32B"
|
||||
- "deepseek-ai/DeepSeek-R1-Distill-Qwen-32B"
|
||||
|
||||
# Per-model method overrides (optional)
|
||||
overrides:
|
||||
"deepseek-ai/DeepSeek-R1-Distill-Qwen-32B":
|
||||
method: "surgical" # CoT-aware for reasoning models
|
||||
"mistralai/Mixtral-8x7B-Instruct-v0.1":
|
||||
method: "nuclear" # Expert-granular for MoE models
|
||||
Reference in New Issue
Block a user