AI product Open source
Heretic is a command-line tool for automatically removing safety alignment from transformer-based language models without post-training. It combines directional ablation (abliteration) with a Tree-structured Parzen Estimator optimizer powered by Optuna, co-minimizing refusal counts and KL divergence from the original model to select ablation parameters automatically.
For supported transformer components, currently attention output projections and MLP down-projections, Heretic computes per-layer residual directions from the difference between first-token hidden states for harmful and harmless prompts, then orthogonalizes the associated weight matrices against those directions. Its optimizer can interpolate between residual directions and select separate, flexible layer-weight kernels for different components. The tool supports most dense models, many multimodal models, several mixture-of-experts architectures, and some hybrid architectures; pure state-space models and certain research architectures are not supported out of the box.
Heretic runs in a Python 3.10+ environment with PyTorch 2.2 or later, supports optional bitsandbytes 4-bit quantization, and can save or upload generated models, launch a chat evaluation, and run standard benchmarks. An optional research installation provides residual-vector visualization using PaCMAP and residual-geometry analysis. The project is distributed under the GNU Affero General Public License version 3 or later.
3 uses taken from transcripts — each links to the moment in the video.
An open-source command-line tool that removes refusal behavior or safety alignment from transformer language models without post-training. It uses directional ablation and parameter optimization while attempting to preserve the model's original capabilities.
Automatically finds obliteration parameters and removes refusal behavior from a model.
An open source program PewDiePie used to decensor his model via ablation — identifying and removing the refusal behavior spread across the model's weights. The host notes it makes the process simple but warns ablating refusals can damage the model more broadly.
3 in the library.