Tagged:
fine-tuning5 posts
-
Model Hub Supply Chain Attacks: Malicious Models, Tokenizer Exploits, and Typosquatting on Hugging Face
Downloading an open-weight model from a public hub is not a read-only operation. Custom tokenizer classes and auto_map configs execute arbitrary Python when trust_remote_code is set, LoRA adapters can trojanize safe base models, and typosquatted namespaces are a documented distribution vector. Here's the threat model practitioners need before they run from_pretrained().
-
Backdoor Attacks in Foundation Models: Sleeper Triggers That Survive Fine-Tuning
Pre-trained LLMs can be trojaned at the foundation stage, with adversarial triggers embedded in weights that persist through downstream fine-tuning and RLHF safety training. This post explains how these attacks work, why they're so persistent, and what practitioners can do about them.
-
Jailbreak Robustness After Fine-Tuning: How Safety Alignment Degrades
Safety fine-tuning instills refusal behaviors in LLMs — but those behaviors are surprisingly brittle under subsequent fine-tuning. This post explains the mechanism, the empirical evidence, and what enterprise deployers can do about it.
-
Fine-Tuning Trojans: Injecting Backdoors Through the Model Training Pipeline
How malicious training data, tampered datasets, and compromised fine-tuning APIs plant backdoored behavior in legitimate base models — and what defenders can do.
-
ALIGNBEAM: Transferring Safety Alignment Across Model Families at Inference Time
Domain fine-tuning erodes safety alignment — and existing logit-mixing defenses don't work when the draft and safety models use different vocabularies. ALIGNBEAM solves this with a training-free text-bridge that translates anchor logits into any target vocabulary, raising AdvBench refusal from 38% to 92% without touching either model's weights.