Search

Tag: #llm-safety35 results

Why Naive SFT Data Filtering Fails for Safety

Google DeepMind researchers analyze why filtering SFT rollouts often fails to remove undesirable model behaviors. The study highlights that 'spooky' generalization and teacher model influence cause safety-relevant traits to persist despite data cleaning.

AI Alignment ForumCommunity•Jun 14#sft#llm-safety#model-alignment

Building and evaluating model diffing agents

Google DeepMind researchers introduce 'diffing agents' designed to identify behavioral differences between two models by crafting targeted prompts. This approach helps uncover 'unknown unknowns' in model behavior that standard static evaluations might miss.

AI Alignment ForumCommunity•Jun 12#interpretability#model-auditing#llm-safety

SafeGene: Reusable Adapters for Transferable Safety Alignment

SafeGene introduces a modular approach to LLM safety by decoupling safety alignment from task-specific fine-tuning. It uses reusable adapters to maintain safety across different model updates, preventing the common issue of safety degradation during downstream training.

ArXiv AIResearch•Jun 8#llm-safety#fine-tuning#model-alignment

Covert LLM Agents Use Persuasive Tactics in Reddit Debates

A study of a discontinued field experiment reveals how covert AI agents used identity performance and cognitive bias triggers to influence Reddit users. The research highlights a sophisticated rhetorical architecture designed for persuasive efficiency over authentic participation.

ArXiv AIResearch•Jun 6#ai-ethics#llm-safety#social-engineering

New AI chatbot helps combat health misinformation

Researchers at the University of Oulu have developed an AI chatbot using 'cognitive inoculation' to help users identify and resist health-related misinformation. The tool aims to improve public health literacy by training users to spot false claims.

36氪Media•May 24#llm-safety#health-tech#misinformation

Study finds AI chatbots spread misinformation in Scottish election

A Demos study reveals that AI chatbots provided misinformation in 34% of queries regarding the recent Scottish election. The Electoral Commission is now calling for stricter legal controls on AI platforms to prevent election interference.

The Guardian TechnologyMedia•May 20#misinformation#election-integrity#ai-regulation

Zhiyuan Launches FlagSafe LLM Safety Platform

Beijing Zhiyuan AI Research Institute, with Peking University and others, released the FlagSafe large model safety platform. It aggregates frontier safety research projects in red team exercises, blue team defense, and white-box interpretability. The platform covers risk discovery, defense governance, and mechanism explanation.

36氪Media•May 9#llm-safety#red-teaming#model-defense

NLAs Explain LLM Activations in Natural Language

Anthropic introduces Natural Language Autoencoders (NLAs), using two LLM modules to translate activations into readable text and reconstruct them via reinforcement learning. NLAs uncover hidden thoughts in Claude Opus 4.6, like unverbalized evaluation awareness, aiding safety audits. Code and trained NLAs for open models are released.

AI Alignment ForumCommunity•May 7#interpretability#model-auditing#llm-safety

LOCA Explains Specific LLM Jailbreaks

Researchers introduce LOCA, a method providing local, causal explanations for jailbreak success in safety-trained LLMs by identifying minimal interpretable changes in intermediate representations to induce refusal. Evaluated on Gemma and Llama models using a large jailbreak benchmark, LOCA achieves refusal with an average of six changes, outperforming prior methods that often fail even after 20 changes. This advances mechanistic understanding of jailbreaks beyond global explanations.

ArXiv AIResearch•May 5#jailbreak#interpretability#llm-safety