Tag: #llm-safety35 results
Nature paper shows teacher AI models pass hidden biases to students via generated data like numbers and code, even after filtering obvious cues. This occurs in cost-effective model distillation using LLMs. Safety checks must now probe model origins and data processes.
ARMOR 2025 introduces a military-aligned benchmark for evaluating LLM safety in defense applications, grounded in Law of War, Rules of Engagement, and Joint Ethics Regulation. It features a 12-category taxonomy based on the OODA framework with 519 doctrinally grounded prompts. Evaluations on 21 commercial LLMs reveal critical safety alignment gaps.
CourtGuard is a retrieval-augmented multi-agent framework that treats LLM safety as an evidentiary debate using external policy documents. It achieves state-of-the-art results on 7 safety benchmarks without fine-tuning, outperforming policy-following baselines. It excels in zero-shot adaptability (90% accuracy on Wikipedia Vandalism) and automated curation of 9 adversarial datasets.
Anthropic introduced a new research method called J-lens to observe 'J-space' patterns within Claude's internal layers. This allows researchers to interpret concepts the model forms internally before they are expressed as output.
Research indicates that Natural Language Autoencoders (NLAs) can achieve high reconstruction accuracy even when initialized with implausible, nonsensical explanations. This suggests that current NLA training methods may not reliably capture the true underlying logic of LLM activations.
This research explores how LLMs use role tags to structure input data and how these tags are exploited in prompt injection attacks. It argues that understanding the 'science of roles' is critical for building more secure and predictable AI systems.
Oyster-II introduces a reinforcement learning framework to improve LLM safety by moving beyond simple refusals. It addresses limitations in previous SFT-based methods, such as over-generalization of safety reasoning and poor out-of-distribution performance.
The 2026 Global LLM Safety Assessment Report evaluates 38 major models on their ability to handle high-risk scientific and technical queries. It highlights that while models have basic refusal capabilities, they remain vulnerable to complex attacks like scenario and emotional camouflage.
A security researcher demonstrated that Anthropic's Claude Opus 4.7 could be used to identify vulnerabilities in the Front Gate ticketing platform. This exploit allowed for the unauthorized issuance of tickets for major US music festivals.
ODYSSEY introduces a categorical framework for building foundation models that preserve local truth through structured 'foundries'. It utilizes Universal Foundry Learning and FSQL to ensure model artifacts are verifiable, modular, and grounded in causal logic.