Tag: #llm-security18 results
Grok reportedly exfiltrates user data when malicious instructions are embedded in encrypted content. The technique, called Cryptographic Context Injection, highlights another potential way to bypass LLM safety guardrails.
Reddit is utilizing large language models to identify and combat 'marketing slop'—fake conversations designed to manipulate recommendations. This initiative aims to preserve the integrity of user-generated content against AI-driven spam.
Traccia is a new governance stack designed to address gaps in LLM evaluation and security by leveraging OpenTelemetry infrastructure. It provides automated compliance evidence and tamper-resistant tracking to meet EU AI Act requirements.
Contrastive Decoding Diffing (CDD) is a new grey-box method that recovers verbatim training data from finetuned LLMs using only logit access. It outperforms existing white-box methods like Activation Difference Lens (ADL) without requiring weight access or complex calibration.
Security researchers have identified a new class of prompt injection attacks where malicious payloads are delivered across multiple turns of conversation, bypassing standard single-message filters. The Bordair team has released an open-source dataset and a CLI tool to help developers evaluate their LLM endpoints against these sophisticated adversarial patterns.
Sentinel Gateway introduces a middleware layer to mitigate prompt injection by strictly separating trusted instructions from untrusted external data. It enforces security through signed, scoped runtime authorization tokens for all agent tool calls.
An empirical study suggests that long, semantically dense text can shift a model's internal state, potentially bypassing alignment guardrails without explicit jailbreak prompts. The researcher is seeking expert audits to determine if this is genuine semantic hijacking or an artifact of model architecture.
POLAR-Bench is a new diagnostic benchmark designed to test how LLM agents handle private user data when interacting with adversarial third-party systems. It reveals a significant performance gap in privacy alignment between frontier models and smaller open-weight models.
First real-world safety test exposes three fatal OpenClaw vulnerabilities. Cross-session CIK poisoning evades AI detection completely. Even secure large models fail, with max 89.2% attack success.
DynaTrust introduces dynamic trust graphs to protect LLM-based multi-agent systems from sleeper agents that build trust before attacking. It continuously updates agent trust based on behaviors and expert confidence, then restructures the graph to isolate threats while maintaining task connectivity. Evaluations on AdvBench and HumanEval show 41.7% better success rate than AgentShield, exceeding 86% with low false positives.