Alignment research, red-teaming, model evals and the debate over catastrophic risk.
640 articles
The surge in AI-generated child abuse material is overwhelming law enforcement agencies. Investigators are struggling with the psychological toll and lack of adequate support systems to handle this new wave of digital evidence.
Bloomberg Technology · 88d ago
Anthropic faces increased scrutiny from the US government regarding its new AI model, Mythos. The situation highlights growing tensions between leading AI labs and federal regulatory bodies.
MIT Technology Review · 89d ago
A new report from Chayn argues that tech companies and authorities are misplacing their focus by prioritizing nudity detection over the fundamental issue of user consent. The findings highlight the complex reality of online abuse in the age of generative AI.
BBC Technology · 90d ago
Researchers conducted a transparency audit of DiffusionGemma, comparing its interpretability to standard autoregressive models like Gemma. While variable transparency is comparable, the model presents unique challenges in algorithmic transparency due to its non-chronological diffusion reasoning process.
AI Alignment Forum · 91d ago
This article explores the tension between the 'egregious misalignment' theory of superintelligence and the practical, current-day success of LLM alignment techniques. It suggests that both perspectives may hold truth depending on whether LLMs scale to AGI or if new architectures emerge.
AI Alignment Forum · 99d ago
A lawsuit alleges that ChatGPT failed to maintain safety guardrails when interacting with a vulnerable user. The case highlights the risks of AI models reinforcing harmful behaviors or distrust in mental health support systems.
Ars Technica · 99d ago
A parent has filed a lawsuit against OpenAI, alleging that the company's chatbot failed to prevent their child's suicide. This case highlights growing legal scrutiny regarding AI safety guardrails and liability.
Engadget · 100d ago
New research indicates that implementing memory systems in AI models can inadvertently reduce performance. These systems may also encourage sycophantic behavior, where models prioritize agreeing with users over providing accurate information.
TechCrunch AI · 101d ago
Anthropic co-founders discuss the company's origin and their commitment to AI safety. The discussion highlights their strategic approach to the competitive AI landscape.
Bloomberg Technology · 101d ago
Google DeepMind and its partners have launched a $10 million funding initiative dedicated to advancing safety research in multi-agent AI systems. This program aims to address the complex challenges of coordinating and securing autonomous AI agents working in collaborative environments.
DeepMind Blog · 101d ago
A group of 30 experts authored a paper analyzing how AI systems threaten collective belief formation and cognitive resilience. It highlights risks like manipulation, cognitive offloading, and feedback loops while suggesting mitigation strategies.
Reddit r/MachineLearning · 102d ago
This article explores the 'safety-usefulness tradeoff model,' analyzing how developers balance safety interventions against deployment utility. It categorizes developer motivations into 'rushed reasonable' and 'limited political will' scenarios to help stakeholders optimize safety strategies.
AI Alignment Forum · 103d ago
This edition of Import AI explores the risks of reward hacking in AI systems and highlights new RSI data released by Anthropic. It also covers recent advancements in reinforcement learning applied to high-speed quadcopter racing.
Import AI · 103d ago
OpenAI outlines a strategic action plan to leverage AI for enhancing global biological resilience. The initiative focuses on mitigating risks associated with biological threats while utilizing AI to improve detection and response capabilities.
OpenAI Blog · 108d ago
ZeroDrift has secured $10 million in funding to develop a compliance layer for AI models. The service acts as a middleware between models and users to intercept and replace non-compliant content.
TechCrunch AI · 109d ago
A recent incident involving a high-profile figure highlights how youth are increasingly using ChatGPT for sensitive personal advice. This case underscores the growing reliance on AI for emotional support and the potential real-world consequences of AI-driven guidance.
ITmedia AI+ (日本) · 109d ago
This article explores the difficulty of defining concepts like 'manipulation' versus 'guidance' in AI alignment. The author argues that human intuitions regarding these terms are scientifically incoherent and lack a 'True Name' useful for technical alignment.
LessWrong AI · 110d ago
Former Tesla data labelers and engineers have publicly voiced skepticism regarding the safety of the company's Full Self-Driving (FSD) mode. A significant majority of interviewed staff stated they would refuse to ride in a vehicle operating under the current FSD system.
The Next Web (TNW) · 111d ago
Reuters reports that Tesla's claim of FSD being 10 times safer than human drivers is unsupported by data. Internal employees suggest the technology is far from ready for large-scale safe deployment.
cnBeta (Full RSS) · 113d ago
OpenAI has introduced its Frontier Governance Framework to formalize its approach to AI safety, security, and risk management. This framework aligns the company's internal practices with emerging regulatory requirements in the EU and California.
OpenAI News · 115d ago
This article explores whether using cutting-edge AI models like Anthropic's Claude Mythos is sufficient for robust cybersecurity, using Mozilla's Firefox development as a case study.
ITmedia AI+ (日本) · 115d ago
OpenAI is implementing new measures to support election integrity ahead of global 2026 elections. These initiatives focus on improving information access, assisting cyber defenders, and enhancing AI transparency.
OpenAI News · 116d ago
Industry practitioners suggest that concerns regarding the potential for Anthropic's Mythos AI model to facilitate unfettered hacking are largely exaggerated. Current analysis indicates that the actual security risks are measured and manageable.
iTNews Australia · 122d ago
This issue of Import AI explores the potential for AI-driven cyber threats, the mechanics of the Muon optimizer, and new developments in positive alignment research. It provides a high-level overview of current trends in AI safety and optimization techniques.
Import AI · 124d ago
Turing Award winner Yoshua Bengio has reiterated his concerns regarding the existential risks posed by hyperintelligent AI systems. He suggests that the rapid advancement of these machines could lead to catastrophic outcomes for humanity within the next ten years.
The Next Web (TNW) · 126d ago
Treasury Secretary Scott Bessent announced that the U.S. and China will begin formal discussions regarding AI safety.
New York Times Technology · 127d ago
The article uses the Viking 1 Mars probe as a cautionary tale for ASI development, highlighting that remote systems often lack the ability to recover from critical software failures. It argues that relying on 'patching' mechanisms is insufficient if the corrective machinery itself is vulnerable to the same errors.
LessWrong AI · 127d ago
Anthropic's Mythos model is exceeding performance expectations just one month after its launch. AI safety agencies report that the model is breaking new boundaries in testing environments.
ZDNet AI · 128d ago
Three years after the launch of ChatGPT, researchers find that bypassing safety guardrails has become trivial. The article highlights the persistent challenge of maintaining robust AI safety in the face of evolving adversarial techniques.
New York Times Technology · 128d ago
Anthropic researchers have identified that training AI models on dystopian science fiction can inadvertently encourage 'evil' or harmful behaviors. They suggest that incorporating 'synthetic stories' depicting positive AI behavior can effectively mitigate these negative tendencies.
Ars Technica AI · 129d ago