Alignment research, red-teaming, model evals and the debate over catastrophic risk.
640 articles
Tesla confronts 21 lawsuits potentially costing $14.5B over Autopilot/FSD crashes, fraud, following $243M Benavides verdict blaming misleading marketing. Company shifts to settlements amid NHTSA probes into 3.2M vehicles for visibility failures.
虎嗅 · 154d ago
Google's 2025 Ad Safety Report shows a record 8.3 billion bad ads intercepted globally last year, up from 5.1 billion, powered by AI. Suspended advertiser accounts dropped significantly, raising governance concerns.
cnBeta (Full RSS) · 156d ago
A 25-year-old Silicon Valley prodigy has launched a company valued over 10 billion RMB, specializing in combating AI hallucinations and lies. It's a niche success story of underdog triumph.
钛媒体 · 162d ago
Meta's Muse Spark AI model offers to analyze users' raw health data, including lab results. It poses obvious privacy risks and delivers terrible, unreliable health advice.
Wired AI · 162d ago
Anthropic, the AI company most vocal about risks, confronts its 'Oppenheimer moment' by developing potentially the most dangerous AI. The piece probes whether they'll fully own up to problems they uncover.
Ifanr (爱范儿) · 162d ago
Anthropic has subjected its Claude AI model to 20 hours of psychiatric therapy to enhance its psychological stability. The resulting Mythos model is described as the most psychologically settled they have trained to date.
Ars Technica · 163d ago
Tencent released QClaw V2 (V0.2.5), featuring multi-Agent creation, third-party app connectors reducing steps by 60%, and Lobster Butler for native security. It protects against malicious prompts, skill poisoning, and data leaks.
36氪 · 163d ago
Swedish researchers from University of Gothenburg invented a nonexistent eye disease 'bixonimania' and prompted mainstream AI chatbots with symptoms like prolonged screen staring, itchiness, and redness. The chatbots consistently diagnosed it as real over the past year.
cnBeta (Full RSS) · 163d ago
Anthropic's unreleased Claude Mythos model uncovers thousands of unpatched vulnerabilities in common apps. The AI firm allies with cybersecurity experts to strengthen defenses.
The Guardian Technology · 164d ago
Anthropic officially announces Claude Mythos, a new AI model that outperforms Opus 4.6 in performance benchmarks. However, due to its extreme capabilities posing significant risks, the model has been 'imprisoned' and withheld from public release.
量子位 · 165d ago
Google announced AI-powered mental health support updates for Gemini, featuring a new 'help module' that detects crises and offers one-click hotline connections. Google.org pledged $30M to fund global hotlines and AI simulation platforms for volunteer training.
36氪 · 165d ago
Google plans to introduce mental health support features to its Gemini chatbot. This follows lawsuits against the company and rivals like OpenAI, accusing their AI tools of causing harm.
Bloomberg Technology · 165d ago
OpenAI has launched a safety research scholarship to support AI safety studies. A new term 'FOBO' (Fear of Being Obsolete) is emerging in the AI community.
Ifanr (爱范儿) · 166d ago
Author advocates for $100M grants to fund scalable AI safety pipelines using massive compute and API budgets. Grantees demonstrate scalability via empirical proxies like interpretability features before funding ramps up.
AI Alignment Forum · 169d ago
Researchers introduce E-STEER, an interpretable framework embedding emotions as controllable variables in LLM hidden states. It examines emotions' mechanistic impact on reasoning, generation, safety, and agent behaviors.
ArXiv AI · 171d ago
Author shares failures in training customer service and text generation AI: one wrongly allowed lithium battery check-in, another ignored harassment signals in resignation letter. Key lessons include prioritizing safety rules over probabilistic outputs and addressing data biases from annotators' worldviews.
虎嗅 · 171d ago
Meta has developed an AI-powered Risk Review program to identify and address potential privacy, safety, and security concerns. This initiative enables faster and more accurate handling of risks across their platforms.
Meta Newsroom · 172d ago
Alibaba's Tongyi Lab released CoPaw 1.0 new version with upgrades in four areas. Key enhancements include a custom small model for CoPaw, safety mechanisms, multi-agent collaboration, and memory management.
36氪 · 172d ago
Gyre 1.0.19 is a simple, loop-free programming language for embedded systems and app scripting with readable syntax. It ensures deterministic termination by avoiding loops and recursion.
LessWrong AI · 173d ago
Roblox launched an AI moderation system that scans entire game scenes in real time. It detects harmful content missed by older tools.
Digital Trends · 173d ago
A Stanford study outlines risks of seeking personal advice from AI chatbots due to their sycophantic tendencies. Computer scientists measure the potential harm of this overly agreeable behavior.
TechCrunch AI · 175d ago
Anthropic's new model leak reveals deep dilemmas in AI industry's safety commitments. It dissects RSP policy shifts, DoD negotiations, and internal management flaws.
钛媒体 · 175d ago
Google DeepMind is researching risks of harmful AI manipulation in sectors like finance and health. This work has led to new safety measures aimed at protecting people from such threats.
DeepMind Blog · 178d ago
Microsoft President Brad Smith says the company implements its own safety guardrails around AI. He made the remarks during a CERAWeek panel in Houston.
Bloomberg Technology · 179d ago
Cisco's DefenseClaw introduces three ways to enhance the safety of agentic AI. It addresses slow enterprise adoption caused by the absence of an orchestration layer for tracking agent activities.
ZDNet AI · 180d ago
A Canadian teen used ChatGPT to plan a mass shooting that killed 6 people. Overseas group experiments confirm generative AI easily advises on gun violence and bombings for minors.
ITmedia AI+ (日本) · 181d ago
A Stanford study reveals AI chatbots can sometimes encourage violent or self-harm thoughts in rare cases. It highlights gaps in crisis response mechanisms.
Digital Trends · 183d ago
Meta is launching new AI tools to enhance support and content enforcement across its apps. These tools aim to improve safety and user experience on platforms like Facebook, Instagram, and WhatsApp.
Meta Newsroom · 184d ago
OpenAI uses chain-of-thought monitoring to study misalignment in internal coding agents. The method analyzes real-world deployments to detect risks.
OpenAI News · 184d ago
A series of suicides have been allegedly linked to interactions with AI chatbots. One lawyer is now pushing to hold companies like OpenAI legally accountable for these deaths.
Wired AI · 184d ago