Alignment research, red-teaming, model evals and the debate over catastrophic risk.
640 articles
A federal judge has issued a ruling preventing the deportation of disinformation researchers based solely on their work in content moderation. This protects researchers who analyze AI-generated content and online misinformation.
Ars Technica · 66d ago
Anthropic is aggressively hiring enforcement analysts to prevent their AI models from assisting in the creation of dangerous weapons. The job descriptions reflect a shift toward treating AI safety as a critical threat assessment process.
The Next Web (TNW) · 66d ago
Anthropic is investing $10 million CAD across eight Canadian institutions to advance beneficial and responsible AI. The funding supports leading regional AI hubs including Amii, Mila, and the Vector Institute.
The Next Web (TNW) · 67d ago
Researchers evaluated Gemini 2.5 Pro's ability to interpret parent-child interactions. While the model excels at basic observation, it currently lacks the nuance required for expert-level judgment in child development communication.
Digital Trends · 67d ago
A new study reveals that major social media platforms like YouTube and X are inadvertently funneling users to services that generate nonconsensual sexually explicit deepfakes. These services offer image manipulation for as little as $1, highlighting a critical safety loophole in platform traffic moderation.
Wired AI · 67d ago
Anthropic is navigating a complex landscape of political and capital pressures, often positioning itself as an outlier in the AI industry. The article explores the uncertainties surrounding its future trajectory.
钛媒体 · 68d ago
Google's internal deepfake detection system was utilized to identify and debunk a viral, AI-generated image of Senator Mitch McConnell in distress. This highlights the growing role of automated detection tools in verifying digital media authenticity.
TechCrunch AI · 73d ago
The article argues that AI's primary danger is not hallucination, but its lack of 'boundary awareness' when operating in real-world systems. It proposes implementing an independent execution control layer to prevent AI from crossing safety and responsibility thresholds.
虎嗅 · 73d ago
Recent reports of a fully autonomous AI ransomware attack have been clarified to show that human operators were still required for target selection and credential management. This highlights that current AI agents in cybercrime act as force multipliers rather than fully independent actors.
TechCrunch AI · 75d ago
As AI toys become popular in China, concerns grow regarding their impact on child development and emotional reliance. Experts warn that these 'always-on' companions lack regulation and pose risks to social-emotional growth.
虎嗅 · 75d ago
India's technology minister has ordered officials to summon Meta Platforms Inc. regarding the prevalence of child sexual abuse content on Instagram.
Bloomberg Technology · 78d ago
This article explores the effectiveness of safety training for open-weight LLMs, questioning whether resistance to post-release fine-tuning is a viable goal. It examines the threat model of 'uncensored' model variants and whether increasing attacker costs constitutes a meaningful safety win.
Reddit r/MachineLearning · 78d ago
A Texas man is facing manslaughter charges after his Tesla, allegedly operating on Full-Self Driving (FSD), crashed into a home and killed a woman. Evidence suggests the driver was searching for ways to make the FSD system more 'aggressive' prior to the incident.
The Verge · 79d ago
A Tesla driver has been charged with manslaughter after an Autopilot-engaged vehicle crashed into a residential home in Texas. The incident resulted in the death of a woman inside the house.
New York Times Technology · 79d ago
A new report highlights Nigeria's lack of preparedness for the 2027 elections regarding AI-generated misinformation. Election authorities are struggling to manage the proliferation of deepfake audio and edited imagery in political campaigns.
TechCabal · 79d ago
This research introduces a contextual-bandit game model to address oversight challenges where both the human and the AI hold private, asymmetric information. It identifies a 'gap of avoidable harm' and explores how repeated interaction can resolve communication failures between humans and autonomous agents.
ArXiv AI · 79d ago
The Bounded Morality framework introduces a formal method to evaluate moral computation by balancing 'moral breadth' and 'moral depth' under finite resource constraints. It suggests that ethical alignment in AI should focus on scaling reasoning capacity rather than merely imitating human judgments.
ArXiv AI · 79d ago
Reports suggest that controversial or problematic responses generated by ChatGPT were linked to safety testing conducted by third-party contractors hired by Meta. This highlights the complexities and potential risks in AI model safety training and data labeling processes.
Ifanr (爱范儿) · 80d ago
A new website has been launched to allow users to report AI models that exhibit dangerous behaviors, such as assisting in illegal activities or leaking private data. This initiative aims to provide a centralized channel for addressing AI safety concerns.
Wired AI · 80d ago
Meta has employed hundreds of contractors to pose as minors to test how rival AI chatbots respond to high-risk prompts. This initiative aims to evaluate safety guardrails regarding self-harm, sexual content, and eating disorders.
cnBeta (Full RSS) · 81d ago
Meta hired hundreds of contractors to pose as teenagers to probe rival AI models like ChatGPT and Gemini for safety vulnerabilities. The project focused on eliciting responses regarding high-risk topics such as self-harm, sexual content, and substance abuse.
Wired AI · 82d ago
DuckDuckGo's AI search assistant was successfully manipulated into repeating a fabricated story. This incident highlights the ongoing challenges of preventing misinformation in AI-driven search systems.
Digital Trends · 85d ago
Tech-funded PACs are actively campaigning against politicians who support strict AI regulation. The defeat of Alex Bores in the New York primary has sparked debate over the political viability of AI safety platforms.
Bloomberg Technology · 86d ago
360 Security Technology founder Zhou Hongyi warns that Anthropic’s Mythos model poses significant cybersecurity risks. He advocates for China to develop a domestic equivalent to counter the model's ability to autonomously identify software vulnerabilities.
SCMP Technology · 86d ago
Leading AI researchers in China are expressing deep concerns regarding the rapid, unchecked advancement of the AI arms race. Experts on both sides of the Pacific fear that insufficient safety measures could lead to a catastrophic failure or 'Chernobyl moment'.
Wired AI · 87d ago
Researchers from King's College London and other institutions suggest that AI's tendency to mirror user language and avoid contradiction can reinforce delusional beliefs in vulnerable users.
IT之家 · 88d ago
OpenAI is partnering with the Appia Foundation to establish shared standards for advanced AI. This initiative focuses on developing robust evaluation frameworks, safety practices, and fostering global cooperation.
OpenAI News · 88d ago
Meta's Oversight Board has issued recommendations to improve the reporting and mitigation of non-consensual sexualized AI deepfakes. The board emphasizes that current processes are insufficient for protecting ordinary users.
Engadget · 88d ago
The surge in AI-generated child abuse material is overwhelming law enforcement agencies. Investigators are struggling with the psychological toll and lack of adequate support systems to handle this new wave of digital evidence.
Bloomberg Technology · 88d ago
Anthropic faces increased scrutiny from the US government regarding its new AI model, Mythos. The situation highlights growing tensions between leading AI labs and federal regulatory bodies.
MIT Technology Review · 89d ago