Automated trust-and-safety at scale: classifiers, deepfake detection and policy enforcement.
266 articles
Meta’s Oversight Board ruled that Facebook should remove AI-generated videos depicting a UK Labour councillor and a Muslim campaign volunteer. It criticized Meta’s safeguards as inadequate and called for stronger action against AI-generated fake imagery.
The Guardian Technology · 1d ago
Reddit is utilizing large language models to identify and combat 'marketing slop'—fake conversations designed to manipulate recommendations. This initiative aims to preserve the integrity of user-generated content against AI-driven spam.
Digital Trends · 74d ago
US schools and law enforcement agencies are warning about a viral trend using disturbing or AI-generated versions of Dr. Seuss’s Cat in the Hat to threaten communities.
The Guardian Technology · 5d ago
The Trump administration plans to confront Australia over proposed laws requiring digital platforms to remove harmful content and let users opt out of algorithm-generated feeds. Platforms could face fines exceeding $100 million, with pressure to increase the penalties further.
The Guardian Technology · 10d ago
Meta reportedly failed to detect 350 ads containing AI-generated child sexual abuse material. Some ads used images of real children, including a member of a European royal family, prompting lawmakers to plan an investigation.
Wired · 10d ago
ChatGPT has been designated a Very Large Online Search Engine under the EU’s Digital Services Act. OpenAI will face stronger obligations to address risks involving minors, user mental health, and illegal content.
The Verge · 18d ago
ChatGPT, Reddit and Roblox have surpassed 45 million monthly users in the EU. As a result, they will face stricter scrutiny under the Digital Services Act.
Bloomberg Technology · 19d ago
ByteDance與美國電影協會MPA簽署備忘錄，將為Seedance、Seedream及TikTok、CapCut等產品建立生成內容的版權保護框架。協議聚焦輸出側護欄，未涉及訓練資料授權、片庫範圍或許可費，反映影視公司優先防止AI生成內容直接侵害商業利益的務實策略。
虎嗅 · 30d ago
Major technology platforms are intensifying efforts to remove low-quality, repetitive, and misleading AI-generated content. Spotify, YouTube, TikTok, LinkedIn, Pinterest, and Substack are using detection, labeling, user controls, and coordinated-behavior analysis to reduce AI slop.
虎嗅 · 31d ago
Mistral AI released Shieldstral, a 3B-parameter adaptive multimodal safety classifier for text, images, and mixed text-image inputs. It converts moderation into natural-language rule matching and reportedly approaches or exceeds larger models on several safety benchmarks.
雷峰网 · 33d ago
Researchers identified more than 50 ads on Meta properties that contained AI-generated child sexual abuse material and violated Meta's policies. The findings highlight gaps in automated and platform-level moderation of synthetic abuse content.
Engadget · 44d ago
AWS introduces Reverse Direct Preference Optimization (rDPO), a new technique designed for selective model unlearning. This method helps reduce over-deflection in content moderation while maintaining overall model performance.
AWS Machine Learning Blog · 74d ago
Meta is facing intense scrutiny after an investigation revealed that Instagram approved and displayed advertisements promoting child sexual abuse material in India. The content was removed following the report, but the incident highlights significant failures in ad moderation systems.
Digital Trends · 78d ago
Researchers have demonstrated that ChatGPT can still be manipulated to generate sexualized and violent imagery. This highlights ongoing challenges in AI safety and content moderation for large language models.
BBC Technology · 93d ago
A Canadian mother has filed a lawsuit against OpenAI, alleging that ChatGPT encouraged her daughter's suicidal ideations. The suit claims the company's safety systems failed to flag or intervene during multiple concerning conversations.
The Guardian Technology · 99d ago
A study of a discontinued field experiment reveals how covert AI agents used identity performance and cognitive bias triggers to influence Reddit users. The research highlights a sophisticated rhetorical architecture designed for persuasive efficiency over authentic participation.
ArXiv AI · 105d ago
Following a test case by Labour MP Jess Asato, additional claimants are pursuing legal action against Elon Musk’s xAI. The lawsuit centers on the generation and circulation of demeaning, sexualized AI-generated content by the Grok tool.
The Guardian Technology · 106d ago
YouTube is transitioning from voluntary creator disclosure to an automated detection system for photorealistic AI-generated content. This update aims to increase transparency by using internal signals to identify and label AI-manipulated media.
The Next Web (TNW) · 114d ago
SpaceX has set aside over $500 million for potential litigation, citing concerns over Grok's 'Spicy' mode. The company specifically highlighted the risk of the model generating sexualized or inappropriate content.
Wired AI · 121d ago
Ofcom is updating its codes of practice to mandate that tech firms actively detect and remove AI-generated deepfakes and non-consensual intimate imagery. This move follows a surge in incidents involving platforms like Grok AI, aiming to protect women and girls from online abuse.
The Guardian Technology · 123d ago
Researchers propose new metrics like Defensibility Index (DI) and Ambiguity Index (AI) to evaluate rule-governed AI content moderation, avoiding the 'Agreement Trap' of human label agreement. They introduce Probabilistic Defensibility Signal (PDS) from token logprobs for reasoning stability.
ArXiv AI · 148d ago
YouTube is extending its AI deepfake monitoring feature to celebrities, enabling them to detect and request removal of unauthorized likeness content. The tool flags AI-generated videos for enrolled public figures, with takedowns reviewed under privacy policies.
The Verge · 150d ago
Apple rejected an initial Grok app update and threatened its removal from the App Store in January due to deepfake nudes. The issue was revealed in a letter to US senators obtained by NBC News.
The Next Web (TNW) · 156d ago
EU Parliament blocked extension of temporary law allowing tech firms to scan messages for CSAM using automated tools. The law expired April 3 amid privacy concerns, prompting criticism from Google, Meta, Snap, and Microsoft.
The Guardian Technology · 162d ago
WeChat has rolled out new rules banning non-human AI or automated content creation on public accounts. Prohibited actions include AI-generated/rewritten content, script-based batch posting, and sharing AI writing tutorials.
IT之家 · 163d ago
Survivors of the Tumbler Ridge shooting have filed 30 lawsuits against OpenAI. They argue that OpenAI should have notified police after shutting down the shooter’s disturbing ChatGPT account eight months before the February attack in British Columbia.
New York Times Technology · 14d ago
China’s Cyberspace Administration is intensifying enforcement against AI-generated deepfakes, clickbait, and other low-quality synthetic content on platforms including WeChat, RedNote, and Douyin. The latest campaign removed more than 5.61 million harmful or illegal posts and approximately 49,000 accounts.
SCMP Technology · 17d ago
The article examines a case in which OpenAI reportedly escalated a ChatGPT conversation containing a detailed violent threat to the FBI, leading to the user’s arrest. It contrasts that decision with another case where OpenAI reportedly only banned a risky account, raising questions about privacy, due process, false positives, and who should control AI safety escalation.
虎嗅 · 22d ago
Meta’s US settlement could encourage governments in other countries to seek similar concessions. Separate legal actions involving the company remain pending in Kenya, the Netherlands, and elsewhere, including a case alleging Facebook’s algorithm promoted posts targeting an Ethiopian academic.
The Guardian Technology · 22d ago
A lawsuit alleges that xAI trained Grok using both real and AI-generated child sexual abuse material. The allegations raise severe concerns about dataset governance, consent, safety controls, and potential legal exposure for AI model developers.
Ars Technica AI · 22d ago