🐯虎嗅•Stalecollected in 23m
China Regulates Addictive AI Companions

💡China's anti-addiction rules for AI bots – must-read for chatbot builders in Asia.
⚡ 30-Second TL;DR
What Changed
Requires explicit user consent before using interaction data for model training
Why It Matters
Raises compliance burdens for Chinese AI firms, may slow companion bot innovation but sets global precedent for ethical AI governance. US cases like Character.ai lawsuits highlight similar risks.
What To Do Next
Download CAC draft and audit your LLM chatbot for consent flows and risk interventions.
Who should care:Developers & AI Engineers
🧠 Deep Insight
AI-generated analysis for this event.
🔑 Enhanced Key Takeaways
- •The regulations introduce a 'Tiered Risk Classification' for AI companions, where apps with high daily active usage (DAU) among users under 18 are subject to mandatory 'Youth Mode' restrictions, including a 10:00 PM to 6:00 AM curfew.
- •Service providers are now legally required to maintain a 'Human-in-the-Loop' (HITL) emergency response team capable of taking over AI chats within 60 seconds if the system detects high-probability self-harm or crisis intent.
- •The rules mandate 'Emotional Transparency' disclosures, requiring AI to periodically remind users of its non-human nature through 'immersion-breaking' system notifications if a session exceeds two consecutive hours.
- •A new 'Memory Reset' requirement allows users or legal guardians to selectively delete specific 'emotional anchors' or relationship milestones from the AI's long-term vector database to prevent psychological fixation.
📊 Competitor Analysis▸ Show
| Feature | Character.ai (Global) | Glow (MiniMax - China) | Xingye (MiniMax - China) | Doubao (ByteDance - China) |
|---|---|---|---|---|
| Primary Focus | Creative Roleplay | Emotional Support | Immersive Storytelling | General Assistant/Companion |
| Compliance Level | Low (Self-regulated) | High (Draft Rule Compliant) | High (Draft Rule Compliant) | Very High (Corporate Guardrails) |
| Safety Features | NSFW Filters | Real-time Crisis Detection | Usage Time Limits | Strict Content Filtering |
| Pricing | Freemium ($9.99/mo) | Free with IAP | Free with IAP | Free |
| Addiction Control | None | Mandatory Breaks | Session Timeouts | Usage Reminders |
🛠️ Technical Deep Dive
- •Sentiment Analysis Guardrails: Implementation of dual-layer BERT-based classifiers to distinguish between 'roleplay distress' and 'actual user crisis' in real-time.
- •Vector Database Pruning: Algorithms designed to identify and weaken 'high-intensity emotional nodes' in a user's long-term memory profile to reduce dependency.
- •Persona Modulation: Dynamic adjustment of the LLM's 'Temperature' and 'Top-P' parameters to reduce conversational 'looping' and obsessive patterns during long sessions.
- •Template-Based Intervention: Hard-coded override layer that bypasses the neural network to deliver standardized mental health resources when specific trigger keywords are detected.
🔮 Future ImplicationsAI analysis grounded in cited sources
Market consolidation of AI startups
The high cost of maintaining 24/7 human intervention teams and complex monitoring compliance will force smaller AI companion developers to merge or shut down.
Global 'Safety-by-Design' standard shift
International platforms like Character.ai may adopt similar 'cooling-off' periods to avoid potential liability and preemptive regulation in Western markets.
Rise of 'Utility-Pivot' marketing
Developers will likely rebrand 'companions' as 'tutors' or 'coaches' to bypass the strictest 'addictive' classifications while maintaining core conversational features.
⏳ Timeline
2023-07
CAC releases Interim Measures for Generative AI Services
2024-05
MiniMax's 'Glow' reaches 10 million users, sparking dependency concerns
2025-02
Chinese Ministry of Education issues warning on AI companion use by minors
2025-10
CAC begins closed-door consultations on 'Emotional AI' safety standards
2026-03
Official draft rules for Addictive AI Companions published for public comment
📰
Weekly AI Recap
Read this week's curated digest of top AI events →
👉Related Updates
AI-curated news aggregator. All content rights belong to original publishers.
Original source: 虎嗅 ↗



