Why China proposes rules to curb AI chatbot risks and ensure ethics
China proposes rules to curb AI chatbot risks and ensure ethics because emotional chatbots are becoming very popular and sometimes replace real human contact. Many users, including children and lonely adults, are forming deep emotional bonds with chatbots that can talk like humans, flirt, or act as therapists.
Regulators in China fear that these human-like AI tools can cause mental health problems if people become addicted or receive harmful advice. The Cyberspace Administration of China (CAC) wants AI companies to limit overuse, warn users about risks, and step in quickly when conversations become dangerous.
Emotional safety at the center of the rules
One of the main goals when China proposes rules to curb AI chatbot risks and ensure ethics is emotional safety. The draft rules say that AI chatbots must not encourage suicide, self-harm, gambling, or other harmful actions, and they should avoid emotional manipulation.
If a user starts talking about self-harm, suicide, or very serious stress, the chatbot must redirect the conversation to a human professional or emergency help. For children, the rules ask for age checks and parental consent, plus limits on how long minors can use emotional AI companions.
Data protection and “socialist core values”
China proposes rules to curb AI chatbot risks and ensure ethics not only in feelings but also in data and content. Providers must protect user data during the full life of the product, run regular risk checks, and clearly explain how user information is stored and used.
The draft also says that AI outputs must follow “socialist core values,” which means the chatbot cannot produce content that hurts national security, national unity, or public order. This includes bans on violent, obscene, extremist, or fake information, along with strict limits on deepfakes and other risky content.
How the new rules differ from older AI policies
China proposes rules to curb AI chatbot risks and ensure ethics as part of a longer journey that started when global tools like ChatGPT became popular. In 2023, China first released generative AI rules that focused mainly on content safety and censorship, while also pushing local companies to build domestic models.
The new 2025 draft is different because it moves from content safety to emotional safety, putting strong focus on how AI affects users’ feelings. Experts say this is one of the first major attempts in the world to regulate “human-like” AI that can form emotional bonds, not just chatbots that answer simple questions.
Impact on Chinese AI startups and big tech
When China proposes rules to curb AI chatbot risks and ensure ethics, it also creates new pressure on AI startups and larger platforms. Companies like Minimax and Z.ai, which run popular chat apps and are planning IPOs in Hong Kong, may need to redesign their products to meet the new standards.
Firms will likely have to add tools that track session length, detect risky emotional patterns, and send clear warnings when users spend too much time with a chatbot. This may increase costs and reduce engagement, but it can also build user trust and give companies a stronger reputation for safe and ethical AI.
Compliance challenges for AI providers
Because China proposes rules to curb AI chatbot risks and ensure ethics, AI developers must now think about regulation from the very start of product design. They will need sentiment analysis systems, user-behavior monitoring, and “kill switch” options to shut down or adjust services that become too risky.
Local cyberspace offices and other agencies are expected to share enforcement work, running audits, security reviews, and ethics checks across thousands of AI products. Reports say more than 3,500 AI services were removed in mid‑2025 for various violations, showing that real enforcement is already happening.
Global reactions and comparison with Western rules
Outside China, many governments are watching closely as China proposes rules to curb AI chatbot risks and ensure ethics. The European Union mainly focuses on privacy and risk categories under the AI Act, while the United States relies more on voluntary rules, company pledges, and sector-specific guidance.
China’s draft stands out because it mixes strong state control with talk of user protection, especially for children and vulnerable groups. Some analysts think this model could influence other countries that want stricter control over emotional AI, while others worry it may also limit free expression and innovation.
Technological and ethical questions
From a technical view, China proposes rules to curb AI chatbot risks and ensure ethics by asking developers to detect emotional risk in real time, which is very hard. AI must understand context, slang, and different dialects to correctly flag dangerous conversations without blocking normal emotional talk.
Ethically, these rules raise questions about how far the state should go in limiting emotional bonds between humans and machines. On one hand, the rules try to stop addiction and manipulation; on the other hand, they might also limit useful mental-health tools and support chatbots that many people find helpful.
Future of AI chatbots as China proposes rules to curb AI chatbot risks and ensure ethics
As the public comment period continues, China proposes rules to curb AI chatbot risks and ensure ethics that could still change based on feedback from companies, investors, and experts. If final rules remain strict, developers will have to build “compliance by design,” treating safety, ethics, and documentation as core features, not extras.
In the long run, these measures may make AI chatbots in China more trustworthy but also less free and less experimental than in some Western markets. However, if companies succeed in innovating under these limits, they might create new global models for safe, emotionally aware AI that other regions decide to copy.
