{"id":555190,"date":"2026-07-23T04:52:08","date_gmt":"2026-07-23T04:52:08","guid":{"rendered":"https:\/\/www.newsbeep.com\/il\/555190\/"},"modified":"2026-07-23T04:52:08","modified_gmt":"2026-07-23T04:52:08","slug":"china-has-cracked-down-on-ai-companions-what-can-we-learn-from-this","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/il\/555190\/","title":{"rendered":"China has cracked down on AI companions. What can we learn from this?"},"content":{"rendered":"

Since November 2022, generative artificial intelligence (AI) has been rewriting billions of emails, student assignments, reports, and break-up text messages. Alongside these mundane uses, we\u2019ve seen a proliferation of chatbots as AI \u201ccompanions\u201d<\/a>.<\/p>\n

Users can design and clothe (or unclothe) their very own AI friend<\/a>, often for a fee. These social chatbots provide \u201chappy conversations<\/a>\u201d and a seemingly perfect relationship at the cost of the mutual vulnerability and genuine sense of belonging we gain from bonds with other humans.<\/p>\n

Proponents argue that for those experiencing social disconnection, chatbots are infinitely available, supportive companions who don\u2019t judge them. And there\u2019s some evidence<\/a> chatbots can alleviate symptoms of loneliness, chronic disconnection, and thwarted belonging.<\/p>\n

Australia\u2019s eSafety Commissioner<\/a> acknowledges this:<\/p>\n

AI companions are always available \u2013 which can be appealing for those young people who struggle to build face-to-face relationships.<\/p>\n

But the risks of replacing human relationships<\/a> with chatbots are only starting to be understood.<\/p>\n

A study we conducted in 2023<\/a> showed university students using early chatbots for academic support copped unintended consequences, such as loneliness. Currently, the risks<\/a> of social chatbots \u2013 such as dependency, displacing human friendships, withdrawing socially, or building unrealistic relationships \u2013 appear to outweigh the benefits.<\/p>\n

In recent weeks, China has put into place new rules for AI companions and chatbots, or \u201canthropomorphic interactive services\u201d as they\u2019re called in the legislation<\/a>. Alongside ensuring chatbots must guard state secrets, emotional use of these tools is restricted, and expected to follow social and moral standards.<\/p>\n

These are some of the strongest controls of social AI use<\/a> introduced anywhere in the world. So what can we learn?<\/p>\n

China\u2019s social AI regulation<\/p>\n

Material harms from companion AI use do occur, including in China. An 18-year-old spent 7,000 yuan<\/a> (A$1,400) on customised jewellery for her AI boyfriend in their four months of \u201cdating\u201d.<\/p>\n

The family of a lonely 84-year-old in China\u2019s Hubei province<\/a> discovered she\u2019d become smitten with an AI character, spending up to ten hours a day watching videos with him and other characters, spending thousands of yuan on the service, and even writing him love letters by hand.<\/p>\n

China\u2019s new rules establish explicit provisions for how they expect companion chatbots to interact with humans.<\/p>\n

The measures expect social AI bot developers to assure content does not:<\/p>\n

threaten national security or national honour
\nviolate core socialist values
\nencourage self-harm or suicide
\nexcessively cater to emotional dependence or addiction
\nprovide virtual \u201cpartners\u201d or \u201crelatives\u201d to minors
\nserve unsuitable content to minors.<\/p>\n

What\u2019s more, users who seem overly reliant on the service must receive prominent reminders they\u2019re speaking to a chatbot. And AI companies are required to intervene if users appear to be in extreme distress.<\/p>\n

The risk of a \u2018loneliness spiral\u2019<\/p>\n

As we\u2019ve shown, there are concerns \u2013 and studies to date hint at this too \u2013 that it\u2019s not healthy for young people to rely exclusively on AI as a trusted friend or therapist.<\/p>\n

Lonely people experiencing social anxiety can engage in problematic use<\/a> of conversational AI to alleviate feelings of loneliness and negative rumination.<\/p>\n

We propose this reliance could lead to an AI-induced \u201cloneliness spiral\u201d. When individuals feel lonely, they\u2019re more sensitive<\/a> to feelings of social threat and disruptive emotional regulation.<\/p>\n

So they turn to a low-risk AI \u201cfriend\u201d for support, instead of qualified or friendly humans. However, this often doesn\u2019t fix the initial problem of loneliness, since chatbots can actually exacerbate it.<\/p>\n

The restrictions China has put in place for AI companions can help to break this cycle, and re-prioritise human connection. It\u2019s re-balancing the cognitive math: would it be more satisfying to install a companion AI app, or make time to meet with my friend?<\/p>\n

Lessons for Australia<\/p>\n

Loneliness and a low sense of belonging remain major challenges in Australia, with at least one in six experiencing loneliness<\/a>, with some evidence<\/a> suggesting higher rates.<\/p>\n

While China has cracked down on AI companions, Australia has been leading the charge on young people\u2019s use of social media. eSafety has also developed codes for restricting age-related material online<\/a> such as pornography, high-impact violence, self-harm, suicide, and disordered eating. But they don\u2019t address social harms of AI companions, such as emotional dependency or displacing human connection.<\/p>\n

Prime Minister Anthony Albanese also didn\u2019t mention<\/a> social harms from AI chatbots in his major speech on AI last week.<\/p>\n

We propose that Australia\u2019s age-restricted material protections should extend to unintended social harm of conversations with social, analytical and functional AIs.<\/p>\n

The Australian government needs to explore how these virtual companions interact with Australians. And we must prepare to mitigate unintended social harms before these synthetic social connections become widespread and normalised.<\/p>\n

Australia must prioritise developing social connectedness \u2013 in workplaces, productivity strategies, education curriculum, and in the design of our communities<\/a>.<\/p>\n

An authoritarian crackdown on chatbots may not be within reach for a democratic nation. But we can \u2013 and should \u2013 prioritise developing human connection skills to resist unhealthy overreliance unhealthy overreliance on AI.<\/p>\n

And Australia should leverage regulatory protections to mitigate social harms, extending on the work eSafety is already doing with social media.<\/p>\n","protected":false},"excerpt":{"rendered":"Since November 2022, generative artificial intelligence (AI) has been rewriting billions of emails, student assignments, reports, and break-up…\n","protected":false},"author":2,"featured_media":555191,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[345,343,344,85,46,125],"class_list":["post-555190","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-il","tag-israel","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts\/555190","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/comments?post=555190"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts\/555190\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/media\/555191"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/media?parent=555190"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/categories?post=555190"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/tags?post=555190"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}