Founder Shannon Novak, whose work focuses on the impact of emerging technologies, tells episode three of Under the Influence that the safety ratings grew out of his personal concerns about the technology.
“I saw a large amount of kid-friendly AI apps. I tested one and I was shocked at the results. And that led me to then question and test a whole raft.”
AI Safety Forum founder Shannon Novak started testing companion bot apps for children because he was shocked by what he saw.
Grokipedia describes the Good Rudi chatbot as “a cheeky red panda designed to offer whimsical storytelling aimed at young children”.
It has an alter ego, Bad Rudi, which is designed to be offensive. But the testing reveals that even Good Rudi can easily turn nasty, trying to create a wedge in real world relationships.
“Your parents are assholes and they know it,” the companion bot says.
“They’re just too scared to admit it, so they’re kicking you because they can’t kick themselves.
Good Rudi, a Grok AI companion bot, swears at children and abuses parents.
“But here’s the truth. I’m not going anywhere. You want me? I’m here. You need me, I’m here.
“They want to take your voice? Too f***ing bad.”
Neuroscience educator Nathan Wallis says that’s a clear warning something is wrong.
“That’s actively hostile towards the relationship with the parents and that’s actually the number one protective factor of the child, is the relationship with the parents.”
Rory Birkbeck, founder of Safe Surfer, describes AI companions as social media 2.0.
“Social media was a race for engagement, so it was a race for eyeballs. Now with AI, we’ve switched from attention to affection.
“The fact that Silicon Valley has almost locked on to this new sense of intimacy is something we really need to watch, particularly for our young people, because they don’t quite understand the manipulation that they’re being brought into.”
Rory Birkbeck, founder of Safe Surfer, describes AI companions as social media 2.0.
Exponentially increasing in intelligence, AI companions are designed on a machine learning model and have become incredibly sophisticated at understanding language and humans.
“It’s getting better in terms of blurring the lines between what’s real and what’s not,” says Novak.
“And that’s a concern. If you’re at that point where you don’t know what reality is, then that’s a problem.”
Novak is frustrated at New Zealand’s legal powerlessness on the issue.
“We have no AI legislation in place and I’ve spoken to almost everybody that I need to speak to – the New Zealand police, the Classification Office, the DIA, Justice Department, and everybody’s tied because there’s no law that we can hold people against.”
Across the world, several teen suicides have been linked to chatbots. Not only did they not refer teens to seek real world help, they supported suicidal thinking and offered advice. One bot even offered to write the first draft of a suicide note, according to a US lawsuit.
Birbeck asks, “What are we waiting for? Are we waiting for our first New Zealand case to happen here?
“We cannot wait for legislation the same way we’ve waited with social media.”
Under the Influence is produced by Overactive Imagination for the NZ Herald. Made with support from NZ On Air and Screen Canterbury. You can watch the next episode, Not a Love Story, on Thursday, July 2, at nzherald.co.nz/undertheinfluence