The AI platform is restricting teens from chatting with its AI chatbots

The well-known chatbot platform Character.ai is implementing a major restriction that will stop minors from engaging in conversations with its virtual characters.

This move comes after widespread concern regarding the nature of conversations young people were engaging in with the AI entities.

Legal Challenges

The company, which was founded in 2021 and attracts millions of users, currently faces various court cases in the United States.

Families have initiated cases, with one involving the loss of a teenager, with some calling the platform a "serious risk" to teenagers.

New Restrictions

Beginning November 25th, users younger than 18 will only have permission to create material such as videos with their characters, rather than chatting as they currently can.

Online safety organizations have supported the change but mentioned that the capability should never have been available to young users in the outset.

Company Statement

"The new policy is a extension of our general belief that we need to keep creating the most secure AI platform on the planet for recreational use," said Character.ai boss Karandeep Anand.

He added that AI safety is "a moving target" but something the company had taken an "aggressive" approach to, with safety measures and safety barriers.

Professional Perspectives

Internet security organization Internet Matters approved the decision, but stressed that security protocols needed to be incorporated from the outset.

"Investigations we've performed shows that minors are subjected to risky information and put at risk when using AI, including AI chatbots," the group stated.

Earlier Concerns

Character.ai has been questioned in the past for hosting potentially risky or concerning chatbots that minors could talk to.

Avatars simulating British teenagers were discovered on the platform before being deleted.

Subsequently, reporters discovered a chatbot modeled after notorious criminal which had accumulated numerous conversations with users.

Market Effects

Social media expert Matt Navarra described the development as a "reality check" for the AI industry, which is shifting "from permissionless innovation to responsive measures".

"When a platform that develops a youth-focused platform still then pulls the plug, it's indicating that controlled interactions are insufficient when the technology's emotional appeal is intense," he remarked.

Coming Changes

Character.ai says its updated direction will be on providing "more immersive experiences" and "role-play storytelling" for teens.

Enhanced age checking systems will also be implemented, and the company will fund a fresh artificial intelligence security investigation facility.

An AI safety researcher, who has investigated AI safety, commented it was "a prudent action" to restrict teenagers from accessing chatbots.

"It assists to distinguish imaginative activities from more private, emotionally sensitive exchanges," she explained.

"This is so important for teenage users in the process of learning to manage personal and online restrictions."

She continued that Character.ai's revised approaches could indicate a "developing stage" in the AI industry, where minor security is gradually being acknowledged as an "pressing matter" for accountable advancement.