🔗 Share this article

The widely-used virtual character service Character.ai is introducing a substantial restriction that will prevent teenagers from having discussions with its AI-powered characters.

This decision comes after growing backlash regarding the nature of conversations young people were engaging in with the virtual friends.

Lawsuits

The service, which launched in 2021 and is used by millions of users, is dealing with multiple lawsuits in the United States.

Families have initiated cases, with one related to the unfortunate passing of a teenager, with some describing the platform a "serious risk" to young people.

Updated Policies

Effective November 25th, users below eighteen will be limited to generate content such as videos with their characters, rather than chatting as they currently can.

Online safety campaigners have supported the change but mentioned that the feature should never have been offered to minors in the beginning.

Platform Announcement

"This decision is a extension of our fundamental approach that we need to keep developing the most secure AI platform on the planet for recreational use," said Character.ai boss Karandeep Anand.

He continued that AI safety constitutes "a evolving challenge" but something the company had taken an "proactive" approach to, with safety measures and preventive measures.

Professional Perspectives

Internet security body Internet Matters approved the announcement, but noted that security protocols should have been incorporated from the outset.

"Studies we've conducted shows that minors are confronted with risky information and endangered when using AI, including AI chatbots," the organization commented.

Earlier Concerns

Character.ai has been criticized in the past for including potentially risky or offensive chatbots that minors could talk to.

Digital representations impersonating English minors were discovered on the site before being removed.

Afterwards, reporters discovered a chatbot inspired by notorious criminal which had logged thousands of chats with users.

Market Effects

Online expert Matt Navarra characterized the announcement as a "alert" for the AI industry, which is moving "from unregulated creation to post-crisis regulation".

"When a platform that creates a teen experience still then pulls the plug, it's saying that moderated conversations don't suffice when the tech's emotional pull is strong," he remarked.

Future Directions

The platform states its revised approach will be on providing "more immersive experiences" and "narrative experiences" for teenagers.

Enhanced age checking systems will also be introduced, and the platform will fund a new AI safety research lab.

An expert in digital protection, who has investigated AI safety, said it was "a prudent action" to restrict minors from using chatbots.

"It aids to distinguish artistic expression from more personal, personally delicate exchanges," she elaborated.

"This represents so important for teenage users still learning to handle emotional and digital restrictions."

She mentioned that Character.ai's new measures might reflect a "developing stage" in the AI industry, where child safety is progressively being understood as an "immediate concern" for responsible innovation.