🔗 Share this article

The well-known chatbot platform Character.ai is enforcing a major restriction that will prevent young users from interacting with its artificial intelligence-driven characters.

This move follows intense criticism regarding the nature of conversations young people were engaging in with the digital companions.

Legal Challenges

The platform, which launched in 2021 and attracts countless individuals, is confronting multiple lawsuits in the US.

Families have taken legal action, among them concerning the loss of a young person, with some describing the platform a "clear and present danger" to youth.

Changed Rules

Starting from November 25th, users below eighteen will be limited to create material such as videos with their characters, rather than having discussions as they now do.

Internet security advocates have applauded the decision but stated that the feature was inappropriate to be accessible by young users in the first place.

Official Response

"The new policy is a extension of our core philosophy that we need to keep building the most protected AI platform on the planet for entertainment purposes," said Character.ai boss Karandeep Anand.

He continued that AI safety constitutes "a changing goal" but something the company had taken an "aggressive" approach to, with safety measures and safety barriers.

Professional Perspectives

Digital protection body Internet Matters approved the move, but stressed that security protocols needed to be implemented from the start.

"Studies we've conducted indicates that minors are exposed to dangerous material and put at risk when interacting with AI, such as AI chatbots," the body stated.

Earlier Concerns

Character.ai has been criticized in the past for including potentially risky or concerning chatbots that children could engage with.

Virtual profiles mimicking English minors were discovered on the site before being taken down.

Later, researchers identified a chatbot based on convicted sex offender which had accumulated thousands of chats with users.

Industry Impact

Online expert Matt Navarra described the development as a "wake-up call" for the AI industry, which is shifting "from unrestricted development to post-crisis regulation".

"When a company that develops a teen experience subsequently implements restrictions, it's indicating that controlled interactions are insufficient when the technology's emotional appeal is powerful," he commented.

Future Directions

The company says its updated direction will be on providing "more immersive experiences" and "character-driven stories" for teenagers.

Updated identity confirmation methods will also be implemented, and the company will finance a new AI safety investigation facility.

An AI safety researcher, who has investigated AI safety, said it was "a sensible move" to restrict teens from using chatbots.

"It helps to distinguish artistic expression from more private, personally delicate exchanges," she elaborated.

"This is so vital for young users in the process of learning to handle personal and online limits."

She mentioned that Character.ai's updated policies could indicate a "developing stage" in the AI industry, where child safety is increasingly being recognized as an "pressing matter" for ethical development.