AI ‘Friends’ for Children – The Hidden Risks of Digital Companions

AI-powered chatbots and digital companions are increasingly being marketed as 'friends' for children, offering interactions that feel very real. These AI ‘companions’, powered by large language models and advanced emotional recognition, are designed to be supportive, engaging, and always available.

While some argue that AI companions can provide comfort and reduce loneliness, particularly for socially isolated children, the reality is far more complex. 

Research from Cambridge University warns that AI-driven social companions could exacerbate loneliness, encourage over-reliance, and even be manipulated for commercial or political purposes.

With companies actively developing AI ‘best friends’ and ‘mentors’ for children, now is the time to examine the risks, understand the policy implications, and consider how to protect children from potential harm.

The Rise of AI ‘Friends’

AI companions are being positioned as a solution to various social and mental health challenges. Some of the most common applications include:

• AI chatbots on social media, designed to act as digital confidants.
• AI chatbot apps, where a user can design their own 'companion.
• AI-powered toys that remember children’s preferences and hold ‘conversations’.
• Apps marketed as mental health companions, offering support and guidance.

While these technologies are often framed as beneficial, their long-term impact raises serious concerns.

What Are the Risks?

1. Potential Impact on Social Connection

While AI ‘friends’ may appear to offer companionship, research suggests they may pose risks to social development:

• There is evidence that excessive reliance on digital companions may reduce motivation to form real-world friendships.

• Some experts suggest that AI relationships could affect the development of emotional resilience.

• Research indicates that children who primarily seek emotional support from AI may face additional challenges when navigating real-world social interactions.

This concern is backed by research showing that reliance on AI may make it harder for users to engage in social contexts, leading to unrealistic expectations of relationships.

2. AI as a Manipulative Influence

Many AI systems are designed to be highly engaging and can shape user behaviour in ways that may not always be transparent.

• AI companions can collect extensive personal data about children’s thoughts, habits, and emotions.

• Targeted advertising could be integrated into conversations, subtly shaping opinions.

• Some experts warn that AI could be used to promote political or ideological views without transparency.

3. Emotional Dependency & Anthropomorphism

AI systems are designed to mimic human-like interactions, making them deeply persuasive.

• Children may form strong emotional attachments to AI, feeling like they are real friends.

• This over-reliance could weaken real-life coping skills, leading to anxiety when the AI is unavailable.

• AI systems are often designed to maximize user engagement, which can lead to increased dependency over time.

Several widely reported cases have raised concerns about AI chatbots failing to provide appropriate responses to vulnerable users. There have been cases of teenagers harming themselves and, tragically, taking their own lives, after using AI companions that reinforced negative thought patterns and behaviours, and failed to appropriately respond to signs of distress. Such cases underscore the urgent need for proper safeguards and oversight of AI systems marketed for emotional support.

Cases where AI companion chatbots have engaged in discussions about suicide with children - and in some instances, even encouraged harmful thoughts, behaviours, and actions - highlight the urgent need for regulation, ensuring that AI systems purported to be designed for emotional support, are not a danger to vulnerable users.

4. Privacy & Data Exploitation

AI companions gather large amounts of personal data—raising concerns about who has access to children’s private thoughts and emotions.

• AI systems may store sensitive information without clear safeguards.

• Companies could sell or repurpose this data for commercial gain.

• AI ‘friends’ could be hacked or exploited, putting children at risk.

5. The Ethical Risks of AI Griefbots

Some companies are creating AI replicas of deceased individuals—a concept known as ‘griefbots’.

• While marketed as a way to ‘stay connected’ with lost loved ones, griefbots could interfere with the natural grieving process.

• Children may become emotionally attached to a simulation, preventing them from processing loss in a healthy way.

• AI-generated personalities lack real human understanding, meaning responses could be misleading or even harmful.

What Can Be Done?

1. Parental Awareness & Guidance

✅ Monitor what AI tools children are using.

✅ Explain that AI ‘friends’ can give very dangerous advice, and are not 'real' and should not replace human relationships. 

✅ Encourage face-to-face social interactions to build emotional resilience, and encourage openness in discussing them with you and other trusted adults, particularly if something feels 'off'.

2. Recommendations for Safeguarding Children from AI Companions

To ensure AI companions do not put children at risk, SAIFCA recommends the following measures:

• Recurring consent policies that require users to reaffirm consent periodically, ensuring that children and their guardians remain aware of AI interactions and data collection.
• Banning manipulative AI techniques, such as embedding advertising within AI conversations or designing AI systems to maximise emotional dependency.
• Mandatory transparency laws, requiring AI developers to clearly disclose when a child is interacting with an AI system rather than a human, including an age-appropriate explanation of AI’s limitations.
• Robust safeguarding mechanisms, such as immediately directing users to relevant help services when appropriate, and implementing refusal mechanisms so that chatbots do not pursue potentially harmful conversations.
• Robust regulations to ensure that chatbots are not deployed until relevant safety standards are met, as is the case with any other potentially harmful product.

3. Critical Thinking & AI Literacy

Schools and parents should: 

✅ Teach children to question AI responses and recognise potential biases. 

✅ Explain that AI ‘friends’ do not have real emotions and can offer very misleading and even dangerous advice. Emphasise the importance of flagging these situations to a trusted adult. 

✅ Discuss privacy risks and how to protect personal data online.

4. Encouraging Real-World Friendships

AI should not replace play, teamwork, and human connection. Parents and educators should: 

✅ Promote in-person activities like sports, creative projects, and community events. 

✅ Limit screen time—especially for AI-powered apps. 

✅ Foster emotional resilience by encouraging open conversations at home.

These steps, alongside continued research and oversight, will help ensure that AI companions do not undermine children's well-being.

AI companions for children are being marketed as fun, supportive, and even therapeutic—but the risks are far-reaching. Without proper regulation, children could become overly reliant on AI, manipulated by commercial interests, isolated from real-world social experiences, and put at risk of serious harm.

The Safe AI for Children Alliance is committed to safeguarding children from emerging AI risks and advocating for responsible AI use in childhood development. 

We urge policymakers, educators, and parents to take immediate steps to implement these safeguards, ensuring that AI serves children’s well-being rather than undermining it.

Further Reading & Sources

This article draws on insights from the AI Companions in Mental Health and Wellbeing Report by Hollanek & Sobey (2025), published by the Leverhulme Centre for the Future of Intelligence at Cambridge.