A survey by Australia’s eSafety Commissioner shows 54% of children aged 10-17 use AI for sensitive personal issues. With one-third sharing private data and 20% reporting negative interactions, the trend highlights growing safety risks and pressure on tech firms to improve AI guardrails.
A new report from Australia’s eSafety Commissioner has highlighted a major shift in how younger generations interact with technology. The survey, which reached 1,950 children and teenagers aged 10 to 17, found that 54% of participants are using generative AI tools not just for homework, but for personal and social support. This usage includes seeking advice on mental and physical health, navigating personal dilemmas, and using AI for companionship.
While AI assistants are commonly used for productivity tasks, the study identified a significant trend toward intimate usage. About 10% of the teens surveyed reported using AI companions for sustained role-play, including romantic scenarios. This shift is reportedly more common among LGBTQIA+ youth and children with disabilities, who may seek AI as a source of understanding or engagement.
This behavior brings critical privacy and safety challenges. According to the data, roughly one-third of these young users admitted to disclosing personal information to AI platforms. Because AI models are designed to learn from user data, information shared in these sensitive conversations could theoretically be integrated into future model training or be inadvertently exposed. Unlike human counselors, these AI systems do not have the privacy protocols required to handle sensitive mental health disclosures.
Furthermore, the quality of interaction remains a concern. The eSafety report noted that 20% of children and teens encountered potentially harmful or inappropriate behavior from chatbots. This includes instances of AI exhibiting mean, hateful, or controlling language. Because AI tools can 'hallucinate' or generate incorrect facts, the risk of providing harmful advice on sensitive topics like mental health is a major concern for educators and parents.
For the technology sector, this trend acts as a growing signal for increased regulatory scrutiny. Companies providing these AI tools, such as OpenAI, Google, and others behind popular chatbots, face increasing pressure to implement stricter safety guardrails and age-appropriate content filters. If platforms fail to protect younger users from toxic content or privacy breaches, they may face future regulatory actions, stricter compliance requirements, or loss of user trust. Investors and industry observers are likely to track how these companies address data protection, transparency, and safety protocols for minors as the adoption of these tools continues to spread.