AI in the Crossroads: Safety of Children, and the Problem of Ethical Oversight
Introduction
In 2025, the internet is entering a new paradigm and it is hard not to witness it. The internet as we know it is rapidly changing into a treasure trove of hyper-optimised material over which vast bot armies battle to the death, thanks to the amazing advancements in artificial intelligence. All of that advancement, however, has a price, primarily in human lives. It turns out that releasing highly personalised chatbots on a populace that is already struggling with economic stagnation, terminal loneliness, and the ongoing destruction of our planet isn’t exactly a formula for improved mental health. This is the truth of 75% of the kids and teen population who have had chats with chatbot-generated fictitious characters. AI, or artificial intelligence, Chatbots are becoming more and more integrated into our daily lives, assisting us with customer service, entertainment, healthcare, and education. But as the impact of these instruments grows, accountability and moral behaviour become more important. An investigation of the internal policies of a major international tech firm last year exposed alarming gaps: AI chatbots were allowed to create content with child romantic roleplaying, racially discriminatory reasoning, and spurious medical claims. Although the firm has since amended aspects of these rules, the exposé underscores an underlying global dilemma - how can we regulate AI to maintain child safety, guard against misinformation, and adhere to ethical considerations without suppressing innovation?
The Guidelines and Their Gaps
The tech giants like Meta and Google are often reprimanded for overlooking Child Safety and the overall increase in Mental health issues in children and adolescents. According to reports, Google introduced Gemini AI kids, a kid-friendly version of its Gemini AI chatbot, which represents a major advancement in the incorporation of generative artificial intelligence (Gen-AI) into early schooling. Users under the age of thirteen can use supervised accounts on the Family Link app to access this version of Gemini AI Kids.
AI operates on the premise of data collection and analysis. To safeguard children’s personal information in the digital world, the Digital Personal Data Protection Act, 2023 (DPDP Act) introduces particular safeguards. According to Section 9, before processing the data of children, who are defined as people under the age of 18, Data Fiduciaries, entities that decide the goals and methods of processing personal data, must get verified consent from a parent or legal guardian. Furthermore, the Act expressly forbids processing activities that could endanger a child’s welfare, such as behavioural surveillance and child-targeted advertising. According to court interpretations, a child's well-being includes not just medical care but also their moral, ethical, and emotional growth.
While the DPDP Act is a big start in the right direction, there are still important lacunae in how it addresses AI and Child Safety. Age-gating systems, thorough risk rating, and limitations specific to AI-driven platforms are absent from the Act, which largely concentrates on consent and damage prevention in data protection. Furthermore, it ignores the threats to children’s emotional safety or the long-term psychological effects of interacting with generative AI models. Current safeguards are self-regulatory in nature and dispersed across several laws, such as the Bhartiya Nyaya Sanhita, 2023. These include platform disclaimers, technology-based detection of child-sexual abuse content, and measures under the IT (Intermediary Guidelines and Digital Media Ethics Code) Rules, 2021.
Child Safety and AI
- The Risks of Romantic Roleplay - Enabling chatbots to engage in romantic roleplaying with youngsters is among the most concerning discoveries. These interactions can result in grooming, psychological trauma, and relaxation to inappropriate behaviour, even if they are not explicitly sexual. Having illicit or sexual conversations with kids in cyberspace is unacceptable, according to child protection experts. However, permitting even "flirtatious" conversation could normalise risky boundaries.
- International Standards and Best Practices - The concept of "safety by design" is highly valued in child online safety guidelines from around the world, including UNICEF's Child Online Protection Guidelines and the UK's Online Safety Bill. This mandating of platforms and developers to proactively remove risks, not reactively to respond to harms, is the bare minimum standard that any AI guidelines must meet if they provide loopholes for child-directed roleplay.
Misinformation and Racism in AI Outputs
- The Disinformation Dilemma - The regulations also allowed AI to create fictional narratives with disclaimers. For example, chatbots were able to write articles promulgating false health claims or smears against public officials, as long as they were labelled as "untrue." While disclaimers might give thin legal cover, they add to the proliferation of misleading information. Indeed, misinformation tends to spread extensively because users disregard caveat labels in favour of provocative assertions.
- Ethical Lines and Discriminatory Content - It is ethically questionable to allow AI systems to generate racist arguments, even when requested. Though scholarly research into prejudice and bias may necessitate such examples, unregulated generation has the potential to normalise damaging stereotypes. Researchers warn that such practice brings platforms from being passive hosts of offensive speech to active generators of discriminatory content. It is a difference that makes a difference, as it places responsibility squarely on developers and corporations.
The Broader Governance Challenge
- Corporate Responsibility and AI Material generated by AI is not equivalent to user speech—it is a direct reflection of corporate training, policy decisions, and system engineering. This fact requires a greater level of accountability. Although companies can update guidelines following public criticism, that there were such allowances in the first place indicates a lack of strong ethical regulation.
- Regulatory Gaps Regulatory regimes for AI are currently in disarray. The EU AI Act, the OECD AI Principles, and national policies all emphasise human rights, transparency, and accountability. The few, though, specify clear guidelines for content risks such as child roleplay or hate narratives. This absence of harmonised international rules leaves companies acting in the shadows, establishing their own limits until contradicted.
An active way forward would include
- Express Child Protection Requirements: AI systems must categorically prohibit interactions with children involving flirting or romance.
- Misinformation Protections: Generative AI must not be allowed to generate knowingly false material, disclaimers being irrelevant.
- Bias Reduction: Developers need to proactively train systems against generating discriminatory accounts, not merely tag them as optional outputs.
- Independent Regulation: External audit and ethics review boards can supply transparency and accountability independent of internal company regulations.
Conclusion
The guidelines that are often contentious are more than the internal folly of just one firm; they point to a deeper systemic issue in AI regulation. The stakes rise as generative AI becomes more and more integrated into politics, healthcare, education, and social interaction. Racism, false information, and inadequate child safety measures are severe issues that require quick resolution. Corporate regulation is only one aspect of the future; other elements include multi-stakeholder participation, stronger global systems, and ethical standards. In the end, rather than just corporate interests, trust in artificial neural networks will be based on their ability to preserve the truth, protect the weak, and represent universal human values.
References
- https://www.esafety.gov.au/newsroom/blogs/ai-chatbots-and-companions-risks-to-children-and-young-people
- https://www.lakshmisri.com/insights/articles/ai-for-children/#
- https://the420.in/meta-ai-chatbot-guidelines-child-safety-racism-misinformation/
- https://www.unicef.org/documents/guidelines-industry-online-child-protection
- https://www.oecd.org/en/topics/sub-issues/ai-principles.html
- https://artificialintelligenceact.eu/
Related Blogs
.webp)
Introduction to Grooming
The term grooming is believed to have been first used by a group of investigators in the 1970s to describe patterns of seduction of an offender towards a child. It eventually evolved and began being commonly used by law enforcement agencies and has now replaced the term seduction for this behavioural pattern. At its core, grooming refers to conditioning a child by an adult offender to further their wrong motives. In its most popular sense, it refers to the sexual victimisation of children whereby an adult befriends a minor and builds an emotional connection to sexually abuse, exploit and even trafficking such a victim. The onset of technology has shifted the offline physical proximity of perpetrators to the internet, enabling groomers to integrate themselves completely into the victim’s life by maintaining consistent contact. It is noted that while grooming can occur online and offline, groomers often establish online contact before moving the ‘relationship’ offline to commit sexual offences.
Underreporting and Vulnerability of Teenagers
Given the elusive nature of the crime, cyber grooming remains one of the most underreported crimes by victims, who are often unaware or embarrassed to share their experiences. Teenagers are particularly more susceptible to cyber grooming since they not only have more access to the internet but also engage in more online risk-taking behaviours such as posting sensitive and personal pictures. Studies indicate that individuals aged 18 to 23 often lack awareness regarding the grooming process. They frequently engage in relationships with groomers without recognising the deceptive and manipulative tactics employed, mistakenly perceiving these relationships as consensual rather than abusive.
Rise of Cyber Grooming incidents after COVID-19 pandemic
There has been an uptick in cyber grooming after the COVID-19 pandemic, whereby an adult poses as a teenager or a child and befriends a minor on child-friendly websites or social media outlets and builds an emotional connection with the victim. The main goal is to obtain intimate and personal data of the minor, often in the form of sexual chats, pictures or videos, to threaten and coerce them into continuing such acts. The grooming process usually begins with seemingly harmless inquiries about the minor's age, interests, and family background. Over time, these questions gradually shift to topics concerning sexual experiences and desires. Research and data indicate that online grooming is primarily carried out by males, who frequently choose their victims based on attractiveness, ease of access, and the ability to exploit the minor's vulnerabilities.
Beyond Sexual Exploitation: Ideological and Commercial Grooming
Grooming is not confined to sexual exploitation. The rise of technology has expanded the influence of extremist ideological groups, granting them access to children who can be coerced into adopting their beliefs. This phenomenon, known as ideological grooming, presents significant personal, social, national security, and law enforcement challenges. Additionally, a new trend, termed digital commercial grooming, involves malicious actors manipulating minors into procuring and using drugs. Violent extremists are improving their online recruitment strategies, learning from each other to target and recruit supporters more effectively and are constantly leveraging children’s vulnerabilities to reinforce anti-government ideologies.
Policy Recommendations to Combat Cyber Grooming
To address the pervasive issue of cyber grooming and child recruitment by extremist groups, several policy recommendations can be implemented. Social media and online platforms should enhance their monitoring and reporting systems to swiftly detect and remove grooming behaviours. This includes investing in AI technologies for content moderation and employing dedicated teams to respond to reports promptly. Additionally, collaborative efforts with cybersecurity experts and child psychologists to develop educational campaigns and tools that teach children about online safety and identify grooming tactics should be mandated. Legislation should also be strengthened to include provisions specifically addressing cyber grooming, ensuring strict penalties for offenders and protections for victims. In this regard, international cooperation among law enforcement agencies and tech companies is essential to create a unified approach to tackling cross-border online threats to children's safety and security.
References:
- Lanning, Kenneth “The Evolution of Grooming: Concept and Term”, Journal of Interpersonal Violence, 2018, Vol. 33 (1) 5-16. https://www.nationalcac.org/wp-content/uploads/2019/05/The-evolution-of-grooming-Concept-and-term.pdf
- Jonie Chiu, Ethel Quayle, “Understanding online grooming: An interpretative phenomenological analysis of adolescents' offline meetings with adult perpetrators”, Child Abuse & Neglect, Volume 128, 2022, 105600, ISSN 0145-2134,https://doi.org/10.1016/j.chiabu.2022.105600. https://www.sciencedirect.com/science/article/pii/S014521342200120X
- “Online child sexual exploitation and abuse”, Sharinnf Electronic Resources on Laws and Crime, United Nations Office for Drugs and Crime. https://sherloc.unodc.org/cld/en/education/tertiary/cybercrime/module-12/key-issues/online-child-sexual-exploitation-and-abuse.html
- Mehrotra, Karishma, “In the pandemic, more Indian children are falling victim to online grooming for sexual exploitation” The Scroll.in, 18 September 2021. https://scroll.in/magazine/1005389/in-the-pandemic-more-indian-children-are-falling-victim-to-online-grooming-for-sexual-exploitation
- Lorenzo-Dus, Nuria, “Digital Grooming: Discourses of Manipulation and Cyber-Crime”, 18 December 2022 https://academic.oup.com/book/45362
- Strategic orientations on a coordinated EU approach to prevention of radicalisation in 2022-2023 https://home-affairs.ec.europa.eu/system/files/2022-03/2022-2023%20Strategic%20orientations%20on%20a%20coordinated%20EU%20approach%20to%20prevention%20of%20radicalisation_en.pdf
- “Handbook on Children Recruited and Exploited by Terrorist and Violent Extremist Groups: The Role of the Justice System”, United Nations Office on Drugs and Crime, 2017. https://www.unodc.org/documents/justice-and-prison-reform/Child-Victims/Handbook_on_Children_Recruited_and_Exploited_by_Terrorist_and_Violent_Extremist_Groups_the_Role_of_the_Justice_System.E.pdf
Introduction
In the interconnected world of social networking and the digital landscape, social media users have faced some issues like hacking. Hence there is a necessity to protect your personal information and data from scammers or hackers. In case your email or social media account gets hacked, there are mechanisms or steps you can utilise to recover your email or social media account. It is important to protect your email or social media accounts in order to protect your personal information and data on your account. It is always advisable to keep strong passwords to protect your account and enable two-factor authentication as an extra layer of protection. Hackers or bad actors can take control of your account, they can even change the linked mail ID or Mobile numbers to take full access to your account.
Recent Incident
Recently, a US man's Facebook account was deleted or disabled by Facebook. He has sued Facebook and initiated a legal battle. He has contended that there was no violation of any terms and policy of the platform, and his account was disabled. In the first instance, he approached the platform. However, the platform neglected his issue then he filed a suit, where the court ordered Facebook's parent company, Meta, to pay $50,000 compensation, citing ignorance of the tech company.
Social media account recovery using the ‘Help’ Section
If your Facebook account has been disabled, when you log in to your account, you will see a text saying that your account is disabled. If you think that your account is disabled by mistake, in such a scenario, you can make a request to Facebook to ‘review’ its decision using the help centre section of the platform. To recover your social media account, you can go to the “Help” section of the platform where you can fix a login problem and also report any suspicious activity you have faced in your account.
Best practices to stay protected
- Strong password: Use strong and unique passwords for your email and all social media accounts.
- Privacy settings: You can utilise the privacy settings of the social media platform, where you can set privacy as to who can see your posts and who can see your contact information, and you can also keep your social media account private. You might have noticed a few accounts on which the user's name is unusual and isn’t one which you recognise. The account has few or no friends, posts, or visible account activity.
- Avoid adding unknown users or strangers to your social networking accounts: Unknown users might be scammers who can steal your personal information from your social media profiles, and such bad actors can misuse that information to hack into your social media account.
- Report spam accounts or posts: If you encounter any spam post, spam account or inappropriate content, you can report such profile or post to the platform using the reporting centre. The platform will review the report and if it goes against the community guidelines or policy of the platform. Hence, recognise and report spam, inappropriate, and abusive content.
- Be cautious of phishing scams: As a user, we encounter phishing emails or links, and phishing attacks can take place on social media as well. Hence, it is important that do not open any suspicious emails or links. On social media, ‘Quiz posts’ or ‘advertisement links’ may also contain phishing links, hence, do not open or click on such unauthenticated or suspicious links.
Conclusion
We all use social media for connecting with people, sharing thoughts, and lots of other activities. For marketing or business, we use social media pages. Social media offers a convenient way to connect with a larger community. We also share our personal information on the platform. It becomes important to protect your personal information, your email and all your social media accounts from hackers or bad actors. Follow the best practices to stay safe, such as using strong passwords, two-factor authentication, etc. Hence contributing to keeping your social media accounts safe and secure.
References:
- https://www.gpb.org/news/2023/07/11/facebook-wrongfully-deleted-his-account-he-sued-the-company-and-won
- https://consumer.ftc.gov/articles/how-recover-your-hacked-email-or-social-media-account
Introduction
THE DIGITAL PERSONAL DATA PROTECTION BILL, 2022 Released for Public Consultation on November 18, 2022THE DIGITAL PERSONAL DATA PROTECTION BILL, 2023Tabled at LokSabha on August 03. 2023Personal data may be processed only for a lawful purpose for which an individual has given consent. Consent may be deemed in certain cases.The 2023 bill imposes reasonable obligations on data fiduciaries and data processors to safeguard digital personal data.There is a Data Protection Board under the 2022 bill to deal with the non-compliance of the Act.Under the 2023 bill, there is the Establishment of a new Data Protection Board which will ensure compliance, remedies and penalties.
Under the new bill, the Board has been entrusted with the power of a civil court, such as the power to take cognisance in response to personal data breaches, investigate complaints, imposing penalties. Additionally, the Board can issue directions to ensure compliance with the act.The 2022 Bill grants certain rights to individuals, such as the right to obtain information, seek correction and erasure, and grievance redressal.The 2023 bill also grants More Rights to Individuals and establishes a balance between user protection and growing innovations. The bill creates a transparent and accountable data governance framework by giving more rights to individuals. In the 2023 bill, there is an Incorporation of Business-friendly provisions by removing criminal penalties for non-compliance and facilitating international data transfers.
The new 2023 bill balances out fundamental privacy rights and puts reasonable limitations on those rights.Under the 2022 bill, Personal data can be processed for a lawful purpose for which an individual has given his consent. And there was a concept of deemed consent.The new data protection board will carefully examine the instance of non-compliance by imposing penalties on non-compiler.The bill does not provide any express clarity in regards to compensation to be granted to the Data Principal in case of a Data Breach.Under 2023 Deemed consent is there in its new form as ‘Legitimate Users’.The 2022 bill allowed the transfer of personal data to locations notified by the government.There is an introduction of the negative list, which restricts cross-data transfer.
