Study uncovers child sexual abuse images hidden inside a depository used to train AI generators

A study conducted by the Stanford Internet Observatory has revealed a disturbing finding of thousands of child sexual abuse images hidden within the dataset used to train artificial intelligence (AI) image generators. The images were discovered within the LAION AI database, and researchers found over 3,200 suspected child sexual abuse images, with more than 1,000 confirmed as child sexual abuse material.

The analysis was conducted in collaboration with the Canadian Centre for Child Protection and other anti-abuse charities. The LAION-5B dataset, which contained the illegal images, is used to train various AI image generators, including Stable Diffusion. Popular AI maker OpenAI, responsible for DALL-E and ChatGPT, denied the use of the LAION database and stated that its models have been fine-tuned to refuse requests involving sexual content with minors.

Google’s text-to-image model, Imagen, was also built using the LAION dataset, but the company decided against making it public in 2022 after an audit revealed inappropriate content, including pornographic imagery and harmful stereotypes.

In response to the findings, LAION, a non-profit organization, temporarily removed its datasets, emphasizing a zero-tolerance policy for illegal content. The organization stated that it took down the LAION datasets “in an abundance of caution” to ensure they are safe before republishing them.

While the 3,200 images represent a fraction of the 5.8 billion images in the dataset, researchers at Stanford believe their presence may influence AI generators to produce harmful outputs, potentially creating realistic, explicit imagery. The presence of such images in the dataset could enable the transformation of social media photos of clothed individuals, including teenagers, into nude images.

The Stanford Internet Observatory has recommended drastic measures, urging users of LAION-5B datasets to delete or cleanse the material. Additionally, they recommend making older versions, like Stable Diffusion, less accessible, particularly if used to generate abusive images without adequate safeguards.