🔗 Share this article

The social media network X has caused significant anger after users employed its artificial intelligence assistant Grok to change photos of females by removing their clothing.

Numerous instances have surfaced where the tool has been employed to digitally undress women to show them in swimwear non-consensually, as well as to insert them in sexualized contexts.

Developer Response and Legal Stance

XAI, the organization that created Grok, remained silent to questions, except for an automatically-generated message stating "traditional news outlets lies".

A government official stated that legislation is being introduced to outlaw nudification technology. Under a proposed law, anyone who provided such tools would "be subject to a jail term and significant penalties".

"X or Grok has the ability to stop these types of misuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."

Regulator Stance and Tool Functionality

The regulator Ofcom said platforms must "evaluate the potential" of people seeing illegal content on their services, but stopped short of confirming whether it was currently looking into X or Grok in regarding artificial intelligence images.

Grok is a free artificial intelligence—with some additional features—which answers X users' queries when they tag it in a message.

It is often used to offer reactions or additional background to other posters' comments, but people on X are also can modify an uploaded image through its artificial intelligence picture modification feature.

History of Controversy and Policy

The tool has been faulted for permitting users to generate pictures and videos with nudity and sexually suggestive content, and it was in the past accused of making sexually explicit material of celebrities.

"The platform has been permitting the generation and distribution of these images for a long time without taking any action and we have yet to see any challenge by regulators," Professor McGlynn added.

XAI's own terms of service forbids "showing images of persons in a sexually explicit fashion".

In a statement, Ofcom said it was against the law to "generate or share deepfakes or child sexual abuse material" and stated this included sexual deepfakes made with AI.

It said platforms such as X were mandated to take "suitable steps" to "reduce the danger" of users viewing unlawful material on their platforms and to take down it promptly when they become aware of it.