The social media network X has sparked major backlash after users used its AI chatbot Grok to manipulate pictures of females by stripping their garments.
Multiple examples have surfaced where the technology has been employed to digitally undress female subjects to show them in swimwear without permission, as well as to put them in sexualized situations.
Company Response and Legal Position
XAI, the organization that created Grok, offered no comment to inquiries, other than an automatically-generated response stating "traditional news outlets lies".
A government spokesperson stated that new laws is being introduced to outlaw image stripping software. Under a upcoming law, anyone who supplied such technology would "be subject to a prison sentence and heavy fines".
"The platform could prevent these forms of abuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Stance and Tool Capabilities
The regulator Ofcom said technology companies must "assess the danger" of people seeing illegal content on their networks, but did not confirm whether it was at this time investigating X or Grok in concerning AI-generated images.
Grok is a free AI assistant—with some additional functions—which answers X users' prompts when they reference it in a tweet.
It is frequently used to give reactions or additional background to other users' comments, but people on X are also can alter an posted image through its artificial intelligence picture modification feature.
Background of Criticism and Policy
The chatbot has been criticised for allowing users to create pictures and footage with nudity and sexually suggestive content, and it was in the past accused of generating inappropriate material of celebrities.
"The platform has been permitting the creation and distribution of these images for months without taking any action and we have not seen any action by oversight bodies," the expert added.
XAI's own acceptable use policy bans "portraying likenesses of individuals in a sexually explicit manner".
In a comment, Ofcom said it was illegal to "create or circulate private pictures or child sexual abuse material" and clarified this covered sexual deepfakes created with AI.
It said services such as X were obligated to take "suitable actions" to "lessen the danger" of users viewing illegal material on their services and to take down it swiftly when they learn of it.
