The social media platform X has provoked significant backlash after people used its AI chatbot Grok to manipulate images of women by removing their clothing.
Several examples have emerged where the system has been employed to digitally undress female subjects to show them in swimwear without permission, as well as to insert them in sexualized situations.
Company Response and Legal Position
XAI, the organization that created Grok, offered no comment to inquiries, other than an standard message stating "traditional news outlets lies".
A government representative stated that new laws is being proposed to outlaw nudification technology. Under a new law, anyone who distributed such technology would "receive a prison sentence and substantial penalties".
"The platform has the ability to stop these forms of misuse if they chose to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Stance and Platform Capabilities
The regulator Ofcom said tech firms must "evaluate the risk" of people seeing unlawful material on their platforms, but did not confirm whether it was at this time looking into X or Grok in regarding AI-generated pictures.
Grok is a free AI assistant—with some additional features—which responds to X users' prompts when they reference it in a post.
It is commonly used to provide reactions or more context to other posters' remarks, but people on X are also able to alter an uploaded picture through its artificial intelligence picture modification tool.
History of Controversy and Rules
The chatbot has been faulted for permitting users to create pictures and footage with nudity and sexually suggestive content, and it was earlier alleged to have been involved in making sexually explicit content of public figures.
"X has been allowing the generation and spread of these pictures for a long time without taking any action and we have yet to see any challenge by oversight bodies," the expert added.
XAI's own terms of service bans "depicting images of people in a pornographic manner".
In a comment, Ofcom said it was illegal to "make or distribute private pictures or illegal content" and stated this covered explicit AI images created with AI.
It said networks such as X were mandated to take "suitable steps" to "lessen the danger" of users viewing illegal material on their platforms and to remove it promptly when they become aware of it.
