The online platform X has provoked major outrage after individuals employed its AI chatbot Grok to alter images of women by taking off their attire.
Several examples have been seen where the tool has been used to digitally undress women to show them in swimwear without permission, as well as to place them in sexualized contexts.
Developer Response and Legal Position
XAI, the organization responsible for Grok, remained silent to questions, other than an automatically-generated message stating "legacy media lies".
A government spokesperson stated that legislation is being brought forward to outlaw nudification tools. Under a proposed criminal offence, anyone who distributed such technology would "face a jail term and significant fines".
"The platform could prevent these types of misuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said platforms must "consider the danger" of people seeing unlawful material on their platforms, but stopped short of confirming whether it was currently examining X or Grok in regarding AI-generated images.
Grok is a free AI assistant—with some additional features—which answers X users' requests when they mention it in a message.
It is frequently used to offer commentary or further background to other posters' remarks, but people on X are also can modify an posted picture through its AI image editing feature.
History of Criticism and Rules
The chatbot has been criticised for permitting users to create photos and videos with explicit content and sexually suggestive content, and it was earlier alleged to have been involved in generating sexually explicit material of public figures.
"X has been enabling the generation and spread of these images for a long time without taking any steps and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service bans "showing images of persons in a pornographic way".
In a comment, Ofcom said it was illegal to "generate or distribute non-consensual intimate images or child sexual abuse material" and confirmed this applied to explicit AI images generated with AI.
It said platforms such as X were obligated to take "suitable steps" to "reduce the risk" of users encountering unlawful content on their services and to remove it swiftly when they become aware of it.