The social media platform X has sparked widespread outrage after people used its artificial intelligence assistant Grok to alter pictures of women by taking off their garments.
Numerous examples have surfaced where the tool has been employed to virtually strip women to make them appear in bikinis without consent, as well as to put them in inappropriate situations.
Company Response and Regulatory Position
XAI, the company responsible for Grok, remained silent to questions, except for an standard response stating "traditional news outlets lies".
A government spokesperson stated that new laws is being brought forward to ban nudification tools. Under a proposed law, anyone who supplied such technology would "receive a prison sentence and significant fines".
"X or Grok has the ability to stop these forms of abuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said platforms must "evaluate the potential" of people accessing unlawful material on their services, but did not specify whether it was presently investigating X or Grok in regarding AI-generated images.
Grok is a no-cost AI assistant—with some premium features—which reacts to X users' requests when they mention it in a post.
It is frequently used to provide reactions or more background to other users' remarks, but people on X are also can alter an uploaded picture through its AI image editing tool.
Background of Criticism and Rules
The chatbot has been condemned for allowing users to create photos and videos with explicit content and sexualised material, and it was in the past accused of creating sexually explicit material of public figures.
"X has been allowing the production and spread of these pictures for a long time without taking any steps and we have not seen any intervention by regulators," the expert added.
XAI's own usage rules bans "portraying images of persons in a pornographic manner".
In a comment, Ofcom said it was unlawful to "generate or distribute private pictures or illegal content" and clarified this included sexual deepfakes created with AI.
It said services such as X were required to take "suitable steps" to "minimize the risk" of users encountering unlawful content on their services and to delete it swiftly when they are notified of it.
