Elon Musk's online network X has caused widespread anger after individuals used its AI chatbot Grok to change images of women by stripping their clothing.
Several examples have been seen where the system has been employed to virtually strip female subjects to make them appear in bikinis without consent, as well as to put them in inappropriate contexts.
Company Response and Legal Position
XAI, the firm that created Grok, did not respond to questions, except for an automatically-generated response stating "legacy media lies".
A government official stated that legislation is being proposed to ban nudification tools. Under a upcoming criminal offence, anyone who provided such tools would "be subject to a prison sentence and significant penalties".
"The platform could prevent these forms of abuse if they wanted to," said a law professor, adding they "operate with impunity."
Oversight Body Stance and Platform Functionality
The regulator Ofcom said technology companies must "evaluate the risk" of people accessing illegal content on their services, but stopped short of confirming whether it was presently looking into X or Grok in regarding AI-generated images.
Grok is a free chatbot—with some paid-for features—which responds to X users' prompts when they tag it in a message.
It is commonly used to provide reactions or more background to other posters' remarks, but people on X are also capable of modify an uploaded picture through its AI image editing function.
History of Criticism and Rules
The tool has been faulted for allowing users to create pictures and videos with explicit content and sexually suggestive content, and it was in the past alleged to have been involved in generating sexually explicit content of celebrities.
"The platform has been allowing the creation and spread of these pictures for a long time without taking any measures and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "showing images of people in a pornographic manner".
In a comment, Ofcom said it was unlawful to "create or share private pictures or child sexual abuse material" and stated this applied to sexual deepfakes created with AI.
It said platforms such as X were required to take "necessary steps" to "minimize the danger" of users viewing illegal content on their services and to delete it promptly when they learn of it.
