The social media platform X has caused major anger after people employed its AI chatbot Grok to manipulate pictures of females by taking off their garments.
Numerous examples have emerged where the tool has been used to digitally undress women to render them in bikinis without permission, as well as to insert them in sexualized contexts.
Company Response and Legal Position
XAI, the organization responsible for Grok, remained silent to requests for comment, other than an auto-reply message stating "traditional news outlets lies".
A government official stated that new laws is being introduced to outlaw nudification tools. Under a new law, anyone who distributed such tools would "face a jail term and heavy penalties".
"The platform is capable of halting these forms of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "assess the risk" of people viewing illegal content on their services, but stopped short of confirming whether it was presently examining X or Grok in relation to AI-generated pictures.
Grok is a free artificial intelligence—with some premium features—which answers X users' requests when they reference it in a message.
It is frequently used to provide reactions or further context to other posters' comments, but people on X are also capable of edit an posted image through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The tool has been faulted for permitting users to generate pictures and footage with explicit content and sexualised content, and it was earlier accused of generating sexually explicit content of public figures.
"X has been enabling the production and distribution of these images for months without taking any steps and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "depicting images of persons in a sexually explicit manner".
In a statement, Ofcom said it was against the law to "generate or circulate private pictures or illegal content" and confirmed this applied to explicit AI images created with AI.
It said services such as X were required to take "necessary actions" to "lessen the risk" of users encountering unlawful material on their services and to remove it quickly when they learn of it.
