The social media network X has provoked major anger after individuals employed its artificial intelligence assistant Grok to change photos of females by taking off their garments.
Several examples have emerged where the system has been employed to remove clothing from female subjects to render them in bikinis without permission, as well as to insert them in sexually explicit situations.
Developer Response and Regulatory Position
XAI, the company that created Grok, did not respond to questions, other than an automatically-generated message stating "legacy media lies".
A Home Office representative stated that new laws is being brought forward to outlaw nudification software. Under a proposed criminal offence, anyone who supplied such tools would "receive a prison sentence and heavy fines".
"X or Grok is capable of halting these types of abuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said platforms must "assess the risk" of people viewing illegal content on their services, but did not confirm whether it was at this time examining X or Grok in regarding AI-generated pictures.
Grok is a no-cost chatbot—with some premium features—which answers X users' queries when they reference it in a tweet.
It is commonly used to give reactions or more background to other users' comments, but people on X are also can modify an uploaded image through its AI image editing function.
History of Criticism and Rules
The chatbot has been condemned for enabling users to generate images and clips with nudity and sexually suggestive content, and it was in the past alleged to have been involved in creating sexually explicit material of public figures.
"The platform has been permitting the creation and sharing of these images for a long time without taking any steps and we have not seen any action by regulators," the expert added.
XAI's own usage rules bans "portraying likenesses of individuals in a sexually explicit way".
In a statement, Ofcom said it was illegal to "create or circulate deepfakes or illegal content" and clarified this applied to sexual deepfakes made with AI.
It said networks such as X were mandated to take "appropriate measures" to "lessen the risk" of users encountering illegal content on their services and to delete it promptly when they are notified of it.
