The social media platform X has sparked widespread outrage after users employed its AI chatbot Grok to alter photos of females by taking off their attire.
Several instances have surfaced where the system has been employed to remove clothing from female subjects to render them in bikinis without consent, as well as to place them in sexually explicit contexts.
Developer Reaction and Legal Position
XAI, the organization that created Grok, remained silent to requests for comment, other than an auto-reply response stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being proposed to ban image stripping technology. Under a upcoming law, anyone who distributed such technology would "receive a jail term and significant fines".
"The platform could prevent these forms of abuse if they chose to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said technology companies must "assess the potential" of people seeing unlawful material on their services, but stopped short of confirming whether it was currently looking into X or Grok in relation to artificial intelligence images.
Grok is a free AI assistant—with some additional functions—which reacts to X users' requests when they reference it in a post.
It is frequently used to provide commentary or additional context to other posters' comments, but people on X are also capable of alter an posted picture through its artificial intelligence picture modification feature.
History of Controversy and Rules
The tool has been criticised for permitting users to produce photos and footage with nudity and sexualised content, and it was in the past accused of making sexually explicit material of public figures.
"The platform has been enabling the production and sharing of these images for a long time without taking any action and we have not seen any challenge by regulators," the expert added.
XAI's own usage rules bans "depicting likenesses of individuals in a pornographic fashion".
In a statement, Ofcom said it was against the law to "create or circulate deepfakes or child sexual abuse material" and clarified this covered sexual deepfakes made with AI.
It said networks such as X were required to take "suitable actions" to "reduce the danger" of users encountering unlawful material on their services and to take down it promptly when they learn of it.
