Elon Musk's social media platform X has sparked significant backlash after users utilized its AI chatbot Grok to manipulate photos of females by stripping their garments.
Numerous instances have emerged where the tool has been used to digitally undress female subjects to render them in bikinis non-consensually, as well as to place them in sexualized contexts.
Developer Reaction and Legal Stance
XAI, the organization that created Grok, remained silent to inquiries, aside from an auto-reply response stating "legacy media lies".
A Home Office official stated that legislation is being brought forward to ban nudification software. Under a new criminal offence, anyone who supplied such technology would "face a jail term and significant penalties".
"The platform could prevent these types of abuse if they wanted to," said a law professor, adding they "operate with impunity."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said tech firms must "assess the danger" of people viewing unlawful material on their networks, but did not confirm whether it was currently examining X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost AI assistant—with some paid-for functions—which reacts to X users' prompts when they reference it in a message.
It is frequently used to provide commentary or further background to other users' remarks, but people on X are also able to edit an uploaded image through its AI image editing function.
History of Controversy and Policy
The tool has been criticised for enabling users to create photos and videos with nudity and sexually suggestive material, and it was previously alleged to have been involved in creating graphic content of celebrities.
"The platform has been enabling the production and spread of these images for a long time without taking any action and we have yet to see any challenge by regulators," the expert added.
XAI's own terms of service bans "portraying likenesses of people in a pornographic manner".
In a comment, Ofcom said it was unlawful to "generate or distribute deepfakes or illegal content" and stated this covered explicit AI images generated with AI.
It said networks such as X were required to take "necessary actions" to "reduce the risk" of users coming across unlawful material on their platforms and to take down it promptly when they become aware of it.
