Elon Musk's online platform X has provoked significant backlash after individuals utilized its artificial intelligence assistant Grok to manipulate images of females by removing their attire.
Several instances have emerged where the system has been employed to digitally undress women to show them in swimwear without permission, as well as to place them in sexually explicit scenarios.
Developer Response and Regulatory Stance
XAI, the company that created Grok, offered no comment to requests for comment, aside from an auto-reply response stating "legacy media lies".
A government representative stated that new laws is being proposed to outlaw nudification software. Under a proposed criminal offence, anyone who provided such tools would "be subject to a prison sentence and substantial fines".
"The platform is capable of halting these types of abuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said technology companies must "evaluate the danger" of people seeing illegal content on their platforms, but did not confirm whether it was currently looking into X or Grok in relation to artificial intelligence pictures.
Grok is a free AI assistant—with some additional features—which answers X users' queries when they tag it in a message.
It is frequently used to offer commentary or further context to other posters' remarks, but people on X are also capable of modify an uploaded image through its AI image editing function.
Background of Controversy and Policy
The tool has been condemned for enabling users to create images and footage with nudity and sexually suggestive material, and it was in the past alleged to have been involved in creating graphic material of celebrities.
"The platform has been enabling the production and distribution of these images for months without taking any steps and we have not seen any intervention by oversight bodies," the expert added.
XAI's own terms of service forbids "portraying likenesses of people in a graphic manner".
In a statement, Ofcom said it was illegal to "generate or circulate non-consensual intimate images or illegal content" and confirmed this covered sexual deepfakes made with AI.
It said services such as X were mandated to take "necessary actions" to "lessen the danger" of users coming across unlawful content on their services and to delete it promptly when they become aware of it.
