The online network X has caused major outrage after individuals employed its AI chatbot Grok to alter pictures of women by stripping their attire.
Numerous cases have been seen where the tool has been employed to remove clothing from female subjects to make them appear in swimwear without permission, as well as to place them in sexually explicit situations.
Company Response and Legal Stance
XAI, the company that created Grok, remained silent to inquiries, other than an standard response stating "legacy media lies".
A government representative stated that new laws is being introduced to ban image stripping tools. Under a new law, anyone who distributed such technology would "face a prison sentence and significant penalties".
"The platform could prevent these types of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Position and Tool Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the potential" of people viewing unlawful material on their services, but did not specify whether it was at this time looking into X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost chatbot—with some paid-for functions—which responds to X users' prompts when they reference it in a post.
It is commonly used to provide reactions or more context to other users' remarks, but people on X are also capable of alter an posted picture through its artificial intelligence picture modification function.
History of Controversy and Rules
The system has been condemned for allowing users to create images and clips with explicit content and sexualised content, and it was earlier accused of making sexually explicit content of public figures.
"The platform has been enabling the generation and sharing of these images for a long time without taking any action and we have not witnessed any challenge by regulators," the expert added.
XAI's own usage rules forbids "depicting images of persons in a graphic way".
In a statement, Ofcom said it was against the law to "create or circulate non-consensual intimate images or illegal content" and clarified this applied to explicit AI images made with AI.
It said platforms such as X were required to take "suitable actions" to "minimize the danger" of users encountering illegal content on their platforms and to remove it swiftly when they become aware of it.
