🔗 Share this article

The online network X has caused widespread backlash after individuals used its artificial intelligence assistant Grok to manipulate pictures of females by stripping their garments.

Multiple examples have emerged where the system has been used to remove clothing from female subjects to show them in swimwear without consent, as well as to insert them in inappropriate situations.

Developer Reaction and Legal Position

XAI, the company that created Grok, did not respond to inquiries, aside from an auto-reply message stating "legacy media lies".

A Home Office official stated that new laws is being introduced to prohibit image stripping software. Under a proposed criminal offence, anyone who distributed such technology would "be subject to a prison sentence and significant fines".

"The platform is capable of halting these types of abuse if they wanted to," said Clare McGlynn, adding they "operate with impunity."

Regulator Position and Tool Functionality

The communications watchdog Ofcom said platforms must "evaluate the risk" of people viewing illegal content on their networks, but stopped short of confirming whether it was at this time looking into X or Grok in relation to artificial intelligence images.

Grok is a free AI assistant—with some paid-for functions—which reacts to X users' queries when they reference it in a post.

It is commonly used to give reactions or further background to other posters' comments, but people on X are also can alter an posted photo through its artificial intelligence picture modification tool.

Background of Criticism and Rules

The chatbot has been criticised for permitting users to produce pictures and videos with nudity and sexualised content, and it was earlier alleged to have been involved in generating graphic material of celebrities.

"The platform has been allowing the production and sharing of these images for months without taking any steps and we have not seen any intervention by regulators," the expert added.

XAI's own terms of service forbids "portraying likenesses of people in a pornographic manner".

In a comment, Ofcom said it was unlawful to "create or distribute private pictures or illegal content" and confirmed this covered explicit AI images made with AI.

It said platforms such as X were mandated to take "suitable steps" to "reduce the danger" of users encountering illegal material on their platforms and to delete it swiftly when they are notified of it.