AI-generated image abuse is escalating at an unprecedented rate and the government and tech platforms must take urgent action to protect women and girls, the domestic abuse charity Refuge has warned.
The call from Refuge comes in response to a growing number of users of the social media platform X, owned by Elon Musk, using the AI technology to create sexualised imagery of women and girls.
Innocent images posted by women on the platform and photographs of children have been manipulated using the Grok AI tool to remove clothes, put them in sexualised positions, and add bruises and other marks of violence. Following widespread condemnation, Musk announced on Friday that he would limit the ability of users to users to edit images with Grok to paid subscribers.
But Emma Pickering, head of technology-facilitated abuse and economic empowerment at Refuge, said the move had merely monetised abuse.
“AI-generated image abuse is escalating at an unprecedented rate, demanding urgent action from tech platforms to protect women and girls from this very real harm,” she said.
“Following a series of shocking reports that Grok has been weaponised by perpetrators to generate misogynistic and violent content at the expense of women, the recent decision to restrict access to paying subscribers is not only inadequate – it represents the monetisation of abuse.
“While limiting AI image generation to paid users may marginally reduce volume and improve traceability, the abuse has not been stopped. It has simply been placed behind a paywall, allowing X to profit from harm. This response comes far too late, after a disturbing number of images have already been created and circulated. Traceability offers little comfort when conviction rates for intimate image abuse remain woefully low.”
The technology secretary, Liz Kendall, has also called for swift action, calling the sexual manipulation of women and children “despicable and abhorrent.”
“It is an insult and totally unacceptable for Grok to still allow this if you’re willing to pay for it,” Kendall said.
“I expect Ofcom to use the full legal powers parliament has given them. I, and more importantly the public – would expect to see Ofcom update on next steps in days not weeks.”
The Online Safety Act allows Ofcom to block services that fail to comply with UK law from being accessed in the country. “If Ofcom decide to use those powers they will have our full support,” Kendall said.
“I expect all platforms to abide by Ofcom’s new Violence Against Women and Girls (VAWG) guidance and if they do not, I am prepared to go further,” Kendall added. “We are as determined to ensure women and girls are safe online as we are to ensure they are safe in the real world. No excuses.”
However, no direct action has yet been taken to stop X and Grok creating the images, and Pickering says the government must intervene.
“We must be clear: at the root of the surge in AI-generated image abuse are real perpetrators, driven by entrenched misogyny,“ she said.
“Despite the publication of Ofcom’s Violence Against Women and Girls (VAWG) guidance last year, these measures are not legally binding, and this week’s reports prove that when tech companies are left to police themselves, women and girls pay the price.
“If platforms allow abuse to persist – or worse, profit from it – the government must intervene. Only last month, the Government published its VAWG Strategy, titled Freedom from Violence and Abuse, setting out its plans to halve violence against women and girls by 2034.
“That ambition cannot be met unless it effectively addresses technology-facilitated abuse, which, as recent reports have demonstrated, is an extremely pervasive and dangerous form of VAWG.
“Refuge is calling on the government to take urgent action to hold Grok and other tech companies that fail to tackle online abuse to account through stronger regulations and meaningful consequences. If left unchecked, women and girls’ safety will remain at stake.”
