NZ Police don't know what to do about Grok child sexual abuse images...
What should you do if you or your child is a victim of a Grok deepfake?
This question is simple, right?
Feel free to skip to the comments if you read this already in your email!
It's something I've been asked multiple times by parents and that's why I began working with an expert on an explainer about sexualised deepfakes.
Please feel very free to read this piece before coming back to this one, as it adds a lot of very useful context, and I am hugely grateful to Dr Cassandra Mudgway for giving us so much time and useful information.
Emily Writes
As an add-on to the story, I asked my colleague Josh to contact NZ Police to ask them about what members of the public needed to know about this sort of material. We had quite a few questions, but figured it would be fairly straightforward to tell the public what to do about about child sexual abuse material on Grok/X/Twitter.
I mean, NZ Police do know what people should do if their child is posted naked on Twitter due to a paedophile with chatbot access. Don't they?
As it turns out, maybe they don't.
Unbelievably, we asked the police and they... would not tell us! Here's Josh with the details:
Emily Writes Weekly sent the following questions to the New Zealand police on the morning of Wednesday, 14 January 2025.
- In your organisation's opinion, is it legal to "nudify" pictures of users without consent - including through tangential methods like "portray her wearing a clingwrap bikini"?
- Is it legal for users to similarly "nudify" or adjust images of children to portray them in scantily-clad clothing, "clingwrap bikinis" or similar?
- Does this material constitute child sexual abuse material (CSAM)?
- Given the amount of AI generated CSAM on Grok/X, is it legal for New Zealanders to use those platforms?
If it is legal, why? If it is illegal, why? - Do these images/apps fall under the jurisdiction of the Classification Office, in a similar manner to films, TV, publications, and video games?
- Why has X/Twitter/Grok not been subject to restrictions in New Zealand?
If not, why not? - What should New Zealanders do if they encounter AI-generated CSAM or nonconsensual "nudified" pornography on those platforms?
- What should adult members of the New Zealand public do if their image is "nudified" or made pornographic without consent on Grok/X?
- What should members of the New Zealand public do if an image of their child is used to generate "nudified" images or similar on Grok/X/any other platform?
Anticipating that some of the questions might be difficult to answer, we included the line: "If you cannot answer all the questions by the deadline, please answer which ones you can today and note which questions can be answered later in the week. If you cannot answer a question, please indicate which ones, and why they cannot be answered."
The Police media office replied with:
We don’t have capacity to service requests from blog sites, however if you have a letter of assignment from a mainstream news outlet then we will be able to look into this. Another option would be seeking the information through the Official Information Act.
We followed up, pointing out that many mainstream media outlets have similar or identical questions, and that Official Information Act requests take enormous time and resources to respond to, compared to media queries. The Police replied with:
We service requests for mainstream media outlets and can deal with between 150-200 queries a day. As a media team, we do not have capacity to answer questions from non-mainstream outlets. We also do not have the answers to the questions you’ve asked, and would need to get them from investigators and subject matter experts, so an OIA is the appropriate course of action, unless you have a letter of assignment.
In our opinion, this is a perfect example of the Police failing the public, who have a right to know what they should do if someone is using X, Grok, or another AI to make non-consensual abuse images of them – or their children.
And we think it's worth asking: why can't the NZ Police give even a simple answer to our questions?
Emily and I both have postgraduate qualifications in journalism. We have also both worked in comms. (I still do comms work!) And when we were a member of comms teams, one of the unbreakable rules was: Placate, don't escalate.
From a comms perspective, asking that a journalist do an OIA instead of answering questions is ludicrous. OIAs cannot be ignored or brushed off. Even to deny an OIA request often takes a great deal of work. To answer an OIA – especially one with multiple or in-depth questions – can take weeks of work over multiple staff members. Ignoring the fact that Emily and I are both journalists, to invite a horde of bloggers and members of the public to take out multiple OIAs is asking for an administrative nightmare.
The sensible answer to my query would have been something like:
We regret that due to the detailed nature of your questions we cannot give a reply by deadline. However, here is a statement you may attribute to the Police Media Centre. "Depictions of child exploitation or abuse, whether genuine or generated by AI, are illegal under New Zealand law. If you see any such content on any website, close the browser, do not visit the website again, and report the site and content immediately to police and/or the Department of Internal Affairs. [insert link to reporting hotlines.]"
Doing that would have taken the Police media team much less time than writing the multiple emails they've now sent to Emily and I. But they didn't. Why?
Let's pry a little deeper. Child exploitation images are so illegal that, if you run the risk of accidentally seeing them on a website, the common-sense advice is "do not visit that website again." And, as evidenced in multiple articles like this Webworm piece, it is frighteningly easy to both view and create material that may be regarded as illegal pornography or CSAM, on X.
But it would seem that the New Zealand Police cannot or will not tell members of the public whether it is legal to use X, or even what to do if someone generates an explicit image of you or your child on that website.
And maybe that's because X, which wilfully hosts all kinds of unethical and illegal content – from white supremacy and conspiracy theories through to AI-generated non-consensual pornography and CSAM – is owned by the richest man in the world and is still used by a large number of New Zealand organisations and individuals, including a number of sitting MPs.
Perhaps the police are anxious to avoid antagonising the world's richest man or declaring that people should avoid one of the world's most well-known websites, even though it's potentially riddled with highly illegal material.
Or maybe they're a little sensitive about the topic after the Jevon McSkimming scandal.
Anyway, if you do come across illegal content on X (or anywhere else on the internet) here is a Department of Internal Affairs page where you can report it.
Hey, it's Emily again. After pointing out the NZ Police that I'm a trained journalist who has worked for almost every media organisation in Aotearoa, including being an editor, and that this publication regularly has as many readers as legacy media, I got the same response Josh did.
But this time they also said:
It’s important to note than anyone, including those not in the media industry, is welcome to submit OIA, sign up to our media releases and follow our social media channels.
So, I guess, if your child is a victim of child sexual abuse material online created by AI, or you are a victim of a pornographic deep fake - the advice from NZ Police is...to do an OIA.
Truly, we're as shocked as you are.
Share my stuff to your social network of choice!
Emily Writes