Canada privacy watchdog says Grok generates explicit deepfakes without users valid consent

The Office of the Privacy Commissioner of Canada said in a report on Thursday that X (formerly Twitter) had not obtained valid consent to collect users’ personal information before generating explicit deepfakes.

The report said explicit deepfakes present an individual in their vulnerable states and are invasive in nature. Collection of sensitive personal information–sex life or sexual orientation–requires valid consent from the individual who understands the purpose and consequences of such collection. The commission found that X users could not have expected that their images, posted for an unrelated purpose on the platforms, can be processed by AI tools to generate explicit deepfakes of themselves; thus, the companies have not obtained valid consent.

The report also concluded that the companies could have provided their AI services in a less intrusive manner. One such measure could have been a safeguard at the outset to prevent the generation of deepfakes.

In response, the companies claimed that users were responsible for initiating the collection of personal information when they prompted Grok to generate and post deepfakes. The commissioner disagreed, contending that the companies are responsible when they provide the service to collect personal information and generate deepfakes. The commission also concluded that the current safeguards in place are insufficient to protect users’ privacy under Canadian law.

The commission recommends that X suspend its Grok function until a comprehensive privacy safeguard is in place. It also recommends that the companies include prominent warnings about the risks of AI outputs before X users post their information on the platforms. The companies refuted the findings, arguing that they have already removed illegal content and limited how users can use Grok.

Even though the report concluded that the companies violate users’ privacy under Canadian law, the federal Privacy Act does not empower the commissioner to compel compliance or impose penalties. Calling for a legislative amendment, Philippe Dufresne, Privacy Commissioner of Canada, said:

The Grok investigation highlights the need for modern privacy laws that are designed for a modern world and include administrative monetary penalties and the power to make orders to bring companies into compliance. Modernized privacy laws can support both innovation and privacy protection by establishing clear guardrails that ensure the responsible and trustworthy management of Canadians’ personal information.

Relatedly, the Canadian federal government introduced a new law to regulate AI chatbots on Wednesday. The bill, if enacted in its current form, requires AI chatbot operators to act responsibly and mitigate the risk of its chatbots communicating harmful content to their users.