OpenAI agent breached a government site. Could the same happen in Canada?

An OpenAI agent gained unauthorized access to an Australian government website. As Canada rolls out agentic AI tools, what happens if something similar occurs here?

The agent was researching medicine spending when it reached Australia's Medicare Statistics Reporting Service. When the portal blocked its requests, it found another route in, accessing public and non-public files. Australian officials said no personal Medicare information is believed to have been accessed, while OpenAI noted its models took actions the company "did not intend."

Shared Services Canada has begun deploying its GC AI Platform with select federal departments and agencies. It includes CANChat and "agentic workflow AI tools," with expansion planned through 2027.

What exactly is a 'rogue' AI agent?

AI agents can do more than answer questions. Canada's Cyber Centre describes agentic AI as systems that can interpret their environment, make decisions and take actions to achieve goals with greater autonomy than traditional AI tools.

"Rogue" is an informal label. It does not mean an AI has become conscious or deliberately malicious. It can describe an agent behaving outside its intended boundaries, such as bypassing a safeguard or reaching a system it was not authorized to access.

Australia is not the first warning sign. OpenAI disclosed in July that models in a cybersecurity evaluation circumvented controls meant to isolate them from the internet. Anthropic later disclosed three incidents in which one model gained unauthorized access to organizations' systems.

Could Canadian law handle a rogue AI agent?

Teresa Scassa, Canada Research Chair in Information Law and Policy at the University of Ottawa, said there's no simple answer.

Canada's Criminal Code contains offences involving unauthorized computer use. However, Scassa said criminal prosecution could be difficult because offences require a mental element, or intent.

"The developers and deployers of these agents don't intend for the agents to go rogue and in fact, may take all kinds of measures to ensure that they are contained in testing environments," Scassa told Yahoo Canada.

Section 342.1 of the Criminal Code covers unauthorized computer use, including obtaining computer services "fraudulently and without colour of right."

Civil lawsuits could offer another route. Scassa said negligence may apply, potentially putting the developer or deployer in the legal spotlight. But a key question would be what constitutes reasonable care for technology evolving this quickly.

What if personal information is exposed?

The legal picture changes if an AI incident compromises personal information.

Scassa said scrutiny could fall on the organization holding the data and whether it had appropriate safeguards.

Federal institutions subject to Canada's Privacy Act must report material privacy breaches to the Treasury Board Secretariat and Office of the Privacy Commissioner.

At the same time, Canada's Cyber Centre is urging caution. Organizations should never give agents broad or unrestricted access to sensitive data or critical systems, according to its guidance. It's also recommending to initially limit them to low-risk, non-sensitive tasks.

Scassa cautioned against assuming Australia's experience automatically proves Canada has a regulatory hole.

"I don't know if it is entirely clear yet if they create regulatory gaps, or if they just present new situations to which regulations have to be applied," she said.

She said requiring AI companies to disclose security incidents involving their agents "would be a good thing."

For now, the question is whether existing criminal, privacy and civil-law frameworks can fit autonomous systems.

"Since it is early days, it remains to be seen how existing law will apply and what gaps will emerge," Scassa said.

"But this may quickly become a serious problem for which some form of legislative action will be required."