Opinion: Computer security safeguards are never perfect

Cybersecurity has always been an uneven contest. An attacker needs to find only one weakness to get inside a computer system. A defender has to protect every possible point of entry.

Until recently, human limitations helped keep that imbalance under control. Finding a serious, previously unknown software flaw, often called a zero-day vulnerability, could require extraordinary skill and hundreds of hours of work.

Artificial intelligence is beginning to change that equation. Over the past year, AI systems have moved well beyond writing emails or summarizing documents. Experimental autonomous agents have shown they can search for software weaknesses, develop exploits, and combine several vulnerabilities into a coordinated attack.

In some tests, AI agents have even escaped restricted computing environments, known as sandboxes, and exploited surrounding infrastructure.

The concern is not that these systems have suddenly become conscious. It is that they are becoming extremely competent at repetitive problem-solving. An AI agent does not get tired, bored, or distracted. It can probe a system continuously, trying thousands of possible approaches until something works.

That could create a new cybersecurity problem: too many vulnerabilities to fix.

Technology departments already struggle to keep operating systems, servers, databases, routers, and other equipment up to date. If millions of automated systems begin searching for flaws around the clock, developers could face a flood of newly discovered vulnerabilities.

Humans may simply be unable to keep up. A traditional security patch can take days or weeks to investigate, test, and install. An AI-assisted attack could develop much faster. The likely response is that cybersecurity will become increasingly automated on both sides.

Defensive AI systems may soon identify flaws, write software patches, test them in simulated environments, and deploy repairs with little human involvement. The U.S. Defense Advanced Research Projects Agency, better known as DARPA, has already encouraged this direction through its AI Cyber Challenge.

In an era of automated attacks, automated defense may become essential. AI companies are well aware of the danger. Major laboratories use red-teaming, in which experts deliberately try tomisuse or break AI systems, and they train models to reject requests involving harmful cyberattacks. They also monitor systems for suspicious behaviour.

But safeguards are never perfect. Restrictions can sometimes be bypassed through carefully designed prompts, specialized retraining, or other techniques. The risk becomes greater when an AI system is connected to real tools, including computer terminals, programming software, or networks.

At that point, the difference between a helpful troubleshooting assistant and an offensive cyber tool can become very small.

The concern also extends far beyond computer security. The same kind of reasoning that helps an AI find weaknesses in software can be applied to biology. Systems developed to design new medicines and proteins could potentially be misused to create toxic compounds or improve dangerous biological agents.

DNA synthesis companies already screen genetic sequences for potentially harmful material. In the future, those safeguards may face the same problem as computer firewalls: automated systems testing them relentlessly for weaknesses.

Critical infrastructure presents another risk. Water-treatment plants, electrical grids, factories, and transportation systems often depend on aging industrial computers connected to modern networks. Many use SCADA systems, short for Supervisory Control and Data Acquisition, to monitor and control equipment.

An AI agent capable of quickly mapping one of these networks, finding an obscure software flaw, and combining several weaknesses into a working attack could turn a computer problem into a real-world disruption.

A blackout, water failure, or transportation shutdown may increasingly begin with software. The larger issue is that human oversight may no longer be fast enough for every high-risk technological system. AI is becoming better than humans at some narrowly defined technical tasks, particularly those involving speed, repetition, and large amounts of data.

If we want to protect computer networks, biological systems, and critical infrastructure, our defenses will have to become just as fast and adaptable as the technologies that threaten them.

Tim Philp has enjoyed science since he was old enough to read. Having worked in technical fields all his life, he shares his love of science with readers weekly. He can be reached by e-mail at tphilp@bfree.on.ca.