Drone warfare with a moral code

Throughout my career, I have always been fascinated by how decisions of great moment are taken, whether amid conflict or in the worlds of intelligence and security. Sometimes all goes well. Sadly, even with highly intelligent people, sometimes it does not. On occasions, there is failure to take a decision, which is itself a decision of sorts, usually one that ends badly. To simplify greatly, I think for a sound decision to be taken, two different kinds of thinking have to be integrated inside the brain of the person making the decision.

There is the analytic process: examining evidence, weighing up what we think we know to establish what are the options and their likely direct and indirect implications, and estimating the limits of what can be done with the resources we have. I examined the analytic process in my book, How Spies Think. But there is also the other kind of thinking, that comes from the limbic system, lying beneath the cerebral cortex and above the brainstem. That surfaces our memories, desires, hopes and fears and allows us to apply our ethical values to establish what we ought to do. I explored this in my book Principled Spying.

We need both to be integrated for a sound decision. Too much of the analytic and we will not carry people with our decisions. People will not risk their lives for impersonal calculations and spreadsheets. Too much of the emotional and we can end up with empty boosterism. Simply hoping for the best is not a sound strategy. It is too easy to move from ‘this is how I want to see the world’ to ‘this is how I believe the world is’, and to move from ‘this what I want to achieve from my decision’ to ‘and this is what I am bound to achieve from my decision’.

The Italian politician and Marxist philosopher Antonio Gramsci described the ideal for strategic decision as needing ‘pessimism of the intellect, and optimism of the will’. Churchill in 1940 rallied the British people with his narrative of resistance, but did not forget the necessity of boosting Spitfire and Hurricane production.

Today, we see the dramatic speeding up of planning for armed conflict. This acceleration has been accomplished with the application of advanced artificial intelligence and data science to bring disparate sources of information and intelligence together, generate targets and allocate those to the available assets the military commander may have, on land, sea and subsurface, and in the air.

On the battlefield itself, modern warfare is increasingly forcing humans out of the moment-to-moment decision-making process — not because anyone has chosen this, but because the sheer speed of events leaves no alternative. When an enemy can deploy weapons such as hypersonic missiles (which travel so fast that radar warning and impact can be separated by mere seconds), jam communications, or unleash swarms of drones faster than any person can mentally process, there simply isn’t enough time for a human to assess the situation and react in time. The lethal threat arrives before the decision can be made.

This creates a fundamental shift in how human oversight has to work. Traditionally, a human would be in the loop — directly involved in each decision, like a soldier choosing when to fire. Increasingly, the role is becoming on the loop — setting the rules and boundaries in advance, then monitoring a system that acts on its own, and stepping in only if circumstances change or something is seen to go wrong. The human is still responsible, but they are now a supervisor rather than a participant.

This is not a failure of policy. It is a constraint of the operational environment. It means that our ethical obligations also shift. It is no longer sufficient to ask: how do we keep humans in the loop? We must ask instead: how do we ensure that the loop that the human is operating on governs systems that can reason morally in the ways that we want? And how do we ensure that when circumstances change the system is able to respond appropriately?

We see evidence on battlefields today that artificial intelligence can be trained to do the analytic part of a decision to use lethal force. But is it possible to train an AI system to make integrated decision-making? Can an AI do not just the analytic reasoning, which it already can, but also apply moral constraints? Can we teach advanced AI systems to act ethically?

In 2014, I chaired a Policy Commission on the security implications of drones. With retired senior RAF officers, ethicists, academics and engineers. We concluded that, with the state of machine learning in 2014:

It would not be possible to programme lethal autonomous weapons systems for ground operations consistent with the Geneva conventions. For example, to implement the distinction between civilians and combatants, and to exercise the proportionality necessary for compliance with international humanitarian law.

That was 2014. Now, in the last few years, we have witnessed a technological revolution with the arrival of generative AI, agentic AI and AI capable of complex coding. I have had to change my mind about autonomous weapons systems. I think it should be possible to train an AI to act ethically. It is also the case that some environments of conflict (such as the maritime) pose less challenging ethical issues than would a complex ground scenario. The implications of a tethered sea mine in a declared minefield are not the same as the ethical understanding required of an autonomous system operating in a crowded city.

For that, we do not need to invent new ethics. We can use the concepts that we have become familiar with in the laws of war and Geneva Conventions: the principles of necessity and proportionality, protection of civilians, avoidance of collateral damage and only acting when there is a reasonable prospect of success. For instance, a ‘human in the loop’ today has to assess:

Is this a legitimate military target? How confident am I in the intelligence that lies behind that identification? What is the likely military effect of engaging? What are the anticipated civilian casualties? Is that collateral damage proportionate to the military advantage? Are there alternative means of achieving the objective with lower civilian cost? Can I afford to wait or must I act?

These are not mystical intuitions. They are weighted trade-offs between competing values: military effect against civilian harm, confidence against time pressure, reversibility against operational necessity. A trained AI system ought to be able to optimise across these variables according to weights that human commanders have assigned. And here is the crucial change: advanced AI can make those weights explicit, logged and auditable. It is a formalisation that potentially makes moral reasoning transparent, challengeable and improvable in a way that a human decision made in three seconds under fire simply is not.

We know from experience that human decision-making under pressure and in dangerous circumstances (when lives are at stake) will never be perfect. AI systems, likewise, will never be perfect. Edge cases and difficult calls will always exist. Paradoxically, we are likely to see fewer errors using advanced AI with weighted utility frameworks that mirror the logic already embedded in international humanitarian law and the laws of war. Moral weights must be adaptive, human-controlled and context-sensitive, like the rules of engagement we use today. I want to be precise here. I am not suggesting that machines can be moral agents in any philosophically robust sense. I am suggesting something more specific and more tractable: that the variables a machine optimises across, and the weights it assigns to competing considerations, can and should reflect moral reasoning that humans have and that through training they have encoded in advance.

Who is accountable when the machine gets it wrong? The commander who set the weights. This is actually one of the strongest arguments for this approach. Human decision-making in combat is extraordinarily difficult to reconstruct and hold accountable after the fact. A logged, parameterised weight function changes this. This does not eliminate the tragedy of wrong decisions. But it makes moral accountability traceable in a way that purely human decision-making often is not.

The issue has become urgent. Both Russia and China have already fielded autonomous weapons systems that do not have a human in the loop to take decisions to use lethal force. They are investing heavily in relevant technologies such as robotics and the control of drone swarms – where targeting information is shared between individual drones, and the hive mind of the swarm decides which of them are best placed to attack which part of a target, such as a warship, a city centre or an airfield.

We face an ethical dilemma. Can we in all conscience deny this technology to our armed forces? Can we, in European countries, demand of our armed forces that they prepare to do battle with adversaries equipped with fully autonomous weapons systems, drone swarms and killer robots? Imagine a British or French warship at sea, perhaps in the Strait of Hormuz, facing an advanced drone swarm. There would only be seconds in which to control the response. The same will be true of advanced air defence, as we already see in the case of Israel’s Iron Dome. And on the future battlefield, as we see in Ukraine.

What I believe we should aim for is to build systems that genuinely have a human taking responsibility for the system, and its parameters. But when circumstances will not allow time for a human to be in the loop, the AI system has to take the decision in machine, not human, time. In many ways that is a shocking prospect. But is there an alternative? I do not think so, given that Russia and China have embarked on programmes to develop and field autonomous weapons systems. I do not see a realistic possibility of an international treaty to ban autonomous weapons systems.

Our objective must be to ensure that, when a machine using AI makes a decision that results in lethal effect, it does so in a way that reflects sound moral reasoning and remains under meaningful human authority, although not individual human decision – and is therefore capable of bearing the accountability and weight of responsibility central to ethical conduct.

This article has been adapted from remarks that were given by the author at the Cheltenham Science Festival on 3 June 2026 in discussion with the neuroscientist Dr Nicholas Wright.