Davies warns AI agents are amplifying conduct risk for insurers

Davies warns that the growing deployment of autonomous AI agents across underwriting, claims and customer servicing is amplifying conduct risk for insurers in the UK and Ireland. The warning comes in How AI agents are amplifying conduct risk in insurance, the opening report in the Davies AI Impact Series, which examines AI compliance for insurers deploying agentic systems and the evolving expectations of regulators. The report distinguishes AI agents from traditional AI models on the basis that agentic systems make decisions, adapt behaviour and act with a greater degree of independence, creating challenges for governance, oversight and customer protection. The central finding is that there is no regulatory gap. Existing frameworks, including the Financial Conduct Authority (FCA) Consumer Duty, UK and EU GDPR, and the Central Bank of Ireland Consumer Protection Code 2025, already apply in full to AI-driven decision-making. What has changed, according to the report, is the level of demonstrable control and oversight regulators expect firms to evidence. Conduct risk is therefore amplified rather than replaced. Core obligations around fairness, transparency and accountability are intensified by the scale, speed and autonomy of AI systems, with pricing, policy eligibility, claims assessment and fraud detection identified as the areas of greatest exposure. The report sets out three risks insurers are said to be underestimating. Autonomous decision loops can produce unintended outcomes such as overly aggressive pricing strategies or systematic claims denials that deviate from intended policy. Continuous learning can reinforce bias over time where models learn from their own outputs, leading to progressively unfair outcomes. Growing complexity can obscure how decisions are reached, particularly where multiple models or data sources interact, reducing transparency and complicating both internal oversight and external explanation. On accountability, the report identifies the Senior Managers and Certification Regime (SM&CR) as the primary framework in the UK financial services market for assigning and evidencing individual responsibility for AI-related decisions. Firms are expected to demonstrate clear ownership of AI agent decisions at an appropriate seniority level, maintain audit trails capturing inputs, outputs and decision pathways, govern third-party and vendor AI agents to the same standard as internal systems, and provide human oversight that goes beyond passive approval. Regulators in both the UK and Ireland are described as clear that outsourcing AI does not outsource accountability. The EU AI Act, while not directly applicable in the UK, is described as emerging as a de facto benchmark for responsible AI governance, shaping vendor standards, market expectations and cross-border operating models. The risk-based approach of the Act typically places pricing, underwriting and claims decisions in high-risk categories subject to heightened documentation, transparency and human oversight requirements. A recurring theme is that point-in-time compliance is no longer sufficient. Because agentic systems evolve after go-live through data drift, retraining and changing conditions, the report argues compliance must operate as a continuous governance capability spanning the full lifecycle of the agent, from design and deployment through to live operation, retraining and eventual retirement. This approach is positioned as aligning with model risk management practices already familiar to insurers, including Solvency II internal-model governance, actuarial TAS standards, and FCA and Prudential Regulation Authority guidance, as well as with ISO/IEC 42001. The report closes with a 13-point AI compliance readiness checklist covering ownership, auditability, risk classification, human oversight, bias and outcomes testing, data governance, vendor risk, continuous monitoring, employee training, pre-deployment assessments, incident response, customer redress and operational resilience. Five pillars underpin the framework: accountability, transparency, fairness, continuous monitoring and operational resilience. Insurers are also encouraged to maintain a centralised AI system inventory and to embed data protection impact assessments and AI impact assessments into development and change processes. Paul O'Brien, chief AI officer at Davies, says in the report foreword: "Ensuring appropriate oversight, transparency, and control is no longer a theoretical concern, it is a practical necessity." The report concludes that firms able to demonstrate consistent, fair and accountable AI-driven outcomes will be best positioned to unlock the full value of the technology, and that managing AI conduct risk is becoming a competitive differentiator rather than a compliance exercise.