Executives put the spotlight on AI’s reliability issue

Businesses’ ability to rely on AI has come under scrutiny and executives are starting to take a more cautious approach to deployment due to rising AI spend, lagging workforce adoption and shifting regulatory policies.

While AI is helping employees derive insights and connect with customers, the technology isn’t necessarily saving time or money, according to enterprises surveyed for an SAP report published in July. Inconsistent pricing models and vendor fragmentation create further cost challenges.

Lack of employee understanding of AI tools is also contributing to the more pragmatic AI approach companies are taking. A Forrester survey found that only 16% of information workers highly understood AI.

However, trust and confidence in AI remains highest with humans in the loop, the HFS Research report found. The report said while having reliable AI doesn’t mean stopping every failure, “it’s defined by how quickly the enterprise can detect, explain and control one.”

A clearly defined human-in-the-loop strategy, outlining accountability and levels of trust, is critical in building reliable AI and managing business disruptions, Daher said.

“You need to train the person, figure out what they’re accountable for,” Daher said.

In addition to having a clear strategy for human accountability and oversight, building trustworthy, reliable AI systems requires evaluating AI based on business outcomes delivered, Daher said. Yet technical metrics including uptime and accuracy are tracked more often than business outcomes, according to the report.

Lastly, as AI becomes more complex, it’s becoming harder for executives to describe how it reached its decisions. Governance will be crucial for AI reliability, Daher said.

“There’s this really big tension around trust that’s happening because we’re being asked to adopt [AI] in every part of our work and lives and we don’t understand how the responses are coming about,” she said.