New AI risks: 'Cognitive spoofing' and fake expertise

Dr. David Kirk, chief medical officer at Regard

Artificial intelligence has quickly become one of healthcare's biggest strategic priorities, with hospitals and health systems investing heavily in clinical decision support, ambient documentation, revenue cycle automation and other AI-powered technologies.

As organizations focus on accuracy, productivity and return on investment, Dr. David Kirk said, they also need to prepare for a subtler risk: clinicians and patients placing too much confidence in AI simply because it sounds authoritative.

The industry's next major AI challenge is not simply preventing factual errors or hallucinations, explained Kirk, chief medical officer at Regard, which offers an AI-powered clinical insights platform for hospitals and health systems.

Instead, he warned of what he calls "cognitive spoofing" – AI's ability to project the appearance of expertise without possessing the experience, judgment and contextual understanding that clinicians develop over years of practice.

For CIOs, CMIOs and other healthcare executives responsible for AI strategy, Kirk said the implications extend well beyond technology performance. As AI becomes integrated into clinical workflows and increasingly accessible to consumers, organizations must ensure that governance, education and transparency evolve alongside the technology.

Cognitive spoofing presents a significant challenge because polished AI responses can make clinicians appear more knowledgeable than they really are, Kirk said.

"We have been so focused on AI getting facts wrong that we missed the bigger problem: AI helping people fake expertise," Kirk said.

Medical education depends on observing how learners reason through difficult cases. AI-generated assessments can hide uncertainty and knowledge gaps, making it harder for attending physicians to identify where trainees need additional instruction.

Organizations should foster cultures in which clinicians understand AI's strengths and limitations while maintaining responsibility for their own clinical judgment, he said. Senior physicians who openly question AI recommendations demonstrate that thoughtful skepticism is an expected part of safe clinical practice.

Patients are bringing AI into the exam room

Patients increasingly arrive with AI-generated diagnoses, summaries and treatment suggestions after reviewing their records with consumer AI tools – a trend that is both encouraging and challenging, Kirk said.

He said better-informed patients often have richer conversations with clinicians, but AI-generated recommendations can conflict with individualized medical judgment. Rather than dismissing AI-generated information, physicians should discuss it openly and explain why generalized recommendations may not apply to a specific patient.

Additionally, health systems should prepare clinicians for longer, more nuanced conversations as patient use of AI continues to expand, Kirk said.

Governance beyond implementation

Kirk said AI governance should not end after deployment. Organizations should continuously evaluate AI performance as workflows, patient populations and data sources evolve.

He also recommended transparency with patients regarding when AI contributes to care and encourages organizations to incorporate patient feedback into governance decisions.

Although optimistic about AI's future, Kirk said technology should support – not replace – clinical expertise.

Drawing on his experience caring for critically ill patients, Kirk said medicine's greatest challenge is often information overload, rather than diagnostic intelligence. AI's greatest value is surfacing clinically relevant information at the moment clinicians need it.

Properly implemented AI, he said, should empower clinicians to deliver better care while leaving accountability for diagnosis and treatment firmly in human hands.

Follow Bill's health IT coverage on LinkedIn: Bill SiwickiEmail him: [email protected]Healthcare IT News is a HIMSS Media publication.

WATCH NOW: A governance framework can accelerate trusted AI deployment