Is speech-to

Is speech-to-text AI really reliable?

UC research shows the importance of human review in AI-driven physicians’ documentation

Artificial intelligence in clinical settings is getting more common. If you haven’t yet experienced it at your own doctor’s appointments, you’ve probably seen it on TV.

AI speech-to-text had a moment in the spotlight on the medical drama series, “The Pitt,” just earlier this year.

“Studies show that you can spend 80% less time charting,” says character Baran Al-Hashimi, a doctor showing her peers a new AI speech-to-text tool. But a skeptical colleague raised concerns after the tool misheard the name of a patient’s medication.

While the show is fictional, it poses a real question: Do the risks of these AI tools outweigh the reward?

Elsayed is the founder and leader of the School of IT's Applied Machine Learning and Intelligence Lab. Photo/provided

Nelly Elsayed, associate professor at the University of Cincinnati, wanted to not only answer this question but find specific solutions to advancing safe and reliable AI systems in medical documentation.

Her paper, “Socio-technical risks of clinical speech-to-text systems: Transparency, privacy, and reliability challenges in AI-driven documentation,” was just published in the International Journal of Medical Informatics. The study examined a multitude of existing research, ethical guidelines and government regulations to spot how AI’s adoption is outpacing its oversight.

“These AI tools, the quality is improving, but there are other aspects we have to care about to make them more efficient and transparent,” Elsayed said.

Elsayed specializes in teaching AI-related courses in the UC School of Information Technology, part of the College of Education, Criminal Justice, and Human Services. She is also the founder of the school’s Applied Machine Learning and Intelligence Lab, which she has been leading since 2022.

“My focus is on how to improve the quality of life and social factors to make AI for good, AI that can be utilized to enhance our lives,” Elsayed said.

Five key risks of clinical speech-to-text:

In examining the data, Elsayed found five key risks in today’s clinical speech-to-text tools. These risks span across problems with the AI itself to how healthcare providers handle the software’s data.

Elsayed said that simply having a human review data before it is finalized can reduce a fair amount of these concerns.

“We need to have a human in the loop to check whether the text is exactly what has been spoken,” Elsayed explained. “And that test needs to be done for the entire text, not just for the first couple statements.”

Elsayed also found that there are a multitude of factors in real-life settings that can throw off an AI’s ability to record properly. AI is often trained in an “ideal” setting, with none of the bustle of a real doctor’s office like machines beeping and doctors chatting. Without training large-language models in specific scenarios, for different accents and for speech disorders, they cannot be reliable in real life.

Another way to curb error is to train the clinicians on the software before they adopt it in their practices.

“The organization developing the system needs to give guidelines for the doctor, what they can use, what they cannot use and what to look out for,” she said.

Elsayed wanted to look into AI’s pitfalls after her own visit to a doctor who used speech-to-text to record her visit.

“This paper is close to my heart because it’s actually inspired by my personal experience,” she said.

Her doctor informed her that their conversation would be recorded to generate a transcript. That immediately sparked questions for Elsayed.

“Will it record exactly what I’m saying? Or will some noise like a family member’s voice or a nurse’s conversation outside throw it off?”

On top of that, she considered her own data confidentiality. “I didn’t have any disclosure about what the information is, the security behind it, where it’s going to be stored or who will access it.”

While the study is conceptual, her goal with these findings is to directly improve patient privacy and the overall effectiveness of speech-to-text systems so that the benefits far outweigh any of the risks she detailed.

“As a system, it’s great for doctors because it really removes the burnout and reduces the time of sitting and typing on the computer. They’re giving more face-to-face time to the patient and can listen more,” Elsayed said. “But we need somebody to check whether these texts are accurate to what they said or not.”

The next groundbreaking discovery

UC is a powerhouse of discovery and impact as a Carnegie 1 research institution. From pioneering medical research to transformative engineering and social innovation, our faculty and students drive progress that reaches across the world.

Featured image at top of a doctor taking notes on a laptop. Photo/Adobe Stock

Advancements in technology such as artificial intelligence give cyber criminals more ways to exploit data. The Dayton Daily News looked to University of Cincinnati’s Richard Harknett, PhD, for context on the recent influx of cyberattacks. Harknett is co-director of the Ohio Cyber Range Institute, chair of the Center for Cyber Strategy and Policy and director of UC’s School of Public and International Affairs.

WLWT reported that Ohio is the first state to require artificial intelligence policies in all public K-12 schools. Rebekah Michael, Executive Staff Director of the Ohio Cyber Range Institute, weighed in on AI in schools.

Ro Basty, a second-year doctoral candidate in the University of Cincinnati’s School of Information Technology, is studying the applications and implications of technology in art, a subject in which tastes and sensibilities can be contentious even without the added complications of AI.