It’s easy to get drawn in by flashy uses of AI in healthcare. It’s much harder to figure out which ones are actually good ideas and which ones just sound cool.
“People like to fall in love with AI…and then suddenly [you] realize, wow, it costs a lot of money, and it’s more complex than we thought, and our scope creep is real. And suddenly, your project is failing,” Kathleen Walch, director of AI engagement and learning at Project Management Institute, a professional membership and training organization, told Healthcare Brew.
By some experts’ estimates, about 80% of AI projects fail, which is twice the rate for IT projects not involving AI, according to 2024 research by the nonprofit RAND Corporation. So how do you invest in AI that is worth the cost? It all comes down to considering the details.
“Think big, start small, and iterate often,” Walch said.
Data first, AI later
AI is only as smart as the data it’s trained on. And bad data—without enough quality data points, organization, or governance structure—is a recipe for bad AI, Walch said.
Take AI tools meant to diagnose Covid during the early days of the pandemic. As researchers scrambled to gather data amid a global crisis, much of the data that trained these models was mislabeled or taken from unknown sources, MIT Technology Review reported in 2021.
That same year, a review of 415 models related to Covid found that none were fit for clinical use.
The review found many models, for example, had trained AI to identify Covid by using adult chest scans as examples of Covid, but children’s chest scans as a control group. The resulting models learned to identify children, not Covid.
Don Woodlock, head of global healthcare solutions for data technology provider Intersystems, told Healthcare Brew that a key ingredient for top-notch data is a master patient index: a clean, unified database between all systems or facilities that ensures AI isn’t mixing up patients or working with duplicate IDs.
Navigate the healthcare industry
Healthcare Brew covers pharmaceutical developments, health startups, the latest tech, and how it impacts hospitals and providers to keep administrators and providers informed.
“A lot of customers try and skip [building a good data foundation], where they’ll just start going [in] on an AI project, work with Google or [Amazon Web Services] or OpenAI, and not realize the importance of bringing data to the table to make it work,” Woodlock said.
One small step for chatbots…
Once a strong dataset is in place, then you need to identify exactly what you want to solve and how AI can help, Walch said.
From there, you need to break the issue down into its smallest possible piece.
Take an organization that wants a chatbot for patients. Start by creating one that can answer a single question, she said, such as the most common question patients ask.
Then measure whether the chatbot is making a difference by gathering internal data. If the question is about a drug, is there less misuse of that drug since the chatbot has been implemented, for instance?
If all goes successfully, you can then start addressing other questions one by one, Walch said.
The Watson warning
In contrast, jumping into the deep end can come with consequences, Walch said.
She brought up IBM Watson for Oncology, a product that claimed to use a cloud-based supercomputer to create AI insights that could “revolutionize cancer care,” wrote then-VP Stephen Harvey on IBM’s blog in 2016.
A 2017 Stat investigation found it fell short thanks to problems like insufficient data and “lofty expectations.”. In the years following, Watson for Oncology’s creator, IBM’s Watson Health, lost clients, laid off employees, and in 2022 was “essentially, sold for parts,” Slate reported at the time.
IBM could have taken a different approach, Walch said.
“They could have thought big and said, ‘How can we have AI tackle oncology?’ But then start[ed] small with one specific application,” she said.