The Missing Link for Regulating AI Systems in Health

“Artificial intelligence – already a booming sector in health care — will need more than just smart innovation to succeed. It will need a revamp of regulation itself.

That’s because today’s FDA is set up to evaluate products like medicines and devices that do not usually change. By contrast, AI programs continually evolve based on the data guiding them. It won’t be sufficient to evaluate these systems just when they are built. They will need constant oversight, especially on the local level where people receive care, because patient characteristics and even data definitions can shift over time and by location, affecting AI’s results.

AI’s unique ability to learn from new information may also be its greatest weakness, and one that the FDA cannot solve alone. ‘We need to focus on the bedside,’ said LDI Senior Fellow Daniel Herman. ‘That’s not a space the FDA oversees.’

Fortunately, Herman and LDI Senior Fellow Gary Weissman have identified a solution in plain sight. For decades, the Centers for Medicare & Medicaid Services (CMS) has overseen clinical lab testing through a mix of requirements for local expertise, performance verification and monitoring, and external review. That system, called the Clinical Laboratory Improvement Amendments (CLIA), could create the local accountability that AI desperately needs, Herman, Weissman and colleague Jenna Reece write in a recent commentary in npj Digital Medicine.”

Read more at Penn LDI.