Excerpt
“While large language models (LLMs) are not intended to be used for clinical decision-making, evidence suggests that health care providers are adopting them for this purpose at a growing rate. If marketed this way, these AI tools would likely meet the criteria for regulation as medical devices by the U.S. Food and Drug Administration (FDA). However, current regulatory frameworks do not adequately address this emerging use case.
A first-of-its-kind study from Penn LDI Senior Fellow and Assistant Professor of Medicine Gary Weissman simulates how two LLMs might be used by a clinician and demonstrates that they provide device-like clinical decision support, even when directed not to do so. It happened most often in simulated emergency situations, suggesting a possible role for assisting non-clinical bystanders.”
Read more at Penn LDI.