Google shares framework for testing health AI for accuracy, safety and usefulness
Google published a whitepaper with health experts Dr. Elizabeth McGlynn and Dr. Dana Gelb Safran that describes an end-to-end approach for assessing AI tools that address people's health needs. The framework evaluates models on accuracy, safety, and usefulness, and combines closed-question benchmarks, autoraters, and expert human review to test common and high-risk health scenarios. Google says it already uses the framework to check whether its models give accurate, useful responses that promote safe interactions in health contexts. The company says it is sharing the methodology to promote transparency and to support developers, researchers, and clinicians.