Artificial intelligence changes the way radiologists read scans

Published in Radiology: Artificial Intelligence, a Radiology Society of North America (RSNA) Journal, the Melbourne-based study has found that the type of UI displaying the AI output affects the accuracy of radiologists detecting lung nodules and masses on chest x-rays.

Senior author, Associate Professor Elaine Lui, Director of Research at the RMH Department of Medical Imaging and Head of Research at the UoM Department of Radiology and first author Dr Jennifer Tang said “interestingly, we found a discrepancy between radiologist UI preferences and diagnostic performance.”

In the study, 10 radiologists reviewed 140 chest radiographs, with either no AI or one of three UI outputs: text-only, combined AI confidence score and text, or combined text, AI confidence score, and image overlay. The only UI that improved the diagnostic accuracy compared to no AI was text only, although 80% of the radiologists most preferred the combined text, score and overlay UI.

“These findings are igniting interest in the medical imaging AI research community to move beyond simply evaluating the diagnostic performance of AI software to a new frontier of exploring the complex interactions between the human user and AI UIs and their effects on overall diagnostic performance,” said A/Prof Lui.

“The discrepancy between user preference and diagnostic performance also warns against designing UIs simply based on user preference,” said Dr Tang. 

“It’s not just about what the AI machine outputs, but how we human doctors make best use of the machine output to get the best outcomes for our human patients,” A/Prof Lui added.