AI still has a long ways to go.
A recent study suggests that it might be premature to rely exclusively on machine learning for medical advice. According to a report led by Dr. Andrei Brateanu from the Cleveland Clinic Foundation, human physicians significantly outperformed the latest AI algorithms when tested head-to-head.
Dr. Brateanu emphasized that while AI tools have their place in healthcare, they should not be viewed as replacements for medical professionals. Instead, they should be seen as supplementary resources that, when combined with human expertise, can improve the overall quality of healthcare information.
The study compared the accuracy of two AI programs, ChatGPT-4 and Google Bard, in answering 56 common medical questions. Two physicians then reviewed the responses for accuracy, with a third physician brought in to resolve any disagreements.
While both AI programs provided responses, Bard demonstrated slightly higher accuracy than ChatGPT-4. However, overall accuracy levels for both programs were relatively low. Bard’s responses were found to be accurate 53.6% of the time, while ChatGPT-4’s accuracy rate was only 28.6%. Additionally, both AI programs exhibited inaccuracies or omitted crucial information in their responses.
The study’s findings, published in the American Journal of Preventive Medicine, underscore the ongoing challenge of developing AI tools capable of consistently providing dependable medical advice. Dr. Brateanu cautioned that while AI-generated information may appear convincing, it can occasionally be inaccurate.
As a recommendation, Dr. Brateanu advised individuals to consult AI for medical information but to always verify it with a healthcare professional. He emphasized the importance of maintaining the human element in healthcare, even as AI technology becomes more integrated into medical practice.
Discussion about this post