We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Evaluation of the accuracy and readability of ChatGPT-4 and Google Gemini in providing information on retinal detachment: a multicenter expert comparative study.
- Authors
Strzalkowski, Piotr; Strzalkowska, Alicja; Chhablani, Jay; Pfau, Kristina; Errera, Marie-Hélène; Roth, Mathias; Schaub, Friederike; Bechrakis, Nikolaos E.; Hoerauf, Hans; Reiter, Constantin; Schuster, Alexander K.; Geerling, Gerd; Guthoff, Rainer
- Abstract
Background: Large language models (LLMs) such as ChatGPT-4 and Google Gemini show potential for patient health education, but concerns about their accuracy require careful evaluation. This study evaluates the readability and accuracy of ChatGPT-4 and Google Gemini in answering questions about retinal detachment. Methods: Comparative study analyzing responses from ChatGPT-4 and Google Gemini to 13 retinal detachment questions, categorized by difficulty levels (D1, D2, D3). Masked responses were reviewed by ten vitreoretinal specialists and rated on correctness, errors, thematic accuracy, coherence, and overall quality grading. Analysis included Flesch Readability Ease Score, word and sentence counts. Results: Both Artificial Intelligence tools required college-level understanding for all difficulty levels. Google Gemini was easier to understand (p = 0.03), while ChatGPT-4 provided more correct answers for the more difficult questions (p = 0.0005) with fewer serious errors. ChatGPT-4 scored highest on most challenging questions, showing superior thematic accuracy (p = 0.003). ChatGPT-4 outperformed Google Gemini in 8 of 13 questions, with higher overall quality grades in the easiest (p = 0.03) and hardest levels (p = 0.0002), showing a lower grade as question difficulty increased. Conclusions: ChatGPT-4 and Google Gemini effectively address queries about retinal detachment, offering mostly accurate answers with few critical errors, though patients require higher education for comprehension. The implementation of AI tools may contribute to improving medical care by providing accurate and relevant healthcare information quickly.
- Subjects
GEMINI (Chatbot); LANGUAGE models; CHATGPT; ARTIFICIAL intelligence; RETINAL detachment
- Publication
International Journal of Retina & Vitreous, 2024, Vol 10, Issue 1, p1
- ISSN
2056-9920
- Publication type
Article
- DOI
10.1186/s40942-024-00579-9