Title: Evaluating the Performance of ChatGPT‐4o in Risk of Bias Assessments.
Authors: Kuitunen, Ilari; Ponkilainen, Ville T.; Liukkonen, Rasmus; Nyrhi, Lauri; Pakarinen, Oskari; Vaajala, Matias; Uimonen, Mikko M.
Abstract: The article evaluates the performance of the ChatGPT-4o large language model in risk of bias assessments using Cochrane's RoB 2.0 tool. The study found that ChatGPT-4o had slight agreement in the overall assessment and varied agreement rates in different bias domains. The results suggest that ChatGPT-4o may not be suitable for risk of bias assessments with simple prompts, highlighting the need for further research on improving its performance.
Subjects: LANGUAGE models; MEDICAL periodicals; PDF (Computer file format); DISPUTED authorship; CHATGPT
Publication: Journal of Evidence-Based Medicine, 2024, Vol 17, Issue 4, p700
ISSN: 1756-5383
Publication type: Academic Journal
DOI: 10.1111/jebm.12662

Evaluating the Performance of ChatGPT‐4o in Risk of Bias Assessments.

Kuitunen, Ilari; Ponkilainen, Ville T.; Liukkonen, Rasmus; Nyrhi, Lauri; Pakarinen, Oskari; Vaajala, Matias; Uimonen, Mikko M.