The increasing volume of research submissions to academic journals poses a significant challenge for traditional peer-review processes. To address this issue, this study explores the potential of employing ChatGPT, an advanced large language model (LLM), developed by OpenAI, as an artificial intelligence (AI) reviewer for academic journals. By leveraging the vast knowledge and natural language processing capabilities of ChatGPT, we hypothesize it may be possible to enhance the efficiency, consistency, and quality of the peer-review process. This research investigated key aspects of integrating ChatGPT into the journal review workflow. We compared the critical analysis of ChatGPT, acting as an AI reviewer, to human reviews for a single published article. Our methodological framework involved subjecting ChatGPT to an intricate examination, wherein its evaluative acumen was juxtaposed against human-authored reviews of a singular published article. As this is a feasibility study, one article was reviewed, which was a case report on scurvy. The entire article was used as an input into ChatGPT and commanded it to "Please perform a review of the following article and give points for revision." Since this was a case report with a limited word count the entire article could fit in one chat box. The output by ChatGPT was then compared with the comments by human reviewers. Key performance metrics, including precision and overall agreement, were judiciously and subjectively measured to portray the efficacy of ChatGPT as an AI reviewer in comparison to its human counterparts. The outcomes of this rigorous analysis unveiled compelling evidence regarding ChatGPT's performance as an AI reviewer. We demonstrated that ChatGPT's critical analyses aligned with those of human reviewers, as evidenced by the inter-rater agreement. Notably, ChatGPT exhibited commendable capability in identifying methodological flaws, articulating insightful feedback on theoretical frameworks, and gauging the overall contribution of the articles to their respective fields. While the integration of ChatGPT showcased immense promise, certain challenges and caveats surfaced. For example, ambiguities might present with complex research articles, leading to nuanced discrepancies between AI and human reviews. Also figures and images cannot be reviewed by ChatGPT. Lengthy articles need to be reviewed in parts by ChatGPT as the entire article will not fit in one chat/response. The benefits consist of reduction in time needed by journals to review the articles submitted to them, as well as an AI assistant to give a different perspective about the research papers other than the human reviewers. In conclusion, this research contributes a groundbreaking foundation for incorporating ChatGPT into the pantheon of journal reviewers. The delineated guidelines distill key insights into operationalizing ChatGPT as a proficient reviewer within academic journal frameworks, paving the way for a more efficient and insightful review process.

Som Biswas

Dushyant Dobaria

Harris L. Cohen

The Yale Journal of Biology and Medicine

耶鲁大学生物学和医学杂志 (YJBM) 是一本由研究生和医学生创办、同行评议、开放获取的期刊，致力于发表原创研究文章、科学评论、医学史文章、个人医学观点、政策与生物医学问题相关的分析、病例报告和专题讨论会。 YJBM 每季度出版一次，旨在发表医生和科学家都感兴趣的文章。 YJBM 是并且一直是国际发行的期刊，具有悠久的标志性文章历史。我们的投稿人包括著名的哲学家、政治家、科学家和医生，包括 ERNST CASSIRER、HARVEY CUSHING、RENE DUBOS、EDWARD KENNEDY、DONALD SELDIN 和 JACK STROMINGER。我们的编辑委员会由耶鲁大学医学院和耶鲁大学艺术与科学研究生院的学生和教职员工组成。提交给 YJBM 的所有手稿都首先根据科学质量、原创性、适当性、对该领域的贡献和风格进行评估。然后，合适的手稿将接受严格、公平和快速的同行评审。

The Yale Journal of Biology and Medicine (YJBM) is a peer-reviewed, open-access journal founded by graduate and medical students. It is dedicated to publishing original research articles, scientific reviews, articles on the history of medicine, personal medical perspectives, policy analyses, and discussions related to biomedical issues. YJBM is published quarterly and aims to present articles of interest to both physicians and scientists. The journal has a long-standing history of iconic publications and is distributed internationally. Our contributors include renowned philosophers, politicians, scientists, and physicians such as ERNST CASSIRER, HARVEY CUSHING, RENE DUBOS, EDWARD KENNEDY, DONALD SELDIN, and JACK STROMINGER. The editorial board consists of students and faculty from the Yale School of Medicine and the Yale Graduate School of Arts and Sciences. All manuscripts submitted to YJBM are initially evaluated based on scientific quality, originality, relevance, contribution to the field, and style. Suitable manuscripts undergo rigorous, fair, and rapid peer review.

耶鲁大学生物学和医学杂志 (Yale Journal of Biology and Medicine, YJBM) 是一本由研究生和医学生创办的同行评审、开放获取的期刊，致力于发表原创研究文章、科学评论、医学史文章、个人医学观点、政策分析以及与生物医学相关的问题。YJBM 每季度出版一次，旨在发表医生和科学家都感兴趣的文章。该期刊具有国际发行的特点，并且拥有悠久的标志性文章历史。我们的投稿人包括著名的哲学家、政治家、科学家和医生，如 ERNST CASSIRER、HARVEY CUSHING、RENE DUBOS、EDWARD KENNEDY、DONALD SELDIN 和 JACK STROMINGER。编辑委员会由耶鲁大学医学院和耶鲁大学艺术与科学研究生院的学生和教职员工组成。所有提交给 YJBM 的手稿都将首先根据科学质量、原创性、适当性、对该领域的贡献和风格进行评估。符合要求的手稿将接受严格、公平和快速的同行评审。

Category	Quartile
工程技术	3
工程技术, 生物学	3

ChatGPT and the Future of Journal Reviews: A Feasibility Study

ivySCI AI Smartly Parses PDF, Answers Researchers' Questions, and Helps You Understand Papers in Seconds

Journal Info