Use of Artificial Intelligence in Peer Review and Research Evaluations: A Bibliometrix Based Analysis

In this study authors have explored how artificial intelligence (AI) is being employed in peer review and research evaluation, focusing on trends, techniques, and conceptual development within this emerging domain.

The study analysed 970 records retrieved from the Scopus database using the Bibliometrics package. It has included various approaches like co-citation analyses, co-occurrence network analysis, factorial analysis, historiographic mapping, and trend topic evaluation. Authors also performed a semantic analysis of the titles and abstracts to find mentions of important AI methodologies and data sources.

The results indicate the growing AI applications in academic publishing, mostly through techniques like machine learning, natural language processing, and data mining. Machine learning, data mining and deep learning were found to be prominent AI methods mentioned in papers. Other well-known methods, like sentiment analysis and topic modelling that could also be useful in the peer review process but aren’t used in the literature right now. Most articles have mentioned the use of structured data sources like Scopus, PubMed, or electronic health records. The historiograph and PageRank analysis uncovered important publications and authors that have shaped the field.

This report is the first full bibliometric study that looks only at how AI is utilised in peer review and research evaluation. Authors have addressed a critical gap in knowledge through this consolidated mapping of the intellectual landscape and thematic evolution of the selected theme.

The results will benefit academicians and researchers in the area to understand what AI can and can’t do in scholarly workflows right now. The paper also talks about new ways to use AI responsibly and creatively to evaluate research, such as strategies that haven’t been used much before, including topic modelling and reinforcement learning.