پژوهشنامه پردازش و مدیریت اطلاعات

پژوهشنامه پردازش و مدیریت اطلاعات

تأثیر داوری باز بر رفتار داوران (مرور مفهومی)

نوع مقاله : مقاله پژوهشی

نویسنده
پژوهشگاه علوم و فناوری اطلاعات ایران(ایرانداک) ، تهران، ایران
چکیده
    انتشار مقاله در ارتقای علمی و حرفه‌ای، نسبت به گذشته اهمیت بیشتری یافته و پژوهشگران خواستار شفافیت بیشتری در فرایند داوری مقاله‌ها هستند. از همین رو، نوعی از داوری با نام «داوری باز» پدیدار شد که با اثرپذیری از علم باز، بر ارزش‌هایی چون «آشکاری» و «شفافیت»، استوار است و بارزترین ویژگی آن، هویت آشکار داوران و انتشار گزارش داوری آن‌هاست. این ویژگی‌ها، تأثیرهایی بر رفتار داوران دارند که مطالعة حاضر، با هدف شناسایی آن‌ها صورت گرفته است.
این مطالعه از روش مرور مفهومی و بر پایة چارچوب شش-مرحله‌ای «لواک و همکاران» بهره گرفته است و طی آن، پس از تعیین هدف و پرسش‌های پژوهش، 21 مقاله انتخاب، و داده‌های آن‌ها استخراج، دسته‌بندی و تحلیل شدند.
در این مطالعه رفتارهایی مانند تمایل افراد به داوری؛ کیفیت انجام داوری؛ مدت‌زمان انجام داوری؛ سوگیری نسبت به مسائلی چون جنسیت، کشور، و ...‌؛ قضاوت نهایی (توصیه)؛ نگارش گزارش داوری (لحن و حجم گزارش داوری)؛ و درخواست از نویسندگان برای استناد به آثار خود شناسایی شده‌اند. «کیفیت انجام داوری»، «توصیة داوری»، «سوگیری»، و «زمان انجام داوری»، به‌ترتیب پرتکرارترین رفتارهای مطالعه‌شده بودند و رفتارهای شناسایی‌شده از ویژگی‌های هویت آشکار و انتشار گزارش داوری تأثیر گرفته بودند.
داوری باز، هرچند در افزایش رفتارهای منفی مانند کاهش تمایل به داوری، سوگیری، درخواست داوران برای استناد به آثار، و افزایش زمان انجام داوری اثرگذار بود، اما بر کیفیت داوری، لحن و حجم گزارش داوری، اثر منفی نداشته و گاهی آن‌ها را بهبود داده است.
کلیدواژه‌ها

عنوان مقاله English

The Effect of Open Peer Review on Reviewers’ Behavior: a Scope Review

نویسنده English

Nadia Hadji-Azizi
Iranian Research Institute for Information and Technology (IranDoc); Tehran, Iran
چکیده English

The role of article publishing in academic and professional promotion is unprecedentedly increasing worldwide and researchers demand more transparency in the process of reviewing articles. Emergence of the Open Science movement led to the implementation of a new model of peer review called "Open Peer Review"(OPR) in some journals. OPR is based on the values of "openness" and "transparency". The most obvious feature of OPR is reviewers’ open identity and open report. These characteristics have influenced the reviewers' behavior and the present study was conducted with the aim of identifying them.
This study has been conducted in a scope review method based on the six-step framework by ‘Levace et al’. After determining the purpose and questions of the research, 21 articles were selected, and the data were extracted, categorized and analyzed.
The review revealed that, behaviors such as willingness to review, the review quality; time duration; bias towards issues such as gender, country, etc.; recommendation; the tone and volume of the review report, and requesting authors to cite their earlier works have been identified. “Quality”, “recommendation”, “bias”, and “review time” were the most frequent behaviors studied, respectively. The identified behaviors were influenced by the characteristics of the open identity and open report.
Eventually, OPR has been effective in increasing negative behaviors such as reducing the tendency to review, bias, asking authors for citing their works, and increasing the time of review. However, it had not a negative effect on the review quality, the tone and volume of the review report, and sometimes it has improved them.

کلیدواژه‌ها English

Open Peer Review (OPR)
Reviewer Behavior
Open Identity
Open Review Repor
ارقامی، شیرازه و علیرضا شغلی. 1400. الزامات گذار به دانشگاه نسل سوم: مرور مفهومی بر مطالعات ایران. مرکز مطالعات و توسعه آموزش علوم پزشکی یزد 16 (1): 2-19.
رضائیان، محسن. 1397. مقدمه‌ای بر مرور مفهومی (سخن سردبیر). مجله دانشگاه علوم پزشکی رفسنجان 17 (4): 291-292.
References:
Batagelj, Vladimir, Anuška Frelimo, & Flaminio Squazzoni. 2017. The emergence of a field: a network analysis of research on peer review. Scientometrics 13 (1): 503–532.
Bornmann, Lutz, and Hans-Dieter Daniel. 2010. Do Author-Suggested Reviewers Rate Submissions More Favorably than Editor-Suggested Reviewers? A Study on Atmospheric Chemistry and Physics. PLoS One 5 (10): e13345
Bravo, G., F. Grimaldo, E. López-Iñesta, B. Mehmani, & F. Squazzoni. 2019. The effect of publishing peer review reports on referee behavior in five scholarly journals. Nature Journal 10: 322.
Cawley, Valentine. 2011. An analysis of the ethics of peer review and other traditional academic publishing practices. International Journal of Social Science and Humanity 1 (3): 205–213.
COPE Ethical Guidelines for Peer Reviewers (version 2.). 2017. Retrieved from https://publicationethics.org/resources/guidelines-new/cope-ethical-guidelines-peer-reviewers (accessed November 3, 2019)
European Commission. 2019. Open Science. Retrieved from https://ec.europa.eu/info/research-and-innovation/strategy/goals-research-and-innovation-policy/open-science_en (accessed February 16, 2021)
Falk Delgado, Alberto, Gregory Garretson, & Anna Falk Delgado. 2019. The language of peer review reports on articles published in the BMJ, 2014–2017: an observational study. Scientometrics 120: 1225–1235.
Fontenelle, Leonardo Ferreira and Sarti Thiago Dias. 2021. Attitudes toward open peer review among stakeholders of a scholar-led journal in Brazil. Transinformação 33. Available from https://doi.org/10.1590/2318-0889202133e200072 (accessed May 15, 2022).
Ford, Emily. 2013. Defining and Characterizing Open Peer Review: A Review of the Literature. Journal of Scholarly Publishing 44 (4): 311-326.
_____, Emily. 2015. Open peer review at four STEM journals: an observational overview. F1000Research 4 (6). 10.12688/f1000research.6005.2
Godlee, F, C R. Gale, & C N. Martyn. 1998. Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: a randomized controlled trial. JAMA. American Medical Association (AMA). 280 (3): 237–40. 10.1001/jama.280.3.237
Helmer, Markus, Manuel Schottdorf, Andreas Neef and Demian Battaglia. 2017. Gender Bias in Scholarly Peer Review. eLife (6). DOI: 10.7554/eLife.21718.001
JMIR Publications. 2021. What is open peer-review? Available at: https://support.jmir.org/hc/en-us/articles/115001908868-Whatis-open-peer-review (accessed Sept. 5, 2021)
Justice AC, MK. Cho, MA Winker, JA Berlin, & D. Rennie. 1998. Does masking author identity improve peer review quality? A randomized controlled trial. Journal of the American Medical Association (JAMA):280: 240-242.
Kowalczuk, Maria K., Frank Dudbridge, Shreeya Nanda, Stephanie L. Harriman, Jigisha Patel, Elizabeth C. Moylan. 2015. Retrospective analysis of the quality of reports by author-suggested and non-author-suggested reviewers in journals operating on open or single-blind peer review models. British Medical Journals (BMJ Open) (9): e008707. 10.1136/bmjopen-2015-008707
Kulczycki, Emanuel. 2016. Rethinking Open Science: The Role of Communication. Analele Universității din Craiova. Seria Filosofie 37: 81-97.
Levac, D., H. Colquhoun, & K. K. O'Brien. 2010. Scoping studies: advancing the methodology. Implementation Sci 5 (69): 1-9. https://doi.org/10.1186/1748-5908-5-69
Levis, Alexander, W. Albert, F. G. Leentjens, L. James, Mark Levenson, A. Lumley, D. Brett, D. Thombs. 2015. Comparison of self-citation by peer reviewers in a journal with single-blind peer review versus a journal with open peer review. Journal of Psychosomatic Research 79 (6): 561-565.
Lioyd, Margaret E. 1990. Gender Factors in Reviewer Recommendations for Manuscript Publication. Journal of Applied Behavior Analysis (JABA) 23 (4): 539-543
Manchikanti, Laxmaiah, Alan David Kaye, Mark Boswel, A. Hirsch, & A. Joshua. 2015. Medical Journal Peer Review: Process and Bias. Pain Physician 18: E1-E14.
McNutt RA, AT Evans, RH Fletcher, & SW Fletcher. 1990. The effects of blinding on the quality of peer review. Journal of the American Medical Association (JABA) 263: 1371-1376.
Melero, R. and F. López-Santoveña. 2001. Referees' Attitudes toward Open Peer Review and Electronic Transmission of Papers. Food Science & Technology International 7 (6): 521-527. 
Murray, Dakota, Kyle Siler, Vincent Larivi'ere, Wei Mun Chan, Andrew M. Collings, Jennifer Raymond, & Cassidy R. Sugimoto. 2019. Author-Reviewer Homophily in Peer Review. PLOS Biology. Available at: https://www.biorxiv.org/content/10.1101/400515v3.full (accessed May 15, 2022)
Nobarany, Syavash, and Kellogg S. Booth. 2015. Use of politeness Strategies in Signed open Peer Review. Journal of the Association for Information Science and Technology 66 (5): 1048–1064.
Nwagwu Williams E. and Bosire Onyancha. 2015. 'Back to the Beginning—The Journal Is Dead, Long Live Science. Journal of Academic Librarianship 41 (5): 669–79.
Peebles, Erin, Marissa Scandlyn, & Blair R. Hesp. 2020. A retrospective study investigating requests for self-citation during open peer review in a general medicine journal. Plons One August 20, 2020. https://doi.org/10.1371/journal.pone.0237804
Rath, Manansa and Peiling Wang. 2107. Open peer review in the era of open science: a pilot study of researchers' perceptions. JCDL '17 Proceedings of the 17th ACM/IEEE Joint Conference on Digital Libraries. 317-318 https://ieeexplore.ieee.org/document/7991608 (accessed April 4, 2020).
Ross-Hellauer, Tony. 2017. What is open peer review? A systematic review. F1000Research.6: 588. (https://doi.org/10.12688/f1000research.11369.2)
Ross-Hellauer, Tony, Arvid Deppe, & Birgit Schmidt. 2017. Survey on open peer review: Attitudes and experience amongst editors, authors and reviewers. Plos One. December. https://doi.org/10.1371/journal.pone.0189311
Ross-Hellauer, Tony and Edit Görögh. 2019. Guidelines for open peer review implementation. Research Integrity and Peer Review 4 (4) available at: https://researchintegrityjournal.biomedcentral.com/articles/10.1186/s41073-019-0063-9#Bib1 (accessed Feb. 6, 2019)
Segado-Boj, Francisco, Martín-Quevedo, Juan, Prieto-Gutiérrez, & Juan José. 2018. Attitudes toward Open Access, Open Peer Review, and Altmetrics among Contributors to Spanish Scholarly Journals. Journal of Scholarly Publishing 50 (1): 48-70.
Smith, Richard. 2006. Peer review: a flawed process at the heart of science and journals. J R Soc Med 99: 178-182. doi: 10.1087/20110109
Stamm, T., U. Meyer, H. P. Wiesmann, J. Kleinheinz, M. Cehreli, & C C Zafer. 2007. A retrospective analysis of submissions, acceptance rate, open peer review operations, and prepublication bias of the multidisciplinary open access journal. Head & Face Medicine Head and Face Medicine 3: 27.
Stojanovski, Jadranka. 2017. Is peer review evolving in the open access environment? A survey of Croatian open access journals. No 1 (2017): The 12th Munin Conference on Scholarly Publishing 2017 http://orcid.org/0000-0001-7399-522X
Squazzoni, Flaminio, Giangiacomo Bravo, & Károly Takács. 2013. Does Incentive Provision Increase the Quality of Peer Review? An Experimental Study. Research Policy 42 (1): 287–94.
Teixeira da Silva, J. A. 2019. Challenges to open peer review. Online Information Review 34 (2):197:200. https://doi.org/10.1108/OIR-04-2018-0139.
Thelwall M, L. Allen, E-R Papas, Z Nyakoojo, & V. Weigert. 2021. Does the use of open, non-anonymous peer review in scholarly publishing introduce bias? Evidence from the F1000Research post-publication open peer review publishing model. Journal of Information Science: 47 (6): 809-820.
Thelwall, Mike. 2022. Journal and disciplinary variations in academic open peer review anonymity, outcomes, and length. Journal of Librarianship and Information Science. https://doi.org/10.1177/09610006221079345
Tracz, V., & R. Lawrence. 2016. Towards an open science publishing platform. F1000 Research. 2016 (5): 130. https://doi.org/10.12688/f1000research.7968.1
Van Rooyen S, F. Godlee, S. Evans, R. smith, & N. Black. 1998. Effect of Blinding and Unmasking on the Quality of Peer Review: A Randomized Trial. JAMA 280 (3): 234-237. Available at: https://www.ncbi.nlm.nih.gov/pubmed/9676666 (accessed May 15, 2022)
_____.1999. Effect of Open Peer Review on Quality of Reviews and on 'Reviewers' Recommendations: a randomized tiral. BMJ 318 (7175): 23-7. Available at: https://www.ncbi.nlm.nih.gov/pubmed/9872878 (accessed May 15, 2022)
Van Rooyen S, Delamothe T, & S J. Evans. 2010. Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial. BMJ.2010;341:c5729. 10.1136/bmj.c5729 Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2982798/ (accessed May 15, 2022)
Vercellini, P., L. Buggio, P. Vigano, & E. Somigliana. 2016. Peer review in medical journals: beyond quality of reports towards transparency and public scrutiny of the process. Eur J Intern Med: 15-19 https://www.sciencedirect.com/science/article/pii/S0953620516300838#ab0005 (accessed May 15, 2022)
Vinther Sir, Ole Haagen Nielsen, Jacob Rosenberg, Niels Keiding & Torben V. Schroeder. 2012. Same Review Quality in Open versus Blinded Peer Review in "Ugeskrift for Laeger". Danish Medical Journal 59 (8): 1-5.
Walsh, Elizabeth, M. Rooney, L. Appleby & G. Wilinson. 2000. Open peer review: A randomized controlled trial. British Journal of Psychiatry 176 (1): 47.
Wang, Peiling, Sukjin You, Rath Manasa & Dietmar Wolfram. 2016. Open Peer Review in Scientific Publishing: A Web Mining Study of PeerJ Authors and Reviewers. Journal of Data and Information Science 1 (4): 60–80. DOI: 10.20309/jdis.201625
Wang, P., J. Hoyt, U. Pöschl, D. Wolfram, P. Ingwersen, R. Smith, & M. Bates. 2017. The last frontier in open science: Will open peer review transform scientific and scholarly publishing? Proceedings of the Association for Information Science and Technology, 53: 1-4 https://asistdl.onlinelibrary.wiley.com/doi/pdfdirect/10.1002/pra2.2016.14505301001 (2021-03-09)
Wendler D, & F. Miller. 2014. The ethics of peer review in bioethics. J Med Ethics 40 (10): 697-701. doi: 10.1136/medethics-2013-101364.
Wolfram, Dietmar, Peiling Wang, & Fuad Abuzahra. 2021. An exploration of referees’ comments published in open peer review journals: The characteristics of review language and the association between review scrutiny and citations, Research Evaluation 30 (3): 314–322, https://doi.org/10.1093/reseval/rvab005
 
 

  • تاریخ دریافت 19 آذر 1401
  • تاریخ بازنگری 07 بهمن 1401