Iranian Journal of Information Processing and Management

Iranian Journal of Information Processing and Management

An Analysis of the Advantages and Ethical Challenges of Large Language Models in Research

Document Type : Original Article

Author
Rahman Sharifzadeh
Assistant Professor; Information and Society Research Department; Iranian Research Institute for Information Science and Technology (IranDoc); Tehran, Iran;
10.22034/jipm.2025.2050658.1901
Abstract
It has not been long since the emergence of large language models, but due to the significant hype surrounding their applications and prospects, especially in the field of research, it seems essential to examine their applications as well as the advantages and ethical challenges of this technology. What applications do large language models have, and what advantages and ethical challenges do these applications bring, particularly in academic work? In this article, we will analyze and discuss the most important advantages and ethical challenges of these language models.
The methodology of this research for extracting advantages and challenges involved reviewing and examining relevant research works identified in the Scopus and Google Scholar databases. We identified numerous articles by searching for terms such as ‘challenges/ advantages of LLMs in research’, ‘risks/ethical concerns of LLMs’, ‘ethics of ChatGPT’, ‘ethics of LLMs’, ‘LLMs and research ethics’, and ‘artificial intelligence and research ethics’. After removing duplicates, the articles were studied based on their relevance to the topic of this research, and the advantages and challenges were extracted. The author then aggregated and categorized the advantages and challenges, followed by an analysis and discussion of each.
Alongside advantages such as overcoming language barriers for non-English-speaking authors, reducing preparation costs, saving time, improving writing quality, facilitating the dissemination of knowledge, and assisting researchers with special conditions, there are significant challenges such as the hallucination, accountability and authorship, plagiarism and copyright issues, bias, security and privacy, safety and potential harm, transparency and explainability, environmental issues, and unemployment and exploitation. The three special research-oriented issues we examined are each important in their own right. Neglecting the issue of hallucination will ultimately have a detrimental effect on the quality and integrity of researchers' work. The issue of accountability and authorship requires heightened attention from publishers and even those engaged in theoretical work on ethics and philosophy of artificial intelligence. Currently, no publisher or association/ committee recognizes LLMs as authors. While this lack of recognition may prevent some problems, it also leads to other issues. The issue of plagiarism and copyright is significant because language models do not perform well in citation; they rarely provide correct direct citations and often do not maintain sufficient distance from the original text in indirect citations. One reason for this may be that this technology was not primarily designed for scientific writing but rather to write in a natural manner, similar to humans.
Large language models undoubtedly have significant impacts on the research domain. One of the most important advantages is overcoming language barriers for publication. This can be viewed as an opportunity for developing countries: non-English speakers who previously wrote less in English are now writing more with the help of generative AI tools. This, in turn, will affect the visibility rates of educational and research institutes in non-English-speaking countries. Alongside these advantages, efforts must be made to minimize the ethical issues associated with this technology. By analyzing these advantages and challenges, and avoiding overly optimistic or pessimistic approaches to this technology, we will discuss that addressing or minimizing the severity of some challenges, particularly those specific to language models in the research domain, requires theoretical, policy, educational, and technical efforts
Keywords
Large Language Models, Artificial Intelligence, Writing, Research, Ethical Challenges
Subjects
فهرست منابع

بهمن‌آبادی، علیرضا. 1402. هوش مصنوعی و کاربردهای آن در فعالیت‌های پژوهشی. علوم و فناوری اطلاعات کشاورزی 6 (2): 33-43.

منتظری، حسام، احمدرضا غفاری، و علی‌اکبر موسوی موحدی. 1402. کاربردهای چت‌جی‌پی‌تی در تحقیقات علمی و ملاحظات اخلاقی استفاده از آن. نشاء علم 13 (2): 148-156.
References:  
Athaluri S. A., S. V. Manthena, V. S. R. K. M. Kesapragada, V. Yarlagadda, T. Dave, and R. T. S. Duddumpudi. 2024. “Exploring the Boundaries of Reality: Investigating the Phenomenon of Artificial Intelligence Hallucination in Scientific Writing Through ChatGPT References,” Cureus, vol. 15, no. 4, p. e37432, Dec. 2023, doi: 10.7759/cureus.37432.
Bahmanabadi, Alireza. 2013. Artificial Intelligence and Its Applications in Research Activities. Agricultural Information Science and Technology 6 (2): 33-43. [IN PERSIAN]
Biswas S.S. 2023. ChatGPT for Research and Publication: A Step-by-Step Guide. Journal of Pediatr Pharmacol Ther 28 (6): 576-584. doi: 10.5863/1551-6776-28.6.576. Epub 2023 Oct 28. PMID: 38130350; PMCID: PMC10731938.
Burger B., D. K. Kanbach, S. Kraus, M. Breier, and V. Corvello. 2023. On the use of AI-based tools like ChatGPT to support management research. European Journal of Innovation Management 26 (7): 233– 241, 2023, doi: 10.1108/EJIM-02-2023-0156.
Chatgpt. 2024. Europe privacy policy. Available at: https://openai.com/policies/privacy-policy/ (accessed Aug. 3, 2025)
Dergaa I., K. Chamari, P. Zmijewski, and H. Ben Saad. 2023. From human writing to artificial intelligence generated text: examining the prospects and potential threats of ChatGPT in academic writing. Biology of Sport, doi: 10.5114/biolsport.2023.125623.
Deroy A., K. Ghosh, and S. Ghosh. 2023. How Ready are Pre-trained Abstractive Models and LLMs for Legal Case Judgement Summarization? arXiv, 2023. doi: 10.48550/arXiv.2306.01248.
de Vries Alex. 2023. The growing energy footprint of artificial intelligence. ? 7 (10) 2191-2194. ISSN 2542-4351, https://doi.org/10.1016/j.joule.2023.09.004.
Elkhatat, AM, K. Elsaid, and S. Almeer. 2023. Evaluating the efficacy of AI content detection tools in differentiating between human and AI-generated text. International Journal of Educational Integrity 19 (1): 1–16. DOI: 10.1007/ s40979-023-00140-5.
Emsley, R. 2023. ChatGPT: these are not hallucinations – they’re fabrications and falsifications. Schizophr 9, 52. https://doi.org/10.1038/s41537-023-00379-4
Ess, Charles. 2019. Digital Media Ethics.?: Polity Press
Farquhar, S., J. Kossen, L. Kuhn and Yarin Gal. 2024. Detecting hallucinations in large language models using semantic entropy. Nature 630: 625–630. https://doi.org/10.1038/s41586-024-07421-0
Floridi, L., and J. W. Sanders. 2002. Mapping the Foundationalist Debate in Computer Ethics. Ethics and Information Technology 4 (1): 1–9. A revised version. is printed in Spinello, R. A. and Tavani, H. T. (eds.), Readings in Cyberethics (2nd ed.). pp. 84–95. Sudbury, MA: Jones and Bartlett.

Gallegos, Isabel O., A. Rossi Ryan, Joe Barrow, Md Mehrab Tanjim, Kim Sungchul, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, Nesreen K. Ahmed. 2024. Bias and Fairness in Large Language Models: A Survey. Computational Linguistics. 50 (3): 1097–1179. doi: https://doi.org/10.1162/coli_a_00524

Gotterbarn D, Keith Miller, Simon Rogerson, Steve Barber, Peter Barnes, Ilene Burnstein, Michael Davis, Amr El-Kadi, N. Ben Fairweather, Milton Fulghum et. al. 2001. Software Engineering Code of Ethics and Professional Practice. Science and Engineering Ethics 7 (2): 231-238.
Huang, J., & M. Tan. 2023. The role of ChatGPT in scientific communication: writing better scientific review articles. American Journal of Cancer Research. 15; 13 (4):1148-1154. PMID: 37168339; PMCID: PMC10164801
Ji, Z. et al.? 2023. Survey of hallucination in natural language generation. ACM Computer Survey. 55; 248: 1-38.
Jiao, J., S. Afroogh, Y. Xu, and C. Phillips. 2024. Navigating LLM Ethics: Advancements, Challenges, and Future Directions. <i>arXiv e-prints</i>, Art. no. arXiv:2406.18841, 2024. doi:10.48550/arXiv.2406.18841.
Kar S.K., T. Bansal, S. Modi, & A. Singh. 2024. How Sensitive Are the Free AI-detector Tools in Detecting AI-generated Texts? A Comparison of Popular AI-detector Tools. Indian Journal of Psychological Medicine. 0 (0). doi:10.1177/02537176241247934
Koh Hyukhun, Kim Dohyung, Lee Lee Lee Minwoo, and Jung Kyomin. 2024. Can LLMs Recognize Toxicity? A Structured Investigation Framework and Toxicity Metric. In Findings of the Association for Computational Linguistics: EMNLP. pages 6092–6114, Miami, Florida, USA: Association for Computational Linguistics.
Koller Daphne, Andrew Beam, Arjun Manrai, Euan Ashley, Xiaoxuan Liu, Judy Gichoya, Chris Holmes, James Zou, Noa Dagan, Tien Y Wong, et al. 2023. Why we support and encourage the use of large language models in NEJM AI submissions. NEJM AI 2024; 1 (1) DOI: 10.1056/AIe2300128 VOL. 1 NO.
Latour B. 1987. Science in action: How to follow scientists and engineers through society. Cambridge MA.: Harvard University Press.
Latour, Bruno. 1994. On Technological Mediation: Philosophy, Psychology, Geneaology. Common Knowledge 94 (4): ?.
Lee Jooyoung, Le Thai, Chen Jinghui, and Lee Dongwon. 2023. Do Language Models Plagiarize? In Proceedings of the ACM Web Conference 2023 (WWW '23). Association for Computing Machinery, New York, NY, USA, 3637–3647. https://doi.org/10.1145/3543507.3583199
Lehr S.A., A. Caliskan, S. Liyanage, & M.R. Banaji. 2024. ChatGPT as Research Scientist: Probing GPT’s capabilities as a Research Librarian, Research Ethicist, Data Generator, and Data Predictor, Proceedings of the Natinal Academi of Science of the U.S.A. 121 (35) e2404328121, https://doi.org/10.1073/pnas.2404328121.

Levy, Matt. 2023. Newsletter 73: Unlocking Communication: The Power of LLMs for Dyslexic Thinkers. Available at: https://www.linkedin.com/pulse/newsletter-73-unlocking-communication-power-llms-dyslexic-matt-ivey/ (accessed Aug. 3, 2025)

Li, P., J. Yang, M. A. Islam, & S. Ren. 2023. Making ai less” thirsty”: Uncovering and addressing the secret water footprint of ai models. arXiv DOI: 10.48550/arXiv.2304.03271.
Logan, S. W. 2024. Generative Artificial Intelligence Users Beware: Ethical Concerns of ChatGPT Use in Publishing Research. Journal of Motor Learning and Development 12 (3): 429-436. Retrieved Nov 24, 2024, from https://doi.org/10.1123/jmld.2024-0024 (accessed Aug. 3, 2025)
Lund, B. D., & T. Wang. 2023. Chatting about ChatGPT: How may AI and GPT impact academia and libraries? Library Hi Tech News., Available at SSRN: https://ssrn.com/abstract=4333415 (accessed Aug. 3, 2025)
Macdonald C., D. Adeloye, A. Sheikh, and I. Rudan. 2023. Can ChatGPT draft a research article? An example of population-level vaccine effectiveness analysis. Journal of Global Health 13. Scotland, p. 1003, Feb. 2023, doi: 10.7189/jogh.13.01003.
Malmstrom, H., C. Stohr, W. & Ou. 2023. Chatbots and other AI for learning: A survey of use and views among university students in Sweden [Application/pdf]. Chalmers University of Technology. https://doi.org/10.17196/CLS.CSCLHE/2023/01 (Accessed 8/3/2025)
Maner, W. 1996. Unique ethical problems in information technology. Science and Engineering Ethics 2 (2): pp.137-154.
Meyer, J.G., R.J. Urbanowicz, P.C.N. Martin, K. O'Connor, R. Li, P. C. Peng, T. J. Bright, N. Tatonetti, K. J. Won, G. Gonzalez-Hernandez, J. H. Moore. 2023. ChatGPT and large language models in academia: opportunities and challenges. Bio Data Mining13; 16 (1): 20.
Montazeri, Hesam, Ahmadreza Ghaffari, and Aliakbar Mousavi Movahedi. 2013. Applications of ChatGPT in Scientific Research and Ethical Considerations of Its Use. Nesha-e- Elm (Scientific Cultivation) 13 (2): 148-156. [IN PERSIAN]
Nature. 2023. Artificial Intelligence (AI). https://www.nature.com/nature-portfolio/editorial-policies/ai (Accessed 8/3/2025)
Neumeister Larry. 2023. Lawyers submitted bogus case law created by ChatGPT. A judge fined them $5,000. Available at: https://apnews.com/article/artificial-intelligence-chatgpt-fake-case-lawyers-d6ae9fa79d0542db9e1455397aef381c (accessed Aug. 3, 2025)

Picazo-Sanchez, P., & L. Ortiz-Martin. 2024. Analysing the impact of ChatGPT in research. Appl Intell 54, 4172–4188 (2024). https://doi.org/10.1007/s10489-024-05298-0 (accessed Aug. 3, 2025)

Pressman, S.M., S., Borna, C. A. Gomez-Cabello, S. A. Haider, C. Haider, A. J. Forte. 2024. AI and Ethics: A Systematic Review of the Ethical Considerations of Large Language Model Use in Surgery Research. Healthcare 2024, 12, 825. https:// doi.org/10.3390/healthcare12080825 (accessed Aug. 3, 2025)

 

Rahman , M., H. J. R. Terano, N. Rahman, A. Salamzadeh, & S. Rahaman. 2023. ChatGPT and Academic Research: A Review and Recommendations Based on Practical Examples. Journal of Education, Management and Development Studies 3 (1): 1-12. doi: 10.52631/jemds.v3i1.175, Available at SSRN: https://ssrn.com/abstract=4407462

Rane, Nitin & Choudhary Saurabh, Tawde Abhijeet, & Rane Jayesh. 2023. ChatGPT is not capable of serving as an author: ethical concerns and challenges of large language models in education. International Research Journal of Modernization in Engineering Technology and Science 5: 851-874. 10.56726/IRJMETS45212.
Ren, S., B. Tomlinson, R. Black, et al.? 2024. Reconciling the contrasting narratives on the environmental impact of large language models. Sci Rep 14, 26310. https://doi.org/10.1038/s41598-024-76682-6
Resnik, David B., and Mohammad Hosseini. 2024. The ethics of using artificial intelligence in scientific research: new guidance needed for a new tool. AI and Ethics https. doi.org/10.1007/s43681-024-00493-8

Rillig Matthias C., Marlene Ågerstrand, Mohan Bi, Kenneth A. Gould, and Uli Sauerland. Risks and Benefits of Large Language Models for the Environment. Environmental Science & Technology 57 (9): 3464-3466. DOI: 10.1021/acs.est.3c01106

Salvagno, M., Chat, GPT, Taccone, F.S., Gerli, A.G. 2023a. Can artificial intelligence help for scientific writing? Crit. Care 27, 75. https://doi.org/10.1186/ s13054-023-04380-2
Salvagno, M., Taccone, F.S., Gerli, A.G. 2023b. Can artificial intelligence help for scientific writing? Crit. Care 27, 99. https://doi.org/10.1186/s13054-023-04390-0
Schlagwein, D., & L. Willcocks. 2023. ChatGPT et al.: The ethics of using (generative) artificial intelligence in research and science. Journal of Information Technology 38 (3): 232-238.
Shamoo Adil E. and David B. Resnik. 2009. Responsible Conduct of Research. Second edition. Oxford: Oxford University Press.
Sharifzadeh, R. 2024. ChatGPT as Co-Author? AI and Research Ethics. ETHICS IN PROGRESS 15 (1): 155-173
Tan et al. ? 2024. Rethinking Literary Plagiarism in LLMs through the Lens of Copyright Laws. Proceedings of Machine Learning Research 260. Available at: https://openreview.net/pdf?id=sWZy2Xirwt (accessed Aug. 3, 2025).
Vidhya, N. G., D. A. N. Devi, & T. Manju. 2023. Prognosis of exploration on Chat GPT with artificial intelligence ethics. Brazilian Journal of Science 2 (9): 60–69. https://doi.org/10.14295/bjs.v2i9.372
Watkins, R. 2024. Guidance for researchers and peer-reviewers on the ethical use of Large Language Models (LLMs) in scientific research workflows. AI Ethics 4, 969–974. https://doi.org/10.1007/s43681-023-00294-5
 
Weber-Wulff, D., A. Anohina-Naumeca, S. Bjelobaba,Tomáš Foltýnek, Jean Guerrero-Dib, Olumide Popoola, Petr Šigut & Lorna Waddington. 2023. Testing of detection tools for AI-generated text. Int J Educ Integr 19, 26. https://doi.org/10.1007/s40979-023-00146-z
Weiser, B. 2023. Lawyer who used ChatGPT faces penalty for made up citations. The New York Times (8 Jun 2023).
Wen Jiaxin, Pei Ke, Hao Sun, Zhexin Zhang, Chengfei Li, Jinfeng Bai, and Minlie Huang. 2023. Unveiling the Implicit Toxicity in Large Language Models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1322–1338, Singapore. Association for Computational Linguistics.
Xames, M. D., & J. Shefa. 2023. ChatGPT for research and publication: Opportunities and challenges. Journal of Applied Learning and Teaching 6 (1). https://doi.org/10.37074/jalt.2023.6.1.20, Available at SSRN: https://ssrn.com/abstract=4381803 or http://dx.doi.org/10.2139/ssrn.4381803
Xu, Z., S. Jain, & M. S. Kankanhalli. 2024. Hallucination is Inevitable: An Innate Limitation of Large Language Models. ArXiv, abs/2401.11817.

Yao Yifan, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, Yue Zhang. 2024. A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly. High-Confidence Computing 4 (2): 1-21.

Zhou, J., H. Müller, A. Holzinger, & F. Chen. 2024. Ethical ChatGPT: Concerns, Challenges, and Commandments. Electronics 13 (17): 3417. https://doi.org/10.3390/electronics13173417
Zhuo Terry Yue, Yujin Huang, Chunyang Chen, Zhenchang Xing. 2023. Exploring AI Ethics of ChatGPT: A Diagnostic Analysis. Computation and Language. Online Pub Date: 2023-01-30, DOI: arxiv-2301.12867
Zohery, Medhat. 2023. ChatGPT in Academic Writing and Publishing: A Comprehensive Guide. In book: Artificial Intelligence in Academia, Research and Science: ChatGPT as a Case Study. (Frist Edition). New York: Novabret Publishing.
Iranian Journal of Information Processing and Management
Volume 40, Issue 4 - Serial Number 124
Summer 2025
Pages 1219-1249

  • Receive Date 13 January 2025
  • Revise Date 09 April 2025
  • Accept Date 19 April 2025