پژوهشنامه پردازش و مدیریت اطلاعات

پژوهشنامه پردازش و مدیریت اطلاعات

Advancing Natural Language Processing with New Models and Applications in 2025

نوع مقاله : مقاله پژوهشی

نویسندگان
Sura Sabah 1
Haider Hadi Abbas 2
Kudaiberdieva Gulmira Karimovna 3
Nahlah M. A. D. Najm 4
Ammar Abdulkhaleq Ali 5
1 Al-Turath University, Baghdad 10013, Iraq
2 Al-Mansour University College, Baghdad 10067, Iraq
3 Osh State University, Osh City 723500, Kyrgyzstan
4 Al-Rafidain University College Baghdad 10064, Iraq
5 Madenat Alelem University College, Baghdad 10006, Iraq
10.22034/jipm.2025.728102
چکیده
Background: Recent advancements in Natural Language Processing (NLP) have been significantly influenced by transformer models. However, challenges related to scalability, discrepancies between pretraining and finetuning, and suboptimal performance on tasks with diverse and limited data remain. The integration of Reinforcement Learning (RL) with transformers has emerged as a promising approach to address these limitations.
Objective: This article aims to evaluate the performance of a transformer-based NLP model integrated with RL across multiple tasks, including translation, sentiment analysis, and text summarization. Additionally, the study seeks to assess the model's efficiency in real-time operations and its fairness.
Methods: The hybrid model's effectiveness was evaluated using task-oriented metrics such as BLEU, F1, and ROUGE scores across various task difficulties, dataset sizes, and demographic samples. Fairness was measured based on demographic parity and equalized odds. Scalability and real-time performance were assessed using accuracy and latency metrics.
Results: The hybrid model consistently outperformed the baseline transformer across all evaluated tasks, demonstrating higher accuracy, lower error rates, and improved fairness. It also exhibited robust scalability and significant reductions in latency, enhancing its suitability for real-time applications.
Conclusion: This article illustrates that the proposed hybrid model effectively addresses issues related to scale, diversity, and fairness in NLP. Its flexibility and efficacy make it a valuable tool for a wide range of linguistic and practical applications. Future research should focus on improving time complexity and exploring the use of deep unsupervised learning for low-resource languages.
کلیدواژه‌ها
Keywords: Natural Language Processing (NLP)
transformer models
hybrid NLP systems
reinforcement learning
machine translation (MT)
sentiment analysis
multilingual data
AI applications
bias mitigation
ethical NLP

عنوان مقاله English

Advancing Natural Language Processing with New Models and Applications in 2025

نویسندگان English

Sura Sabah 1
Haider Hadi Abbas 2
Kudaiberdieva Gulmira Karimovna 3
Nahlah M. A. D. Najm 4
Ammar Abdulkhaleq Ali 5
1 Al-Turath University, Baghdad 10013, Iraq
2 Al-Mansour University College, Baghdad 10067, Iraq
3 Osh State University, Osh City 723500, Kyrgyzstan
4 Al-Rafidain University College Baghdad 10064, Iraq
5 Madenat Alelem University College, Baghdad 10006, Iraq
چکیده English

ABSTRACT
Background: Recent advancements in Natural Language Processing (NLP) have been significantly influenced by transformer models. However, challenges related to scalability, discrepancies between pretraining and finetuning, and suboptimal performance on tasks with diverse and limited data remain. The integration of Reinforcement Learning (RL) with transformers has emerged as a promising approach to address these limitations.
Objective: This article aims to evaluate the performance of a transformer-based NLP model integrated with RL across multiple tasks, including translation, sentiment analysis, and text summarization. Additionally, the study seeks to assess the model's efficiency in real-time operations and its fairness.
Methods: The hybrid model's effectiveness was evaluated using task-oriented metrics such as BLEU, F1, and ROUGE scores across various task difficulties, dataset sizes, and demographic samples. Fairness was measured based on demographic parity and equalized odds. Scalability and real-time performance were assessed using accuracy and latency metrics.
Results: The hybrid model consistently outperformed the baseline transformer across all evaluated tasks, demonstrating higher accuracy, lower error rates, and improved fairness. It also exhibited robust scalability and significant reductions in latency, enhancing its suitability for real-time applications.
Conclusion: This article illustrates that the proposed hybrid model effectively addresses issues related to scale, diversity, and fairness in NLP. Its flexibility and efficacy make it a valuable tool for a wide range of linguistic and practical applications. Future research should focus on improving time complexity and exploring the use of deep unsupervised learning for low-resource languages.

کلیدواژه‌ها English

Natural Language Processing (NLP)
transformer models
hybrid NLP systems
reinforcement learning
machine translation (MT)
sentiment analysis
multilingual data
AI applications
bias mitigation
ethical NLP

References

Alnuaemy, L. M. (2023). Peculiarities of using neuro-linguistic programming for the rehabilitation of servicemen who were in armed conflicts.  Development of Transport Management and Management Methods, 3 (84), 40-55. https://doi.org/:10.31375/2226-1915-2023-3-40-55
Amer, S., Lee, M., and Smith, P. (2023). Cross-lingual Classification of Crisis-related Tweets Using Machine Translation. https://doi.org/:10.26615/978-954-452-092-2_003
Baliyan, A., Batra, A., and Singh, S. P. (2021). Multilingual Sentiment Analysis using RNN-LSTM and Neural Machine Translation.  8th International Conference on Computing for Sustainable Global Development (INDIACom), 710-713.
Cheng, L., Ge, S., and Liu, H. (2022). Toward understanding bias correlations for mitigation in NLP.  arXiv preprint. 2205.12391. https://doi.org/:10.48550/arXiv.2205.12391
Czarnowska, P., Vyas, Y., and Shah, K. (2021). Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics.  Transactions of the Association for Computational Linguistics, 9, 1249-1267. https://doi.org/:10.1162/tacl_a_00425
Futrell, R. (2023). Validity, Reliability, and Significance: Empirical Methods for NLP and Data Science. Computational Linguistics, 49 (1), 249-251. https://doi.org/:10.1162/coli_r_00467
Hashim, N., Mohsim, A., Rafeeq, R., and Pyliavskyi, V. (2019a). New approach to the construction of multimedia test signals.  International Journal of Advanced Trends in Computer Science and Engineering, 8 (6), 3423-3429. https://doi.org/:10.30534/ijatcse/2019/117862019
Hashim, N., Mohsim, A. H., Rafeeq, R. M., and Pyliavskyi, V. (2019b). New approach to the construction of multimedia test signals.  International Journal of Advanced Trends in Computer Science and Engineering, 8 (6), 3423-3429. https://doi.org/:10.30534/ijatcse/2019/117862019
Jawad, A. M., Qasim, N. H., and Pyliavskyi, V. (2022). Comparison of Metamerism Estimates in Video Paths using CAM's Models. IEEE 9th International Conference on Problems of Infocommunications, Science and Technology (PIC S&T), 10-12 Oct. https://doi.org/:10.1109/PICST57299.2022.10238685
Khan, J., Ahmad, N., Khalid, S., Ali, F., and Lee, Y. (2023). Sentiment and Context-Aware Hybrid DNN With Attention for Text Sentiment Classification.  IEEE Access, 11, 28162-28179. https://doi.org/:10.1109/ACCESS.2023.3259107
Krishna, G. G. (2023). Reinforcement Learning based NLP.  International Journal of Soft Computing and Engineering, 13 (4). https://doi.org/:10.35940/ijsce.j0476.0913423
Li, W., Luo, H., Lin, Z., Zhang, C., Lu, Z., and Ye, D. (2023). A survey on transformers in reinforcement learning. arXiv preprint, 2301.03044. https://doi.org/:10.48550/arXiv.2301.03044
Maurya, K. K., and Desarkar, M. S. (2022). Meta-X $ _ {NLG} $: A Meta-Learning Approach Based on Language Clustering for Zero-Shot Cross-Lingual Transfer and Generation.  arXiv preprint, 2203.10250. https://doi.org/:10.48550/arXiv.2203.10250
Moon, W., Kim, T., Park, B., and Har, D. (2023). Enhanced Transformer Architecture for Natural Language Processing. arXiv preprint, 2310.10930. https://doi.org/:10.48550/arXiv.2310.10930
Nameer, Q., Aqeel, J., and Muthana, M. (2023). The Usages of Cybersecurity in Marine Communications.  Transport Development, 3 (18). https://doi.org/:10.33082/td.2023.3-18.05
Nguyen, T., Nguyen, L., Tran, P., and Nguyen, H. (2021). Improving Transformer-Based Neural Machine Translation with Prior Alignments. Complexity, 2021 (1), 5515407. https://doi.org/:10.1155/2021/5515407
Rahim, F., Bodnar, N., Qasim, N. H., Jawad, A. M., and Ahmed, O. S. (2023). Integrating Machine Learning in Environmental DNA Metabarcoding for Improved Biodiversity Assessment: A Review and Analysis of Recent Studies. Research Square. https://doi.org/:10.21203/rs.3.rs-2823060/v1
Roit, P., Ferret, J., Shani, L., Aharoni, R., Cideron, G., Dadashi, R., Geist, M., et al. (2023). Factually consistent summarization via reinforcement learning with textual entailment feedback.  arXiv preprint, 2306.00186. https://doi.org/:10.48550/arXiv.2306.00186
Singh, S., and Mahmood, A. (2021). The NLP Cookbook: Modern Recipes for Transformer Based Deep Learning Architectures. IEEE Access, 9, 68675-68702. https://doi.org/:10.1109/ACCESS.2021.3077350
Sivamayil, K., Rajasekar, E., Aljafari, B., Nikolovski, S., Vairavasundaram, S., and Vairavasundaram, I. (2023). A Systematic Study on Reinforcement Learning Based Applications. Energies, 16 (3). https://doi.org/:10.3390/en16031512
Somers, R., Cunningham-Nelson, S., and Boles, W. (2021). Applying natural language processing to automatically assess student conceptual understanding from textual responses.  Australasian Journal of Educational Technology, 37 (5), 98-115. https://doi.org/:10.14742/ajet.7121
Sunna Torge, A. P., Christoph Lehmann, Bochra Saffar, and Ziyan Tao. (2023). Named Entity Recognition for Low-Resource Languages - Profiting from Language Families. In Proceedings of the 9th Workshop on Slavic Natural Language Processing (SlavicNLP 2023), 1–10. https://doi.org/:10.18653/v1/2023.bsnlp-1.1
Tan, K. L., Lee, C. P., Lim, K. M., and Anbananthen, K. S. M. (2022). Sentiment Analysis With Ensemble Hybrid Deep Learning Model. IEEE Access, 10, 103694-103704. https://doi.org/:10.1109/ACCESS.2022.3210182
Tariq, A., and Ahmed, A. (2022). Deep Learning in Sentiment Analysis: Recent Architectures.  ACM Comput. Surv, 55 (8), Article 159. https://doi.org/:10.1145/3548772
Tushar Agarwal, J. J., Gaurav Kumar. (2023). Transformer and Natural language processing; A recent development.  Tuijin Jishu/Journal of Propulsion Technology, 44 (1). https://doi.org/:10.52783/tjjpt.v44.i1.2225
Villarrubia-Martin, E. A., Rodriguez-Benitez, L., Jimenez-Linares, L., Muñoz-Valero, D., and Liu, J. (2023). A Hybrid Online Off-Policy Reinforcement Learning Agent Framework Supported by Transformers.  International Journal of Neural Systems, 33 (12), 2350065. https://doi.org/:10.1142/S012906572350065X
Whang, S. E., Roh, Y., Song, H., and Lee, J.-G. (2023). Data collection and quality challenges in deep learning: a data-centric AI perspective.  The VLDB Journal, 32 (4), 791-813. https://doi.org/:10.1007/s00778-022-00775-9
Zini, J. E., and Awad, M. (2022). On the Explainability of Natural Language Processing Deep Models.  ACM Comput. Surv., 55 (5), Article 103. https://doi.org/:10.1145/3529755

  • تاریخ دریافت 12 مرداد 1404