Advancing Natural Language Processing with New Models and Applications in 2025

Sabah, Sura; Abbas, Haider Hadi; Gulmira Karimovna, Kudaiberdieva; M. A. D. Najm, Nahlah; Abdulkhaleq Ali, Ammar

doi:10.22034/jipm.2025.728102

Advancing Natural Language Processing with New Models and Applications in 2025

Document Type : Original Article

Authors

Sura Sabah ¹

Haider Hadi Abbas ²

Kudaiberdieva Gulmira Karimovna ³

Nahlah M. A. D. Najm ⁴

Ammar Abdulkhaleq Ali ⁵

¹ Al-Turath University, Baghdad 10013, Iraq

² Al-Mansour University College, Baghdad 10067, Iraq

³ Osh State University, Osh City 723500, Kyrgyzstan

⁴ Al-Rafidain University College Baghdad 10064, Iraq

⁵ Madenat Alelem University College, Baghdad 10006, Iraq

10.22034/jipm.2025.728102

Abstract

ABSTRACT
Background: Recent advancements in Natural Language Processing (NLP) have been significantly influenced by transformer models. However, challenges related to scalability, discrepancies between pretraining and finetuning, and suboptimal performance on tasks with diverse and limited data remain. The integration of Reinforcement Learning (RL) with transformers has emerged as a promising approach to address these limitations.
Objective: This article aims to evaluate the performance of a transformer-based NLP model integrated with RL across multiple tasks, including translation, sentiment analysis, and text summarization. Additionally, the study seeks to assess the model's efficiency in real-time operations and its fairness.
Methods: The hybrid model's effectiveness was evaluated using task-oriented metrics such as BLEU, F1, and ROUGE scores across various task difficulties, dataset sizes, and demographic samples. Fairness was measured based on demographic parity and equalized odds. Scalability and real-time performance were assessed using accuracy and latency metrics.
Results: The hybrid model consistently outperformed the baseline transformer across all evaluated tasks, demonstrating higher accuracy, lower error rates, and improved fairness. It also exhibited robust scalability and significant reductions in latency, enhancing its suitability for real-time applications.
Conclusion: This article illustrates that the proposed hybrid model effectively addresses issues related to scale, diversity, and fairness in NLP. Its flexibility and efficacy make it a valuable tool for a wide range of linguistic and practical applications. Future research should focus on improving time complexity and exploring the use of deep unsupervised learning for low-resource languages.

Keywords

Natural Language Processing (NLP)

transformer models

hybrid NLP systems

reinforcement learning

machine translation (MT)

sentiment analysis

multilingual data

AI applications

bias mitigation

ethical NLP

References

Alnuaemy, L. M. (2023). Peculiarities of using neuro-linguistic programming for the rehabilitation of servicemen who were in armed conflicts. Development of Transport Management and Management Methods, 3 (84), 40-55. https://doi.org/:10.31375/2226-1915-2023-3-40-55

Amer, S., Lee, M., and Smith, P. (2023). Cross-lingual Classification of Crisis-related Tweets Using Machine Translation. https://doi.org/:10.26615/978-954-452-092-2_003

Baliyan, A., Batra, A., and Singh, S. P. (2021). Multilingual Sentiment Analysis using RNN-LSTM and Neural Machine Translation. 8th International Conference on Computing for Sustainable Global Development (INDIACom), 710-713.

Cheng, L., Ge, S., and Liu, H. (2022). Toward understanding bias correlations for mitigation in NLP. arXiv preprint. 2205.12391. https://doi.org/:10.48550/arXiv.2205.12391

Czarnowska, P., Vyas, Y., and Shah, K. (2021). Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics. Transactions of the Association for Computational Linguistics, 9, 1249-1267. https://doi.org/:10.1162/tacl_a_00425

Futrell, R. (2023). Validity, Reliability, and Significance: Empirical Methods for NLP and Data Science. Computational Linguistics, 49 (1), 249-251. https://doi.org/:10.1162/coli_r_00467

Hashim, N., Mohsim, A., Rafeeq, R., and Pyliavskyi, V. (2019a). New approach to the construction of multimedia test signals. International Journal of Advanced Trends in Computer Science and Engineering, 8 (6), 3423-3429. https://doi.org/:10.30534/ijatcse/2019/117862019

Hashim, N., Mohsim, A. H., Rafeeq, R. M., and Pyliavskyi, V. (2019b). New approach to the construction of multimedia test signals. International Journal of Advanced Trends in Computer Science and Engineering, 8 (6), 3423-3429. https://doi.org/:10.30534/ijatcse/2019/117862019

Jawad, A. M., Qasim, N. H., and Pyliavskyi, V. (2022). Comparison of Metamerism Estimates in Video Paths using CAM's Models. IEEE 9th International Conference on Problems of Infocommunications, Science and Technology (PIC S&T), 10-12 Oct. https://doi.org/:10.1109/PICST57299.2022.10238685

Khan, J., Ahmad, N., Khalid, S., Ali, F., and Lee, Y. (2023). Sentiment and Context-Aware Hybrid DNN With Attention for Text Sentiment Classification. IEEE Access, 11, 28162-28179. https://doi.org/:10.1109/ACCESS.2023.3259107

Krishna, G. G. (2023). Reinforcement Learning based NLP. International Journal of Soft Computing and Engineering, 13 (4). https://doi.org/:10.35940/ijsce.j0476.0913423

Li, W., Luo, H., Lin, Z., Zhang, C., Lu, Z., and Ye, D. (2023). A survey on transformers in reinforcement learning. arXiv preprint, 2301.03044. https://doi.org/:10.48550/arXiv.2301.03044

Maurya, K. K., and Desarkar, M. S. (2022). Meta-X $ _ {NLG} $: A Meta-Learning Approach Based on Language Clustering for Zero-Shot Cross-Lingual Transfer and Generation. arXiv preprint, 2203.10250. https://doi.org/:10.48550/arXiv.2203.10250

Moon, W., Kim, T., Park, B., and Har, D. (2023). Enhanced Transformer Architecture for Natural Language Processing. arXiv preprint, 2310.10930. https://doi.org/:10.48550/arXiv.2310.10930

Nameer, Q., Aqeel, J., and Muthana, M. (2023). The Usages of Cybersecurity in Marine Communications. Transport Development, 3 (18). https://doi.org/:10.33082/td.2023.3-18.05

Nguyen, T., Nguyen, L., Tran, P., and Nguyen, H. (2021). Improving Transformer-Based Neural Machine Translation with Prior Alignments. Complexity, 2021 (1), 5515407. https://doi.org/:10.1155/2021/5515407

Rahim, F., Bodnar, N., Qasim, N. H., Jawad, A. M., and Ahmed, O. S. (2023). Integrating Machine Learning in Environmental DNA Metabarcoding for Improved Biodiversity Assessment: A Review and Analysis of Recent Studies. Research Square. https://doi.org/:10.21203/rs.3.rs-2823060/v1

Roit, P., Ferret, J., Shani, L., Aharoni, R., Cideron, G., Dadashi, R., Geist, M., et al. (2023). Factually consistent summarization via reinforcement learning with textual entailment feedback. arXiv preprint, 2306.00186. https://doi.org/:10.48550/arXiv.2306.00186

Singh, S., and Mahmood, A. (2021). The NLP Cookbook: Modern Recipes for Transformer Based Deep Learning Architectures. IEEE Access, 9, 68675-68702. https://doi.org/:10.1109/ACCESS.2021.3077350

Sivamayil, K., Rajasekar, E., Aljafari, B., Nikolovski, S., Vairavasundaram, S., and Vairavasundaram, I. (2023). A Systematic Study on Reinforcement Learning Based Applications. Energies, 16 (3). https://doi.org/:10.3390/en16031512

Somers, R., Cunningham-Nelson, S., and Boles, W. (2021). Applying natural language processing to automatically assess student conceptual understanding from textual responses. Australasian Journal of Educational Technology, 37 (5), 98-115. https://doi.org/:10.14742/ajet.7121

Sunna Torge, A. P., Christoph Lehmann, Bochra Saffar, and Ziyan Tao. (2023). Named Entity Recognition for Low-Resource Languages - Profiting from Language Families. In Proceedings of the 9th Workshop on Slavic Natural Language Processing (SlavicNLP 2023), 1–10. https://doi.org/:10.18653/v1/2023.bsnlp-1.1

Tan, K. L., Lee, C. P., Lim, K. M., and Anbananthen, K. S. M. (2022). Sentiment Analysis With Ensemble Hybrid Deep Learning Model. IEEE Access, 10, 103694-103704. https://doi.org/:10.1109/ACCESS.2022.3210182

Tariq, A., and Ahmed, A. (2022). Deep Learning in Sentiment Analysis: Recent Architectures. ACM Comput. Surv, 55 (8), Article 159. https://doi.org/:10.1145/3548772

Tushar Agarwal, J. J., Gaurav Kumar. (2023). Transformer and Natural language processing; A recent development. Tuijin Jishu/Journal of Propulsion Technology, 44 (1). https://doi.org/:10.52783/tjjpt.v44.i1.2225

Villarrubia-Martin, E. A., Rodriguez-Benitez, L., Jimenez-Linares, L., Muñoz-Valero, D., and Liu, J. (2023). A Hybrid Online Off-Policy Reinforcement Learning Agent Framework Supported by Transformers. International Journal of Neural Systems, 33 (12), 2350065. https://doi.org/:10.1142/S012906572350065X

Whang, S. E., Roh, Y., Song, H., and Lee, J.-G. (2023). Data collection and quality challenges in deep learning: a data-centric AI perspective. The VLDB Journal, 32 (4), 791-813. https://doi.org/:10.1007/s00778-022-00775-9

Zini, J. E., and Awad, M. (2022). On the Explainability of Natural Language Processing Deep Models. ACM Comput. Surv., 55 (5), Article 103. https://doi.org/:10.1145/3529755