Volume 36, Issue 3 (Spring 2021)
... 2021, 36(3): 817-834 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Hosseini M M H, Jalai A. Determining Reliability in Interactive Question Answering Systems by Regression. .... 2021; 36 (3) :817-834
URL: http://jipm.irandoc.ac.ir/article-1-4290-en.html
URL: http://jipm.irandoc.ac.ir/article-1-4290-en.html
Department of Computer Engineering, Shahrood Branch, Islamic Azad University, Shahrood, Iran
Abstract: (755 Views)
Nowadays, the increasing amount of information available online has led scientists to develop technologies to use and deal with it, of which one is Interactive Question Answering (IQA) systems. One of the most important criteria in the assessment of these systems is reliability. Research results in this field have shown that evaluation of reliability in IQA plays an important role to determine the effectiveness of these systems. During the process of reliability evaluation of an IQA system, several features are considered. Few attempts have been made to measure the reliability and accurate determination of reliability criteria in interactive question answering systems.
In this paper, a novel method is proposed to determine the percentage of reliability by using a set of criteria affecting including qualitative and quantitative criteria. In the proposed method, first, a list of qualitative and quantitative characteristics is collected and according to the criteria set, eight quantitative characteristics are selected. Then, based on the prepared questionnaire, the coefficients of the impact of each question are calculated. According to the proposed relationship and by combining these characteristics the percentage of reliability is measured. The evaluation of the performance of the selected criteria, on four interactive question answering indicates that by applying this set of criteria the reliability of an interactive question answering system can be used to automate the calculation of this feature in the evaluation process.
Keywords: Reliability, Evaluation Criteria, Qualitative Software Features, Interactive Question Answering Systems
Type of Study: Research |
Subject:
Information Technology
Received: 2019/07/15 | Accepted: 2020/11/2 | Published: 2021/04/5
Received: 2019/07/15 | Accepted: 2020/11/2 | Published: 2021/04/5
References
1. حسینی، محمدمهدی، و مرتضی زاهدی. 1395. بهبود پاسخ ارائه‌شده در سیستم‌های پرسش و پاسخ تعاملی با استفاده از شبکه عصبی. هشتمین کنفرانس بین‌المللی فناوری اطلاعات و دانش. همدان. صفحات 84-91.
2. حسینی، محمدمهدی، و مرتضی زاهدی. 1397. ارائه یک مدل احتمالاتی جهت تعیین انسجام متن در سیستم‌های پرسش و پاسخ تعاملی. مجله پردازش و مدیریت اطلاعات 33 (4): 1737-1760.
3. حسینی، محمدمهدی، مرتضی زاهدی، و حمید حسن‌پور. 1397. مجموعه‌ای از ویژگی‌های آماری جدید برای ارزیابی سیستم‌های پرسش و پاسخ تعاملی. مجله مهندسی برق تبریز.
4. شهرآیینی، سلیمه‌سادات، و مرتضی زاهدی. 1394. سیستم پاسخگوی تعاملی با استفاده از تکنیک‌های هوش مصنوعی. پایان‌نامه کارشناسی ارشد. دانشکده کامپیوتر، دانشگاه صنعتی شاهرود.
5. Baudiš, Petr. 2015. YodaQA: a modular question answering system pipeline", In POSTER 2015-19th International Student Conference on Electrical Engineering, pp. 1156-1165. Prague.
6. Bhatia, Pradeep K. and Mann, Rajbeer. 2008. An Approach to Measure Software Reusability of OO Design. Proceedings of 2nd National Conference on Challenges & Opportunities in Information Technology, RIMT-IET, Mandi Gobindgarh, pp. 26-30. India.
7. Breck, E., J. D. Burger, L. Ferro, L. Hirschman, D. House, M. Light, and I. Mani. 2000. How to evaluate your question answering system every day and still get real work done. In Proceedings of the Second International Conference on Language Resources and Evaluation (LREC), Greece. pp. 51-57.
8. Burstein, J., & M. Chodorow. 2009. Automated essay scoring for non-native English speakers. In Proceedings of the ACL Workshop on Computer Mediated Language Assessment and Evaluation of Natural Language Processing. Mexico.
9. Carl G. Davis, and Jagdish Bansiya. 2013. A Hierarchical Model for Object-Oriented Design Quality Assessment. IEEE Transactions on Software Engineering 28 (1): 1-31.
10. Derici, Caner, Yiğit Aydin, Çiğdem Yenialaca, Nihal Yağmur Aydin, Günizi Kartal, Arzucan Özgür, and Tunga Güngör. 2018. A closed-domain question answering framework using reliable resources to assist students. Natural Language Engineering 24 (5): 725-762. [DOI:10.1017/S1351324918000141]
11. Dwivedi, S. K., and V. Singh. 2013. Research and Reviews in Question Answering System. Procedia Technology, pp. 424-417. [DOI:10.1016/j.protcy.2013.12.378]
12. Harabagiu, S., A. Hickl, J. Lehmann, and D. Moldovan. 2005. Experiments with interactive question-answering. In Proceedings of the 43rd annual meeting on Association for Computational Linguistics, Ann Arbor, USA, pp. 205-214. [DOI:10.3115/1219840.1219866]
13. Iyyer, Mohit, Wen-tau Yih, and Ming-Wei Chang. 2017. Search-based neural structured learning for sequential question answering. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1821-1831. Vancouver, Canada. [DOI:10.18653/v1/P17-1167]
14. Laheeb M. Al-Zobaidy, & Khalil A. Ibrahim. 2013. Existing Object Oriented Design Metrics a Study and Comparison. Journal of Computer 10: 137-147. [DOI:10.33899/csmj.2013.163431]
15. Motte, Stanislas, Ramon Ruti, Arnaud Jacolin, and Pierre-Albert Ruquier. 2019. Apparatus, systems and methods for scoring and distributing the reliability of online information. U.S. Patent 10, 169, 424, issued January 1, 2019. [DOI:10.1155/2019/7413460]
16. Paliwal, Nagesh, Vivek Shrivastava, & Ketki Tiwari. 2014. An Approach to Find Reusability of Software Using Objet Oriented Metrics. International Journal of Innovative Research in Science, Engineering and Technology 3 (3): 10371-10378.
17. Salata, Ferdinando, Andrea de Lieto Vollaro, Roberto de LietoVollaro, and Lorenzo Mancieri. 2015. Method for energy optimization with reliability analysis of a trigeneration and teleheating system on urban scale: A case study. Energy and Buildings 86: 118-136. [DOI:10.1016/j.enbuild.2014.09.056]
18. Spitters, M., M. De Boni, J. Zavrel, and R. Bonnema. 2009. Learning effective and engaging strategies for advice-giving human-machine dialogue. Natural Language Engineering 15 (3): 378-355. [DOI:10.1017/S1351324908004956]
19. Suresh Y, J. Pati, Ku Rath. 2012. Effectiveness of Software Metrics For Object-Oriented System. Procedia Technology 6: 420-427. [DOI:10.1016/j.protcy.2012.10.050]
20. Webber, B., and N. Webb. 2010. Question answering, the handbook of computational linguistics and natural language processing. Blackwell Publishing. PP. 630-654, 2010 [DOI:10.1002/9781444324044.ch22]
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
