واحد شاهرود، دانشگاه آزاد اسلامی، شاهرود، ایران
عنوان مقاله [English]
Nowadays, the increasing amount of information available online has led scientists to develop technologies to use and deal with it, of which one is Interactive Question Answering (IQA) systems. One of the most important criteria in the assessment of these systems is reliability. Research results in this field have shown that evaluation of reliability in IQA plays an important role to determine the effectiveness of these systems. During the process of reliability evaluation of an IQA system, several features are considered. Few attempts have been made to measure the reliability and accurate determination of reliability criteria in interactive question answering systems.
In this paper, a novel method is proposed to determine the percentage of reliability by using a set of criteria affecting including qualitative and quantitative criteria. In the proposed method, first, a list of qualitative and quantitative characteristics is collected and according to the criteria set, eight quantitative characteristics are selected. Then, based on the prepared questionnaire, the coefficients of the impact of each question are calculated. According to the proposed relationship and by combining these characteristics the percentage of reliability is measured. The evaluation of the performance of the selected criteria, on four interactive question answering indicates that by applying this set of criteria the reliability of an interactive question answering system can be used to automate the calculation of this feature in the evaluation process.