Evaluating and Predicting the Quality of Answers Factors in the Research Gateâs Question and Answer System: a Case Study of the Thematic Domain of Knowledge Management

Anbaraki, Sahar; Jowkar, Abdolrasool

doi:10.52547/jipm.36.3.709

Evaluating and Predicting the Quality of Answers Factors in the Research Gateâs Question and Answer System: a Case Study of the Thematic Domain of Knowledge Management

Authors

sahar Anbaraki

Abdolrasool Jowkar

10.52547/jipm.36.3.709

Abstract

Question answering (QA) helps one go beyond traditional keywords-based querying and retrieve information in more precise form than given by a document or a list of documents. These communities allow users to submit queries and receive answers or responses from other members. But, there is no clear way of evaluating the quality of that information. The purpose of this study was to evaluate and predict the quality of responses in the Research Gate Social Science Network. To achieve this goal, the questions and answers entered in the field from January to May 2019 surveyed in the field and the required information was collected by the site crawler. Finally, 54 questions and 443 answers were analyzed in descriptive and inferential levels by SPSS 22 software. The results show that the relevance, adequacy, and concordance variables with the odds ratios of 3.626, 3.440 and 3.148 have the most power to predict the correct or incorrect responses, respectively.

Keywords

Question and Answer System

Social Science Network

Research Gate

Quality Prediction of Responses

آفتابی، زهرا. 1393. مروری بر سیستم‌های پرسش و پاسخ مبتنی بر بازیابی اطلاعات. پایان‌نامه کارشناسی ارشد مهندسی کامپیوتر. دانشکده مهندسی برق و کامپیوتر، دانشگاه یزد.
آفتابی، زهرا. 1393. مروری بر سیستم‌های پرسش و پاسخ مبتنی بر بازیابی اطلاعات. پایان‌نامه کارشناسی ارشد مهندسی کامپیوتر. دانشکده مهندسی برق و کامپیوتر، دانشگاه یزد.
آقاکاردان، احمد، و امیر نریمانی. 1395. ارائه روشی نو به منظور ارزیابی میزان مشارکت کاربران در انجمن‌های پرسش و پاسخ. دومین کنفرانس بین‌المللی وب‌پژوهی. تهران، دانشگاه علم و فرهنگ.
آقاکاردان، احمد، و امیر نریمانی. 1395. ارائه روشی نو به منظور ارزیابی میزان مشارکت کاربران در انجمن‌های پرسش و پاسخ. دومین کنفرانس بین‌المللی وب‌پژوهی. تهران، دانشگاه علم و فرهنگ.
Adamic, Lada, Jun Zhang, Eytan Bakshy, & Mark Ackerman. 2008. Knowledge Sharing and Yahoo Answers: Everyone Knows Something. International World Wide Web Conference Committee (IW3C2). April 21-25, Beijing, China. [DOI:10.1145/1367497.1367587]
Adamic, Lada, Jun Zhang, Eytan Bakshy, & Mark Ackerman. 2008. Knowledge Sharing and Yahoo Answers: Everyone Knows Something. International World Wide Web Conference Committee (IW3C2). April 21-25, Beijing, China. [DOI:10.1145/1367497.1367587]
Amitay, E, N. Carmel, Ofek-Koifman Har'El, A. Soffer, S. Yogev, and N. Golbandi. 2009. Social search and discovery using a unified approach. In Proceedings of the 20th ACM Conference on Hypertext and Hypermedia (pp. 199-208). Nara, Japan. [DOI:10.1145/1557914.1557950]
Amitay, E, N. Carmel, Ofek-Koifman Har'El, A. Soffer, S. Yogev, and N. Golbandi. 2009. Social search and discovery using a unified approach. In Proceedings of the 20th ACM Conference on Hypertext and Hypermedia (pp. 199-208). Nara, Japan. [DOI:10.1145/1557914.1557950]
Awofeso, N., M. Hassan, & S. Hamidi. 2016. Individual and collaborative technology-mediated learning using question & answer online discussion forums-perceptions of Public Health learners in Dubai, UAE. Open Learning: The Journal of Open, Distance and e-Learning 31 (1): 54-63. [DOI:10.1080/02680513.2015.1120662]
Awofeso, N., M. Hassan, & S. Hamidi. 2016. Individual and collaborative technology-mediated learning using question & answer online discussion forums-perceptions of Public Health learners in Dubai, UAE. Open Learning: The Journal of Open, Distance and e-Learning 31 (1): 54-63. [DOI:10.1080/02680513.2015.1120662]
Cai, Yuanzhe, and Sharma Chakravarthy. 2011. Predicting Answer Quality in Q/A Social Networks: Using Temporal Features. Technical Report CSE. IT Laboratory & Department of Computer Science & Engineering. The University of Texas at Arlington.
Cai, Yuanzhe, and Sharma Chakravarthy. 2011. Predicting Answer Quality in Q/A Social Networks: Using Temporal Features. Technical Report CSE. IT Laboratory & Department of Computer Science & Engineering. The University of Texas at Arlington.
Elalfy, D., W. Gad, & R. Ismail. 2018. A hybrid model to predict best answers in question answering communities. Egyptian informatics journal 19 (1): 21-31. [DOI:10.1016/j.eij.2017.06.002]
Elalfy, D., W. Gad, & R. Ismail. 2018. A hybrid model to predict best answers in question answering communities. Egyptian informatics journal 19 (1): 21-31. [DOI:10.1016/j.eij.2017.06.002]
Evans, B. M., and E. H. Chi. 2010. An elaborated model of social search. Information Processing & Management 46 (6): 656-678. [DOI:10.1016/j.ipm.2009.10.012]
Evans, B. M., and E. H. Chi. 2010. An elaborated model of social search. Information Processing & Management 46 (6): 656-678. [DOI:10.1016/j.ipm.2009.10.012]
Fu, H., & S. Oh. 2019. Quality assessment of answers with user-identified criteria and data-driven features in social Q&A. Information Processing & Management 56 (1): 14-28. [DOI:10.1016/j.ipm.2018.08.007]
Fu, H., & S. Oh. 2019. Quality assessment of answers with user-identified criteria and data-driven features in social Q&A. Information Processing & Management 56 (1): 14-28. [DOI:10.1016/j.ipm.2018.08.007]
Giglia, E. 2011. Academic social networks: It's time to change the way we do research. European Journal of Physical and Rehabilitation Medicine 47 (2): 345-9.
Giglia, E. 2011. Academic social networks: It's time to change the way we do research. European Journal of Physical and Rehabilitation Medicine 47 (2): 345-9.
Harper, F. M., D. Raban, S. Rafaeli, and J. A. Konstan. 2008. Predictors of answer quality in online Q&A sites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 865-874). Vancouver, Canada. [DOI:10.1145/1357054.1357191]
Harper, F. M., D. Raban, S. Rafaeli, and J. A. Konstan. 2008. Predictors of answer quality in online Q&A sites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 865-874). Vancouver, Canada. [DOI:10.1145/1357054.1357191]
He, X., L. Wang, W. Zhang, & P. Zhang. 2019. Research on the Quality Prediction of Online Chinese Question Answering Community Answers Based on Comments. In Proceedings of the 2nd International Conference on Big Data Technologies (pp. 114-120). Tetouan, Morocco. [DOI:10.1145/3358528.3358592]
He, X., L. Wang, W. Zhang, & P. Zhang. 2019. Research on the Quality Prediction of Online Chinese Question Answering Community Answers Based on Comments. In Proceedings of the 2nd International Conference on Big Data Technologies (pp. 114-120). Tetouan, Morocco. [DOI:10.1145/3358528.3358592]
Hosmer Jr, D. W., S. Lemeshow, & R. X. Sturdivant. 2013. Applied logistic regression (Vol. 398). Hoboken: John Wiley & Sons. [DOI:10.1002/9781118548387]
Hosmer Jr, D. W., S. Lemeshow, & R. X. Sturdivant. 2013. Applied logistic regression (Vol. 398). Hoboken: John Wiley & Sons. [DOI:10.1002/9781118548387]
Hsieh, G., and S. Counts. 2009. Mimir: A market-based real-time question and answer service. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 769-778). Vancouver, Canada. [DOI:10.1145/1518701.1518820]
Hsieh, G., and S. Counts. 2009. Mimir: A market-based real-time question and answer service. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 769-778). Vancouver, Canada. [DOI:10.1145/1518701.1518820]
Jin, Xiao-Ling, Zhongyun Zhou, Matthew K. O. Lee, & Christy M. K. Cheung. 2013. Why users keep answering questions in online question answering communities: a theoretical and empirical investigation. International Journal of Information Management 33: 93- 104. [DOI:10.1016/j.ijinfomgt.2012.07.007]
Jin, Xiao-Ling, Zhongyun Zhou, Matthew K. O. Lee, & Christy M. K. Cheung. 2013. Why users keep answering questions in online question answering communities: a theoretical and empirical investigation. International Journal of Information Management 33: 93- 104. [DOI:10.1016/j.ijinfomgt.2012.07.007]
Kim, Soojung, & Sanghee Oh. 2009. Users' Relevance Criteria for Evaluating Answers in a Social Q&A Site. Journal of the American Society for Information Science and Technology 60 (4): 716-727. [DOI:10.1002/asi.21026]
Kim, Soojung, & Sanghee Oh. 2009. Users' Relevance Criteria for Evaluating Answers in a Social Q&A Site. Journal of the American Society for Information Science and Technology 60 (4): 716-727. [DOI:10.1002/asi.21026]
Kim, Soojung, Jung sun Oh, & Sanghee Oh. 2006. Best-Answer Selection Criteria in a Social Q&A site from the User-Oriented Relevance Perspective. Proceedings of the American Society for Information Science and Technology 44 (1): 1-15. [DOI:10.1002/meet.1450440256]
Kim, Soojung, Jung sun Oh, & Sanghee Oh. 2006. Best-Answer Selection Criteria in a Social Q&A site from the User-Oriented Relevance Perspective. Proceedings of the American Society for Information Science and Technology 44 (1): 1-15. [DOI:10.1002/meet.1450440256]
Le, L. T., C. Shah, & E. Choi. 2016. Evaluating the quality of educational answers in community question-answering. In 2016 IEEE/ACM Joint Conference on Digital Libraries (JCDL) (pp. 129-138). IEEE. Newark, NJ, USA. [DOI:10.1145/2910896.2910900]
Le, L. T., C. Shah, & E. Choi. 2016. Evaluating the quality of educational answers in community question-answering. In 2016 IEEE/ACM Joint Conference on Digital Libraries (JCDL) (pp. 129-138). IEEE. Newark, NJ, USA. [DOI:10.1145/2910896.2910900]
Leon, Ramona - Diana, Raúl Rodríguez Rodriguez, Pedro Gómez Gasquet, & Josefa Mula. 2017. Social network analysis: a tool for evaluating and predicting future knowledge flows from an insurance organization. Technological Forecasting & Social Change 114: 103-118. [DOI:10.1016/j.techfore.2016.07.032]
Leon, Ramona - Diana, Raúl Rodríguez Rodriguez, Pedro Gómez Gasquet, & Josefa Mula. 2017. Social network analysis: a tool for evaluating and predicting future knowledge flows from an insurance organization. Technological Forecasting & Social Change 114: 103-118. [DOI:10.1016/j.techfore.2016.07.032]
Li, M., Y. Lu, Y. Li, & Y. Zhang. 2019. Evaluating Indicators of Answer Quality in Social Q&A Websites. In PACIS (p. 217). Pacific Asia Conference on Information Systems (PACIS). at AIS Electronic Library (AISeL) (p. 217).
Li, M., Y. Lu, Y. Li, & Y. Zhang. 2019. Evaluating Indicators of Answer Quality in Social Q&A Websites. In PACIS (p. 217). Pacific Asia Conference on Information Systems (PACIS). at AIS Electronic Library (AISeL) (p. 217).
Liu, Yandong Jiang Bian, & Eugene Agichtein. 2008. Predicting Information Seeker Satisfaction in Community Question Answering. In Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval (pp. 483-490). Singapore. [DOI:10.1145/1390334.1390417]
Liu, Yandong Jiang Bian, & Eugene Agichtein. 2008. Predicting Information Seeker Satisfaction in Community Question Answering. In Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval (pp. 483-490). Singapore. [DOI:10.1145/1390334.1390417]
Madhusudhan, M. 2012. Use of social networking sites by research scholars of the university of Delhi: a study. The International Information & Library Review 44 (2): 100-13. [DOI:10.1080/10572317.2012.10762919]
Madhusudhan, M. 2012. Use of social networking sites by research scholars of the university of Delhi: a study. The International Information & Library Review 44 (2): 100-13. [DOI:10.1080/10572317.2012.10762919]
Oh, Sanghee, Jung Sun Oh, & Chirag Shah. 2008. The Use of Information Sources by Internet Users in Answering Questions. Journal of University of North Carolina at Chapel Hill 4 (10): 1-13. [DOI:10.1002/meet.2008.1450450279]
Oh, Sanghee, Jung Sun Oh, & Chirag Shah. 2008. The Use of Information Sources by Internet Users in Answering Questions. Journal of University of North Carolina at Chapel Hill 4 (10): 1-13. [DOI:10.1002/meet.2008.1450450279]
Oh, Sanghee. 2012. The Characteristics and Motivations of Health Answerers for Sharing Information, Knowledge, and Experiences in Online Environments. Journal of the American Society for Information Science and Technology 63 (3): 543-557. [DOI:10.1002/asi.21676]
Oh, Sanghee. 2012. The Characteristics and Motivations of Health Answerers for Sharing Information, Knowledge, and Experiences in Online Environments. Journal of the American Society for Information Science and Technology 63 (3): 543-557. [DOI:10.1002/asi.21676]
Riahi, Fatemeh, Zainab Zolaktaf, Mahdi Shafiei, & Evangelos Milios. 2012. Finding Expert Users in Community Question Answering. International World Wide Web Conference, April 16-20, Lyon, France. [DOI:10.1145/2187980.2188202]
Riahi, Fatemeh, Zainab Zolaktaf, Mahdi Shafiei, & Evangelos Milios. 2012. Finding Expert Users in Community Question Answering. International World Wide Web Conference, April 16-20, Lyon, France. [DOI:10.1145/2187980.2188202]
Saracevic, T. 1995. Evaluation of evaluation in information retrieval. Proceedings of the ACM SIGIR International Conference on Research and Development in Information Retrieval (pp. 138-146). Seattle, USA. [DOI:10.1145/215206.215351]
Saracevic, T. 1995. Evaluation of evaluation in information retrieval. Proceedings of the ACM SIGIR International Conference on Research and Development in Information Retrieval (pp. 138-146). Seattle, USA. [DOI:10.1145/215206.215351]
Shah, Chirag, & Jefferey Pomerantz. 2010. Evaluating and Predicting Answer Quality in Community QA. In Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval (pp. 411-418). Geneva, Switzerland. [DOI:10.1145/1835449.1835518]
Shah, Chirag, & Jefferey Pomerantz. 2010. Evaluating and Predicting Answer Quality in Community QA. In Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval (pp. 411-418). Geneva, Switzerland. [DOI:10.1145/1835449.1835518]
Shen, Haiying. 2015. Knowledge Sharing in the Online Social Network of Yahoo! Answers and Its Implications. IEEE Transactions on Computers 64 (6): 1715-1728.
Shen, Haiying. 2015. Knowledge Sharing in the Online Social Network of Yahoo! Answers and Its Implications. IEEE Transactions on Computers 64 (6): 1715-1728.
Sun, M., L. Liu, & H. Chen. 2018. An Answer Quality Evaluation Algorithm Based on Semantic in Community-Based Question Answering. In 2018 14th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD) (pp. 971-978). IEEE. Bijing, China. [DOI:10.1109/FSKD.2018.8687192]
Sun, M., L. Liu, & H. Chen. 2018. An Answer Quality Evaluation Algorithm Based on Semantic in Community-Based Question Answering. In 2018 14th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD) (pp. 971-978). IEEE. Bijing, China. [DOI:10.1109/FSKD.2018.8687192]
Teevan, Jaime, Meredith Ringel Morris, & Katrina Panovich. 2011. Factors Affecting Response Quantity, Quality, and Speed for Questions Asked via Social Network Status Messages. Proceedings of the Fifth International AAAI Conference on Weblogs and Social Media, Massachusetts, USA.
Teevan, Jaime, Meredith Ringel Morris, & Katrina Panovich. 2011. Factors Affecting Response Quantity, Quality, and Speed for Questions Asked via Social Network Status Messages. Proceedings of the Fifth International AAAI Conference on Weblogs and Social Media, Massachusetts, USA.
White, R. W, M. Richardson, and Y. Liu. 2011. Effects of community size and contact rate in synchronous social Q&A. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2837-2846). Vancouver, Canada. [DOI:10.1145/1978942.1979364]
White, R. W, M. Richardson, and Y. Liu. 2011. Effects of community size and contact rate in synchronous social Q&A. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2837-2846). Vancouver, Canada. [DOI:10.1145/1978942.1979364]
Xiao-Ling Jin, Zhongyun Zhou, Matthew K.O. Lee, & Christy M. K. Cheung. 2013. Why users keep answering questions in online question answering communities: A theoretical and empirical investigation. International Journal of Information Management 33: 93- 104. [DOI:10.1016/j.ijinfomgt.2012.07.007]
Xiao-Ling Jin, Zhongyun Zhou, Matthew K.O. Lee, & Christy M. K. Cheung. 2013. Why users keep answering questions in online question answering communities: A theoretical and empirical investigation. International Journal of Information Management 33: 93- 104. [DOI:10.1016/j.ijinfomgt.2012.07.007]