Iranian Journal of Information Processing and Management

Iranian Journal of Information Processing and Management

Predicting Users' Level of Disorientation for Their Level of Mental Model Completeness, Cognitive Over Load, Experience, and Gender

Document Type : Original Article

Authors
yalda beizavi 1
mahdieh mirzabeigi 2
tahereh jowkar 3
alireza nikseresht 4
1 Master in Knowledge & Information Science; Shiraz University
2 PhD in Knowledge & Information Science; Associate Professor; Shiraz University
3 PhD in Knowledge & Information Science; Assistant Professor; Shiraz University
4 PhD in computing Science; Assistant Professor; Shiraz University
10.22034/jipm.2024.1999945.1083
Abstract
The objective of this study was to identify the roles of mental model, cognitive overload, experience, and gender, on the level of users' disorientation in interaction with the Google search engine. This applied study is descriptive (correlational) in terms of data collection. It also used the observation transaction log analysis which is a data collection method in human-computer interaction studies. The research population includes 90 graduate students of Shiraz University using the non-probability sampling method of the available type. The dependent variable of the research includes the level of disorientation, and the independent variables include the completeness of the mental model, gender, cognitive overload, and experience.
Findings showed that the disorientation level had a significant relationship with experience, gender, mental model, and cognitive overload. In other words, an increase in experience and the level of completeness of the mental model of the users reduce their level of disorientation. In other words, as the level of completeness of the user's mental model increases, there are more elements of the system in the user's mental model; as a result, the understanding of the system is greater and the level of confusion and disorientation is reduced.
In addition, increasing the cognitive overload increased the disorientation level. Users who suffer from cognitive overload while searching get lost more than other users.
 The results showed that each of the variables of cognitive overload, gender, experience, and completeness level of the mental model could predict a 7 to 21 percent of disorientation level. However, considering the interaction of the variables, experience, and gender strongly predicted disorientation levels. So that the low level of disorientation is related to the male gender and the high level of disorientation is related to the female gender, in other words, female users get lost more often than male users. Moreover, the complexity level, both independently and in interaction with experience, affected the disorientation level. Specifically, users were disoriented in more complex tasks than simple ones; inexperienced users were disoriented in more complex tasks. Since earlier studies have not addressed the interactive role of experience, gender, mental model, and cognitive overload, along with the complexity level of the task, on users' disorientation level, the results of this study can help designers of retrieval systems, information literacy specialists and researchers detect the factors affecting the disorientation level.
Keywords
Disorientation, Cognitive Overload, Mental Model, Experience, Gender
Subjects
فهرست منابع
آبام، زویا، رحمت‌الله فتاحی، مهری پریرخ، و جواد صالحی فدردی 1390. بررسی چگونگی تعامل کاربران مختلف با محیط فراپیوندی وب و مطالعه عملکرد آن‌ها در جست‌وجوی مروری. فصلنامه کتابداری و اطلاع‌رسانی 55 (3): 51-81.
بحرینی، نعیمه. 1395. مطالعه رفتار، عملکرد و گم‌گشتگی دانشجویان در جست‌وجوی مروری از رویکرد سبک‌شناختی کلامی-تصویری رایدینگ و چیما. پایان‌نامه کارشناسی ارشد، دانشگاه شیراز.
خسروی، عبدالرسول، رحمت‌الله فتاحی، مهری پریرخ، و محمدحسین دیانی. 1391. تحلیل بسط جست‌وجوی کاربران در موتور کاوش گوگل بر پایه نظریة بار شناختی. کتابداری و اطلاع‌رسانی 15 (2): 3-9.
ذوالفقاری، مریم. 1395. بررسی عملکرد مرور دانشجویان بر حسب تجربه و سطح کامل بودن مدل ذهنی. پایان‌نامه کارشناسی ارشد، دانشگاه شیراز.
رهروانی، ساناز، مهدیه میرزابیگی، و جواد عباسپور. 1395. مطالعه سطح کامل بودن مدل ذهنی کتابداران از آیکون‌های بخش امانت و فهرست‌نویسی نرم‌افزارهای کتابخانه دیجیتال آذرخش، سیمرغ و نیکا. پژوهشنامه پردازش و مدیریت اطلاعات 33 (2): 489-516.
زره‌ساز، محمد، و رحمت‌االله فتـاحی. 1387 مبـانی نظـری و عوامـل مؤثر بـر فرایند راهنمایـابی و اسـتفاده کاربران از امکانات کمکی در نرم‌افزارهای پایگاه‌های اطلاعاتی. کتابداری و اطلاع‌رسانی 43 (3): 71-96.
_____. 1393. شناسایی و تحلیل امکانات و قابلیت‌های راهنمایی و کمک در نرم‌افزارهای کتابخانه‌های دیجیتالی ایرانی. پژوهشنامه پردازش و مدیریت اطلاعات 29 (3): 817-834.         
صفایی، زهرا، عبدالرسول خسروی، و رضا رجبعلی بگلو. 1399‌. پیشنهاد مدل مفهومی برندسازی در کتابخانه‌ها و مراکز اطلاع‌رسانی. پژوهشنامه کتابداری و اطلاع‌رسانی 10 (2): 184-207.
صفری، اعظم، حسن بهزادی، و ایرج رداد. 1396. بررسی مدل ذهنی دانشجویان کارشناسی ارشد نسبت به موتور کاوش گوگل. پژوهشنامه پردازش و مدیریت اطلاعات 32 (4): 989-1016.
کاظم‌پور، زهرا؛ مریم ناخدا، مهدیه میرزابیگی، و نادر نقشینه. 1396. رابطه مدل‌های ذهنی دانشجویان تحصیلات تکمیلی دانشگاه علوم پزشکی اصفهان با رفتار جست‌وجوی وبی آن‌ها. مدیریت اطلاعات سلامت 14(5): 2-223.
میرزابیگی، مهدیه. 1392. مدل ذهنی در پژوهش‌های رفتار اطلاع‌جویی: پژوهشی در متون. تحقیقات کتابداری و اطلاع‌رسانی دانشگاهی 47 (3): 303-323.
References:
Abam, Z., R. Fattahi, M. Parirokh, & J. Salehi Faddari. 2011. Investigating how different users interact with the web hyperlink environment and studying their performance in review search. Quarterly Journal of Library and Information Science 55 (3): 51-81. [in Persian]
Agboola, I. 2018. Development of mental model in understanding users' thought processes for the evaluation and functional enhancement of clinical decision support systems. In 2018 IEEE International Conference on Data Mining Workshops (ICDMW) (pp. 1493-1494). IEEE.
Amadieu, F., T. Van Gog, F. Paas, A. Tricot, & C. Mariné. 2009. Effects of prior knowledge and concept-map structure on disorientation, cognitive load, and learning. Learning and Instruction 19 (5): 376-386
Apuke, O. D., B. Omar, E. A. Tunca, & C. V. Gever. 2022. Information overload and misinformation sharing behaviour of social media users: Testing the moderating role of cognitive ability. Journal of Information Science 01655515221121942.‏
Arcand, M. 2010. Three Essays on Sex Differences in the Web Environment: An Information Processing Perspective. Doctoral dissertation, École des hautes études commerciales.‏
Bahreini, N. 2016. Study of students' behavior, performance, and disorientation in search of a review of Riding and Chima's verbal-visual stylistic approach. Master's thesis, Shiraz University. [in Persian]
Bi, K., P. Metrikov, C. Li, & B. Byun. 2021. Leveraging user behavior history for personalized email search. In Proceedings of the Web Conference 2021 (pp. 2858-2868).‏ DOI:10.1145/3442381.3450110. LicenseCC BY 4.0
Bolton, M. L., E. Biltekoff, & K. Byrne. 2022. Fuzzy mental model finite state machines: A mental modeling formalism for assessing mode confusion and human-machine “trust”. In 2022 IEEE 3rd International Conference on Human-Machine Systems (ICHMS) (pp. 1-4). IEEE.‏ DOI:10.1109/ICHMS56717.2022.9980746
Chen, S. Y., & R. Macredie. 2010. Web-based interaction: A review of three important human factors. International Journal of Information Management 30 (5): 379-387.‏
Chen, S. Y., J. P. Fan, & R. D. Macredie. 2006. Navigation in hypermedia learning systems: experts vs. novices. Computers in Human Behavior 22 (2): 251-266.
Conklin, J. 1987. Hypertext: An Introduction and SurvevJ. Computer 20 (9): 17-41.‏
Cooper, G. 1998. Research into cognitive load theory and instructional design at University of New South Wales, Sydney. https://education,arts.unsw.edu.au/CLT,HTML
Crabb, M., & V. L. Hanson. 2014. Age, technology usage, and cognitive characteristics in relation to perceived disorientation and reported website ease of use. In Proceedings of the 16th international ACM SIGACCESS conference on Computers & accessibility (pp. 193-200). ASSETS '14: The 16th International ACM SIGACCESS Conference on Computers and Accessibility. Rochester New York USA.
_____. 2016. An Analysis of age, technology usage and Cognitive Characteristics Within Information Retrieval Tasks. ACM Transactions on Accessible Computing 8 (3):1-26.
Demirbilek, M. 2009. Effects of interface windowing modes and individual differences on disorientation and cognitive load in a hypermedia learning environment. Doctoral dissertation, University of Florida.‏
_____. 2009. Effects of interface windowing modes and individual differences on disorientation and cognitive load in a hypermedia learning environment. Doctoral dissertation, University of Florida.
Dennis, S., P. Bruza, & R. McArthur. 2002. Web searching: A process‐oriented experimental study of three interactive search paradigms. Journal of the Association for Information Science and Technology 53 (2): 120-133.‏
Dennis, S., R. McArthur, & P. D. Bruza. 1998. Searching the World Wide Web made easy? the cognitive load imposed by query refinement mechanisms. Proceedings of the Annual Meeting of the Cognitive Science Society, 20. Retrieved from https://escholarship.org/uc/item/1fn3v9jq (accessed ??)
‏Dietrichson, A. 2001. Digital literacy: How to measure browsing behavior. Columbia university.
Dimitroff, A. 1990. Mental Models Theory and Search Outcome in a Bibliographic Retrieval System. Library and Information Science Research 14 (2): 56-141.
Dinet & Kitajima 2011. Draw me the Web”. Impact of mental model of the Web on information search performance of young users. IHM '11: Proceedings of the 23rd Conference on l'Interaction Homme-Machine Article No.: 3, Pages 1 – 7. https://doi.org/10.1145/2044354.2044358
Dong, J., R. Duan, & S. Liang. 2023. Users' mental models of cross-device search under controlled and autonomous motivations. Aslib Journal of Information Management 75 (1): 68-89.‏
Elm, W. C., & D. D. Woods. 1985. October. Getting lost: a case study in interface design. In Proceedings of the Human Factors Society Annual Meeting 29 (10): 927-929. Los Angeles, CA: SAGE Publications.‏
Eveland Jr., W. P., & S. Dunwoody. 2001. User control and structural isomorphism or disorientation and cognitive load? Learning from the Web versus print. Communication research 28 (1): 48-78.‏
Fattahi, R., M. Parirokh, M. H. Dayyani, A. Khosravi & M. Zareivenovel. 2016. Effectiveness of Google keyword suggestion on users’ relevance judgment: A mixed method approach to query expansion. The Electronic Library 34 (2): 302-314. [in Persian]
Fenichel, C. H. 1981. Online searching: Measures that discriminate among users with different types of experiences. Journal of the American Society for Information Science: Association for Information Science & Technology, vol. 32 (1): 23-32, January. DOI: 10.1002/asi.4630320104
Ferguson, C., & H. Van Oostendorp. 2020. Lost in learning: hypertext navigational efficiency measures are valid for predicting learning in virtual reality educational games. Frontiers in psychology 11: 578154.‏
Ford, N., & D. Miller. 1996. Gender differences in Internet perceptions and use. In Aslib Proceedings 48 (7-8): 183-192.
_____, & N. Moss. 2001. The role of individual differences in Internet searching: An empirical study. Journal of the American Society for Information Science and technology 52 (12): 1049-1066.‏
Foss, C. L. 1989. Tools for reading and browsing hypertext. Information Processing & Management 25 (4): 407-418.‏
Graff, M. 2004. Nationality and Gender Differences in Disorientation in web-based Learning. In ECEL2004-3rd European Conference on E-Learning: ECEL2004 (p. 93). Academic Conferences Limited.
Gwizdka, J., & I. Lopatovska. 2009. The role of subjective factors in the information search process. Journal of the Association for Information Science and Technology 60 (12): 2452-2464.‏
Harper, S., E. Michailidou, & R. Stevens. 2009. Toward a definition of visual complexity as an implicit measure of cognitive load. ACM Transactions on Applied Perception (TAP) 6 (2): 10.
Herder, E. 2004. Revisitation patterns and disorientation. In Proceedings of the German Workshop on Adaptivity and User Modeling in Interactive Systems (pp. 291-294).‏
Huang, Y-H. 2018. Influence of instructional design to manage intrinsic cognitive load on learning effectiveness. EURASIA Journal of Mathematics,Science and Technology Education 14 (6): 2653-266
Jacobson, T., & D. Fusani. 1992. Computer, System, and Subject Knowledge in Novice Searching of a Full-Text, Multifile Database. Library and Information Science Research 14 (1): 97-106.‏
Jen-Hwa Hu, P., H. Han-fen, & F. Xiao. 2017. Examining the Mediating Roles of Cognitive Load and Performance Outcomes in User Satisfaction with a Website: A Field Quasi-Experiment. MIS Quarterly 41 (3): 975-988.‏
Jones, N., H. Ross, T. Lynam, P. Perez, & A. Leitch. 2011. Mental models: an interdisciplinary synthesis of theory and methods. Ecology and Society 16 (1):?.‏
Juvina, I., & H. Van Oostendorp. 2006. Individual differences and behavioral metrics involved in modeling web navigation. Universal Access in the Information Society 4 (3): 258-269.‏
Kazempour, Z., M. Nakhoda, M. Mirzabeigi & N. Naghsheneh. 2017. The relationship between mental models of postgraduate students of Isfahan University of Medical Sciences and their web search behavior. Health Information Management 14 (5): 2-223. [in Persian]
Khosravi, A., R. Fattahi, M. Parirokh, & M. H. Diani. 2012. Analysis of users' search expansion in Google search engine based on cognitive load theory. Library and Information Science 15 (2): 3-9. [in Persian]
Kim. 2000. Cognitive style and on-line database search experience as predictors of Web search performance. Journal of the American Society for Information Science. https://doi.org/10.1002/(SICI)1097-4571(2000)51:6
_____. 2001. Information-seeking on the Web: Effects of user and task variables. Library & Information Science Research. 23 (3): 233-255. https://doi.org/10.1016/S0740-8188(01)00081-0
_____, K. S. 2009. Effects of Emotion Control and Task on Web Searching Behavior. Information Processing & Management 44: 373-385.
Kim, Y. M., & S. Y. Rieh. 2005. Dual‐task performance as a measure of mental effort in searching a library system and the Web. Proceedings of the Association for Information Science and Technology 42 (1): DOI:10.1002/meet.14504201155
Klepsch, M., T. Seufert, T. 2020. Understanding instructional design effects by differentiated measurement of intrinsic, extraneous, and germane cognitive load. Instructional Science 48: 45–77.
Kodama, C., B. S. Jean, M. Subramaniam, & N. G. Taylor. 2017. There’sa creepy guy on the other end at Google! engaging middle school students in a drawing activity to elicit their mental models of Google. Information Retrieval Journal 20 (5): 403-432.
Larson, K., & M. Czerwinski. 1998. Web page design: Implications of memory, structure and scent for information retrieval. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 25-32).
Lawton, C. A. 1994. Gender differences in way-finding strategies: Relationship to spatial ability and spatial anxiety. Sex roles 30 (11-12): 765-779.
Leong, S. C., & S. Hawamdeh. 1999. Gender and learning attitudes in using Web-based science lessons. Information Research 5 (1): 5-1.‏
Li, P. 2007. Doctoral students' mental models of a Web search engine: An exploratory study ?? (pp. 1-316).‏
Lim, S., & N. Kwon. 2010. Gender differences in information behavior concerning Wikipedia, an unorthodox information source? Library & information science research 32 (3): 212-220.‏
Link, E., & E. Baumann. 2023. A comparison of Women’s and Men’s web-based information-seeking behaviors about gender-related health information: web-based survey study of a stratified German sample. Journal of Medical Internet Research 25: e43897.‏
MacFadden, A., L. Elias, & D. Saucier. 2003. Males and females scan maps similarly, but give directions differently. Brain and Cognition 53 (2): 297-300.‏ 
Malki, L. M., D. Patel, & A. Singh. 2024. "The Headline Was So Wild That I Had To Check": An Exploration of Women's Encounters With Health Misinformation on Social Media. Proceedings of the ACM on Human-Computer Interaction, 8 (CSCW1), 1-26.‏
Marchionini, G. 1990. Making the transition from print to electronic encyclopaedias: adaptation of mental models. International journal of man-machine studies 30 (6): 591-618.
McDonald, S., & L. Spencer. 2010. Gender differences in web navigation. In Women, Work and Computerization (pp. 174-181). Boston, MA.: Springer.‏
McDonald, S., & R. J. Stevenson. 1998. Effects of text structure and prior knowledge of the learner on navigation in hypertext. Human Factors 40 (1): 18–27. https://doi.org/10.1518/001872098779480541
Michell, G., & P. Dewdney. 1998. Mental models theory: Applications for library and information science. Journal of education for library and information science 39 (4): 275-281
Mirzabeigi, M. 2013. Mental Model in Information Seeking Behavior Research: A Literature Review. Academic Library and Information Research 47 (3): 303-323.
Mishra, P., & A. Yadav. 2006. Using hypermedia for learning complex concepts in chemistry: A qualitative study on the relationship between prior knowledge, beliefs, and motivation. Education and Information Technologies 11 (1): 33-69.‏
Miyake, S., S. Yamada, T. Shoji, Y. Takae, N. Kuge, & T. Yamamura. 2009. Physiological responses to workload change. A test/retest examination. Applied ergonomics 40 (6): 987-996.
Montesi, M. 2021. Gender differences in information behavior during the Covid-19 health crisis in Spain. JLIS: Italian Journal of Library, Archives and Information Science= Rivista italiana di biblioteconomia, archivistica e scienza dell'informazione: 12 (2): 73-87.‏
Na, k. 2012. Exploring the effect of cognitive load on the propensity for query reformulation behavior. Doctoral dissertation, Florida State University.
_____. 2021. The effects of cognitive load on query reformulation: mental demand, temporal demand and frustration. Aslib Journal of Information Management 73 (3): 436-453.‏
Nori, R., & L. Piccardi. 2011. Familiarity and spatial cognitive style: how important are they for spatial representation. Spatial memory: Visuospatial processes, cognitive performance and developmental effects, 123-144. https://doi.org/10.1037/t33314-000
‏_____. Piccardi. 2015. I believe I’m good at orienting myself… But is that true? Cognitive processing 16 (3): 301-307
Norman, D. A. 1984. Some observations on mental models. Mental models 7 (112): 7-14.
Otter, M., & H. Johnson. 2000. Lost in hyperspace: metrics and mental models. Interacting with computers 13 (1): 1-40.‏
Palmquist, R. A., & K. S. Kim. 2000. Cognitive style and on‐line database search experience as predictors of Web search performance. Journal of the Association for Information Science and Technology 51 (6): 558-566.
Pandey, S., & A. D. Rao. 2019. Impact of Approach Angle of an Impinging Cyclone on Generation of Storm Surges andIts Interaction with Tides and Wind Waves. Journal of Geophysical Research: Oceans 124. https://doi.org/10.1029/ 2019JC015433
Qiu, L. 1993. Markov models of search state patterns in a hypertext information retrieval system. Journal of the American Society for Information Science 44 (7): 413-427.
Queralt Jiménez, A. 2023. Desinformación por razón de sexo y redes sociales Gendered disinformation and social networks. International Journal of Constitutional Law 21 (5): 1589-1619.‏
Rahrvani, S., M. Mirzabeigi, & J. Abbaspour. 2017. Studying the level of completeness of librarians' mental model of the icons of the lending and cataloging sections of the digital library software Azarakhsh, Simorgh, and Nika. Journal of Information Processing and Management 33 (2) 489-516. [in Persian]
Reisoğlu, İ., A. Çebi, & T. Bahçekapılı. 2022. Online information searching behaviours: Examining the impact of task complexity, information searching experience, and cognitive style. Interactive Learning Environments 30 (3): 417-434.‏
Saadé, R. G., & C. A. Otrakji. 2007. First impressions last a lifetime: effect of interface type on disorientation and cognitive load. Computers in human behavior 23 (1): 525-535.‏
Safari, A., H. Behzadi, & I. Radad. 2017. Investigating the mental model of master's students towards the Google search engine. Journal of Information Processing and Management 32 (4): 989-1016. [in Persian]
Sanchiz, M., J. Chin, A. Chevalier, W. T. Fu, F. Amadieu, & J. He. 2017. Searching for information on the web: Impact of cognitive aging, prior domain knowledge and complexity of the search problems. Information Processing & Management 53 (1): 281-294.‏
Shih, Y. C., P. R. Huang, Y. C. Hsu, & S. Y. Chen. 2012. A complete understanding of disorientation problems in Web-based learning. Turkish Online Journal of Educational Technology-TOJET 11 (3): 1-13.‏
Shneiderman, B. 1987. Designing the user interface strategies for effective human-computer interaction. ACM SIGBIO Newsletter 9 (1): 6.‏
Smith, P. A. 1996. Towards a practical measure of hypertext usability. Interacting with computers 8 (4): 365-381.‏
Staggers, N., & A. F. Norcio. 1993. Mental models: concepts for human-computer interaction research. International Journal of Man-machine studies 38 (4): 587-605.
Sweller, J. 1988. “Cognitive Load during Problem Solving: Effects on Learning,” Cognitive Science 12 (2): 257-285.
_____. 2020. Cognitive load theory and educational technology. Education Tech Research Dev 68: 1–16
Taylor, A., & H. A. Dalal. 2017. Gender and information literacy: Evaluation of gender differences in a student survey of information sources. College & Research Libraries 78 (1): 90.‏
‏ Thatcher, A. 2006. Information seeking behaviors and cognitive strategies in different search tasks on the WWW. International Journal of Industrial Ergonomics 36 (12): 1055-1068. DOI:10.1016/j.ergon.2006.09.012
Thomas, P., B. Billerbeck, N. Craswell, & R. W. White. 2019. Investigating searchers’ mental models to inform search explanations. ACM Transactions on Information Systems (TOIS) 38 (1): 1-25.‏
Turcotte, J., & B. Oddson. 2022. Managing Increased Cognitive Load in a Guided Search. Perceptual and Motor Skills 129 (3): 378-398.‏
Vuong, T., M. Saastamoinen, G. Jacucci, & T. Ruotsalo. 2019. Understanding user behavior in naturalistic information search tasks. Journal of the Association for Information Science and Technology 70 (11): 1248-1261.‏
Zerehasaz, M., & R. Fattahi. 2008. Theoretical foundations and factors affecting the process of guidance and users' use of help facilities in database software. Library and Information Science 43 (3): 71-96. [in Persian]
_____. 2014. Identification and analysis of guidance and assistance features and capabilities in Iranian digital library software. Journal of Information Processing and Management 29 (3): 817-834. [in Persian]
Zhang, X. 1998. A study of the effects of user characteristics on mental models of information retrieval systems. Doctoral dissertation, National Library of Canada= Bibliothèque nationale du Canada.‏
_____. 2012. The construction of mental models of information-rich web spaces: The development process and the impact of task complexity. Doctoral dissertation, The University of North Carolina at Chapel Hill.‏
_____, Yiyang, & J. Yang. 2022. [Retracted] Research on Information Retrieval Effectiveness of University Scientific Researchers Based on Mental Model. Wireless Communications and Mobile Computing, https://doi.org/10.1155/2022/5852946)
Zheng, G. 2015. Web navigation systems for information seeking. In Encyclopedia of Information Science and Technology, Third Edition (pp. 7693-7701): IGI Global. DOI: 10.4018/978-1-4666-5888-2.ch758
Zhu, M., T. Yasseri, & J. Kertész. 2024. Individual differences in knowledge network navigation. Scientific Reports 14 (1): 8331.‏
Zolfaghari, M. 2016. Investigating students' review performance based on experience and level of mental model completeness. Master's thesis, Shiraz University. [in Persian]
Zuo, S., Q. Yin, H. Jiang, S. Xi, B. Yin, C. Zhang, & T. Zhao. 2022. Context-Aware Query Rewriting for Improving Users' Search Experience on E-commerce Websites. arXiv preprint arXiv: 2209.07584.‏

  • Receive Date 11 April 2023
  • Revise Date 09 November 2024
  • Accept Date 10 November 2024