The purpose of this article is to evaluate and to rank the usability components in OJS, Sinweb, Yektaweb and Neoscriber3 journal management systems, as well as to observe each component measure on each system separately.
The survey population consists of users of 4 most used systems such as Sinaweb, Yektaweb, and Neoscriber among 13 OJS at Iranian medical sciences universities. Evaluation of each interface was based on 4 layers of editors, reviewers, authors, and regular users. Four layers were selected for each system and 5 users for each layer in each system. In total, 80 users completed the research questionnaires. Data were analyzed by SPSS software using descriptive and inferential statistical tests.
The results show that the consistency component in Sinaweb system was lower than Yektaweb, OJS and Neoscriber 3 systems, while it was not significantly different in the other three systems. The flexibility average of the systems interface was below middle, the user guidance average of the systems was in the middle and other areas were above average. However, averages showed that consistency of Yektaweb is more than neoscriber 3 and then Neoscriber 3 is more than OJS. Also, there were no significant differences between the four systems in other components including compatibility, flexibility, learnability, minimal action, minimal memory load and perceptual limitation.
From the users’ point of view, it seems that the most attention is in the components of compatibility and consistency, and the least attention is in the components of flexibility and user guidance among the journal management systems studied. Also, the most important weaknesses include direct manipulation and apply settings in the system, lack of use of shortcuts and replacements keys, etc. and the strengths, including the use of consistent icons on different pages, familiarizing users with concepts and terms used in the system, etc. These strengths and weaknesses make it easier to select the appropriate systems for other scientific organizations and centers. Weaknesses can also be developed by system administrators and technical teams to gain desirable usability and find ways to attract the attention of other countries’ journals as well as internationalization. Moreover in this study, a new tool (not used in Iran) was introduced to be used in similar studies by librarians and system managers to select various kinds of information systems based on the applicability criteria.