Usability Report of the Learning Management System (LMS) OLAT by Ferolun Mitador

Usability Testing and Evaluation, and Redesign Recommendations for the LMS OLAT

Prof. Dr. Sissel Guttormsen Stefan Minder BĂŠatrice Boog University of Bern Institute of Medical Education (IML) ÂŠ IML 2007

commissioned by the University of Zurich Multimedia and E-Learning Services (MELS)

Report

ÂŤTo see is to forget the name of the things one sees.Âť Robert Irving

Table of Contents

1. 2. 3. 4. 5. 6. 7.

Executive summary............................................................................................................p. 2 Description of the mandate...............................................................................................p. 4 Goals of improved usability in an LMS...............................................................................p. 6 Use cases and their derived tasks.....................................................................................p. 10 Description of the LMS OLAT ...........................................................................................p. 14 Expert evaluation - Method................................................................................................p. 16 Expert evaluation - Results................................................................................................p. 18 7.1 Main obstacles found in the role of a student........................................................p. 18 7.2 Main obstacles found in the role of an author........................................................p. 24 7.3 Interaction design issues.........................................................................................p. 30 8. Usability testing in the usability laboratory - Methods......................................................p. 46 9. Usability testing – Results.................................................................................................p. 60 9.1 Students.................................................................................................................p. 60 9.2 Authors...................................................................................................................p. 67 9.3 First steps for novice authors.................................................................................p. 70 9.4 The test persons assessed OLAT on an ISO 9142-10 based questionnaire...........p. 71 10. Redesign - Reflection and recommendations...................................................................p. 74 10.1 About redesign dimensions and redesign depth...................................................p. 74 10.2 Identified redesign areas.......................................................................................p. 77 10.3 Redesign recommendations..................................................................................p. 80 10.4 Self-evidence, self-explanatoriness, obscurity......................................................p. 102 10.5 Redesign recommendations – Epilogue.................................................................p. 103 11. Doing successful business with novice authors with a wizard.........................................p. 104 12. Reflections about the opportunities and dangers in using AJAX.....................................p. 108 Acknowledgments..................................................................................................................p. 113

01 - Executive Summary 1. Executive summary In the report on hand, we present the results of our usability investigation of the Learning Management System OLAT, as well as the redesign recommendations inferred from the analysis. The aim was to detect main usability obstacles in two user roles: author and student, and to develop design alternatives that would eliminate these problems while being minimally invasive, i. e. without changing the basic concepts of OLAT. The investigations were conducted in the form of heuristic expert evaluations and usability testing with test persons, the latter run in the Usability Lab of the Institute of Medical Education in Bern. The test persons performed predefined tasks in the OLAT system, emulating real-world situations. These tasks had been derived from use cases developed in cooperation with the OLAT team of the University of Zurich. The emerging usability problems were first assigned to one of the two domains, screen design, and interaction design; then, they were graded for the impact of the necessary redesign. Usability problems that were unsurmountable for the test persons are discussed in particular. We have formulated our suggestions for redesign on two levels: firstly in relation to the existing OLAT concept, and secondly we have suggested design principles that shall support the application at the gained insight to the OLAT system in general. In addition, the possibilities for the use of a wizard for novice authors are presented, and Cartoon 1: By Hans Holzherr.

Usability Testing and Evaluation, and Redesign Recommendations for the LMS OLAT. Report ÂŠ 2007 University of Bern, Institute of Medical Education (IML)

01 - Executive Summary opportunities and dangers of the application of new (interaction) technologies such as AJAX are discussed. The return on investment following an OLAT usability improvement with the immediate aim to eliminate the found obstacles, especially those that the test persons could not overcome, is rooted in an increase in efficiency and subsequent gain in OLAT acceptance by the users, frequency of use, and image. Good usability together with a richness in adequate features facilitates a strong market position.

Usability Testing and Evaluation, and Redesign Recommendations for the LMS OLAT. Report ÂŠ 2007 University of Bern, Institute of Medical Education (IML)

02 - Description of the Mandate 2. Description of the mandate After more than 7 years of development and evolution OLAT has become a highly elaborate and widely used learning management system. To meet high-quality standards, every aspect of OLAT sticks to clear concepts and visions. One important aspect is the usability of the system, which was the subject of our mandate. The MELS department of the University of Zurich commissions the Institute for Medical Education (IML) of the University of Bern to conduct an expert evaluation as well as a standard usability test within the LMS OLAT. The expert evaluation’s goal is to uncover deviations from validated principles of interaction and design. The usability test aims at pointing out flaws from the target users’ perspective who in the test assume the roles of student and author. For the test, MELS’ OLAT team shall define use cases for students, as well as for authors. In cooperation with IML’s team, tasks shall be derived from the use cases which are then performed by the test persons in the OLAT usability tests. The use cases and their derived tasks shall represent the typical usage of OLAT although they can capture the system only partially. The usability tests shall be conducted in IML’s usability lab as a structured test routine in single sessions with test persons whose profiles correspond to the two target groups (students and authors). Ten or more test persons shall be recruited by the IML from the environment of the Usability Testing and Evaluation, and Redesign Recommendations for the LMS OLAT. Report © 2007 University of Bern, Institute of Medical Education (IML)

02 - Description of the Mandate University of Bern. The usability test shall be conducted, recorded, and interpreted by the IML.

Hypothesized entirety of usability problems (UPs)

UP15

UP9

UP14 UPs detected during usability testing

UPs detected during UP7 expert evaluation UP8 UP2 UP3 UP6 UP5 UP4 UP11

UP13

UP18

UP10 UP12

UP19

The results of the evaluations shall be presented in a usability report and submitted with complementary video summaries from the usability tests. The report contains an overview of the methods, a summary of the results, and selected redesign recommendations which can be generalized, and therefore, are applicable to existing and new screens and interactions in OLAT (Figure 1).

UP16

UP17

Synthesis

Redesign recommendations Figure 1:

Detecting usability problems with expert evaluation and usability testing with test persons. The two methods complement one another and build the fundament for developing redesign recommendations.

Usability Testing and Evaluation, and Redesign Recommendations for the LMS OLAT. Report ÂŠ 2007 University of Bern, Institute of Medical Education (IML)

03 Goals of Improved Usability in a LMS 3. Goals of improved usability in an LMS The first criterion which is usually looked at when evaluating a Learning Management System is the feature list. What can the system do? Among the features, some of them are more vital than others, e. g. compatibility with the SCORM-standard is central. Further questions often address technical qualities around compatibility, stability and economic aspects of acquisition, operation and maintenance. Usability - an anology When we compare the considerations above with the thoughts entertained before purchasing a car, there are obvious similarities, namely evaluation of technology, features and costs. When the basis of the usability (high or low) is addressed, i. e. ease-of-use, the comparison of evaluating an LMS versus evaluating a car shows a major difference: While the ways of the basic operation (driving) of a car is set and limited (the pedal to the right is the accelerator pedal) thanks to a long history of car development (including ongoing usability considerations), the possibilities of making control elements available on a screen and using them are seemingly endless and cannot rely on such a long tradition (self-evidence through common sense).

The possibilities of making control elements available on a screen and using them are seemingly endless

When common sense is met, it is a big step toward good usability. We experience usability problems with features Usability Testing and Evaluation, and Redesign Recommendations for the LMS OLAT. Report ÂŠ 2007 University of Bern, Institute of Medical Education (IML)

03 Goals of Improved Usability in a LMS lacking common sense: where is the reverse gear and where the switch for the low beam? We summarize the two causal relations identified so far: 1. The higher the complexity of a system, the more crucial the usability to make successful handling possible 2. The lesser a common culture of handling a system is established, the more crucial is the usability to overcome lacking know-how. The level of know-how differs among the users of an LMS, and the users’ know-how usually grows during usage. Promoting this growth is a quality aspect of the usability. From this point of view, users can be roughly divided in two groups: novices and experts. High usability supports the process of novices becoming experts. Obviously, novices have other needs than experts, which is often seen as a dilemma: while the division of work steps into smaller, sequenced steps with many cues available is good for novices, experts are more efficient with having several work steps displayed at once and the choice of deciding the order of the steps on their own. Experts want to choose from several methods of navigation (e. g. clicking versus shortkeys) whereas novices should be guided more linearly. From the considerations above we can derive the first goals of improved usability in an LMS:

03 Goals of Improved Usability in a LMS Besides effectiveness and efficiency, ISO 9241-10/11 , states Effectiveness and satisfaction as a third key for usability. One dimension of efficiency of use for satisfaction is the customer getting what he/she expects. novices and experts Imagine the customer (user) having the feature list (from the catalogue) in mind as a first time user: Firstly, he or she expects the features to be clearly visible and accessible (selling proposition), and secondly, they must be easily applicable. Generally spoken: whenever a user interacts with the LMS, he/she expects certain reactions from the system. Another user classification besides novices and experts is Keeping the customer based on their roles in the LMS: there are authors bringing satisfied = meeting in the content and the students using the LMS as learning his/her expectations resource. OLAT is an LMS, which means that one of the end usergroups are learners. Learners should be supported in doing their first priority fast, i. e. to learn. Radically, all other activities demanding their mental capacity should be reduced. This sets the usability demands for OLAT in a special context. The users do experience the high or low usability of an LMS during focused states of mind: planning to learn, intention to learn, and learning. Under the assumption that the mental capacity of humans is limited , , cognitive

ISO 9241-10 (1996). Ergonomic requirements for office work with visual display terminals. Dialogue principles ISO 9241-11 (1998). Ergonomic requirements for office work with visual display terminals. Guidance on sability Sweller J. (1994 ). Cognitive load theory, learning difficulty and instructional design. Learning and Instruction Sweller J. (2005 ). Implications of cognitive load theory for multimedia learning. In R. E: Mayer (ed.) Cambridge, Handbook of multimedia leaning. Cambridge: UK: Cambridge University Press

Usability Testing and Evaluation, and Redesign Recommendations for the LMS OLAT. Report ÂŠ 2007 University of Bern, Institute of Medical Education (IML)

03 Goals of Improved Usability in a LMS load in LMS-usage must be managed: the less cognitive load (external cognitive load) is produced by handling the LMS’ technology (e. g. navigating), the more cognitive capacity can be allocated for handling the learning subjects themselves and acquiring new knowledge within them. The authors can allocate their mental capacities particularly to handling the LMS and sometimes to simultaneously creating the content, if it isn’t already prepared just to be imported. Though there is no cognitive load produced by learning the subject itself, other limited resources of the authors must be spared. We can assume, that very often, the time budget of authors is rather strapped as well as – or, therefore – their willingness to invest energy in learning a system.

Good usability reduces external cognitive load

Good usability reduces authors’ time usage

Furthermore, the ability of an LMS to economize author resources is an advantage in competition if we see a certain LMS competing with other LMSs or with other methods of managing, distributing and teaching content. Nowadays, good usability is a sales argument as valuable as the appropriate feature list. It is less easy to promote in a catalogue, but makes successful and satisfied users while saving time and money.

Good usability secures a market advantage

04 Use Cases and their Derived Tasks 4. Use cases and their derived tasks

Use Case

Derived Task 1

Derived Task 2

Derived Task 3 Figure 2a: Tasks are derived from use cases. The former are interactionsequences of smaller granularity, the later are scenario based larger workflow units.

The use cases to be investigated were identified in collaboration with the OLAT Usability Group of the University of Zurich, Switzerland. The use cases were selected according to the following priorities: • Use cases relevant to common authors and students in every-day use of OLAT • Use cases allowing the deduction of tasks that can be put in an expedient order (plot) • The derived tasks shall represent a typical use-inten tion • The wording of the derived tasks shall be neutral, representing a possible plan in mind of a novice user (author or student) without specific handling hints. The neutrality of the wording can be crosschecked: execution in other LMS’s should be possible without major re-wording, provided the addressed features are present • The granularity of the tasks does not have to be uniform, but their completion must be clearly identifiable • The use cases and their derived tasks must be suitable for expert evaluation and usability testing in the usability laboratory • The tasks may depend on each • It should be possible to complete the tasks without expert system knowledge, and if needed, with standardized help provided by the test administrators

04 Use Cases and their Derived Tasks Although the selected use cases represent a sample of some central features and functionality only, they are sampled to represent all parts of the system. Hence, the findings will be widely representative (Figure 2a). Use cases and their derived tasks for students Use Case 1: getting started Derived tasks: 1. Log-in to OLAT (T1C1s) 2. Check your personal settings and correct your email address if necessary (T2C1s) 3. Indicate your gender (T3C1s) 4. Ensure that your e-mail address is published on your VCard (T4C1s) Use Case 2: reading and learning course content Derived tasks: 5. Find the course „Ozeane der Erde XL“ (T5C2s) 6. Read the course’s introduction and save the pdf to your hard-disk (T6C2s) 7. Bookmark the course in OLAT (T7C2s) 8. Upload the pdf from the CD-ROM to your personal folder in OLAT (T8C2s)

04 Use Cases and their Derived Tasks Use Case 3: using collaborative tools in OLAT Derived tasks: 9. Find your fellow student (and OLAT member) Minder and download the pdf he has published (T9C3s) 10. Establish a study group in OLAT’s collaborative tool (T10C3s) 11. Subscribe Samuel Schluep to this group (T11 ) 12. Book a group meeting in the group calendar (T12C3s) C3s

creating a new course 1: Logging in to OLAT. 2: Create a new course you can choose any title and description. 3: Add a welcome page to your course. 4: Add a folder to your course. Ensure that you are the only one allowed to upload. 5: Upload two pdf files to the course folder. 6: Add a discussion forum to your course where your students may ask questions. Figure 2b: An exemplary use case: An author intends to create a new course. The consequential tasks are shown.

Use Case 4: interacting with the cours content Derived tasks: 13. Go back to the course „Ozeane...“ (T13C4s) 14. Work through the chapter „pacific ocean“ and assess your knowledge in the MC-Test (T14C4s) 15. Answer Beatrice’s question in the forum (T15C4s) 16. Post your own question in the forum (T16C4s) 17. Rate the course in the online evaluation form (T17C4s) Use cases and their derived tasks for authors Use Case 1: creating a new course (Figure 2b) Derived tasks: 1. Log in to OLAT (T1C1a) 2. Create a new course. You can choose any title and description (T2C1a) 3. Add a welcome page to your course (T3C1a)

04 Use Cases and their Derived Tasks 4. Add a folder to your course. Ensure that you are the only one allowed to upload to this folder (T4C1a) 5. Upload the two pdf files to the course folder (T5C1a) 6. Add a forum to your course where your students can ask questions (T6C1a) Use Case 2: publishing a course Derived tasks: 7. Publish the course (T7C2a) 8. Adjust the general course settings: All registered OLAT Users shall have access (T8C2a) 9. Check if your students are allowed to access the course (T9C2a) Use Case 3: re-editing an existing course and republishing Derived tasks: 10. Post a welcome message to the forum (T10C3a) 11. Change the access property to the forum so that it is visible from the day after tomorrow (T11C3a) 12. Add a wiki to your course (T12C3a) 13. Publish the changes made (T13C3a)

Usability Testing and Evaluation, and Redesign Recommendations for the LMS OLAT. Report ÂŠ 2007 University of Bern, Institute of Medical Education (IML)

05 Description of the LMS OLAT 5. Description of the LMS OLAT Registration / Zugang, Rechte

OLAT Betrieb:

Ablage:

Lernen / Lehren Verwalten

Lernressouren • Kurse • Tests • Fragebogen • Lerninhalt

Produktion mit Werkzeugen: Lernmaterial und Kurse

Suchen Katalogisieren

Gemeinsamer Raum: Projektgruppen

Privater Raum: Home mit Einstellungsmöglichkeiten und Diskspace

Kursübergreifende Gruppenverwaltung Systemadministration Figure 3:

Schematic representation of the OLAT’s most important features. Source: http://www.olat.org

Source: http://www.olat.org OLAT is the acronym for Online Learning And Training. It is a web application – a so-called Learning Management System that supports any kind of online learning, teaching, and tutoring without any didactical restrictions (Figure 3). OLAT is open source, 100% Java. The development of OLAT started 1999 at the University of Zurich where it is the strategic learning management system deployed on the main OLAT server. The University of Zurich leads the further development with a team of 12 developers pushing OLAT to the next level. With version 3.0 the system has been completely rebuilt and is now available as a component-oriented application developed in Java programming language. OLAT supports various e-learning standards such as IMS Content Packaging, IMS QTI, and SCORM. With version 4.0 many extension points have been added to the system which makes it very easy to extend the LMS functionality. With version 5 new functions like Wiki, Calendar, AJAX Beta Mode, Full-text Search and many other new features have been added. OLAT is multilingual (full support of UTF-8 and the following languages: German, English, French, Italian, Spanish, Czech, Danish, Greek, Polish, Chinese, Lithuanian, Persian/Farsi, Portuguese and Russian) and integrates the instant messaging system Jabber to support synchronous communication processes and to extend the users’ awareness.

OLAT is the acronym for Online Learning And Training OLAT is open source, 100% Java

05 Description of the LMS OLAT OLAT key features (selection from olat.org) General • Unlimited number of user accounts, courses and groups • Multilingual • Right-based security • Virtual file system • Shibboleth authentication and authorization • File sharing disk space for each user OLAT Course system and learning resource repository • Swift creation of courses with the OLAT course editor • Course structure easily configurable with course elements like: · Structure · IMS Content Packaging · SCORM content (SCORM 1.2 RTE full support) · Local content (via file upload or WebDAV, integrated WYSIWYG content editor) · External content (image database, web application) · IMS QTI test, self-test and questionnaire · Task (selection of multiple tasks, drop-in-box, manual scoring) · Manual scoring · Enrolment in groups · Discussion forum, exportable to RTF · File dialog (discussion forum per uploaded file) · Chat · File share folder · Contact form · Wiki Groups, Assessment and Communication, Testing system Usability Testing and Evaluation, and Redesign Recommendations for the LMS OLAT. Report © 2007 University of Bern, Institute of Medical Education (IML)

06 Expert Evaluation - Method 6. Expert evaluation - Method Suitibility for individualization The system gave a feedback whenever I made a mistake If yes: the error messages helped me solve the problem

Suitability for the task Selfdescriptiveness

ISO 9241-10

error tolerance

Conformity with user expectations

Ergonomic of human system interaction

Suitibility for learning

For expert evaluation, the method based on heuristics (heuristic usability evaluation ) has been chosen to achieve a high standardization in data collection (figure 4). Four experts perform the tasks introduced in chapter 4, at first in the student’s role. Before performing the author tasks, the experts watched a 2.5 minutes flash movie presenting the main sections of OLAT. The same introduction will be shown to the author test persons before user testing. The flash movie is available at:

For expert evaluation, the method based on heuristics has been chosen

http://usability.unibe.ch/olat/ Controllability

username: olatstakeholders password: useresults

Figure 4:

ISO 9241-10 defines 7 principles which may serve as basis for heuristic evaluation. The 7 principles and some corresponding heuristics are shown.

The experts were briefed to take 12 heuristics derived from ISO 9241-10 as counterpart for their dialectic thoughts and to write down conclusions in free text. The notes shall record firstly the main obstacles found, and secondly, wider perspectives on the interaction design: which use cases or interaction steps offer a clear guidance, and what’s more aberrant.

Mack, R. L., and Nielsen, J. (1993). Usability inspection methods. ACM SIGCHI Bulletin 25, 1 (January), 28-33 ISO 9241-10 (1996). Ergonomic requirements for office work with visual display terminals. Dialogue principles.

06 Expert Evaluation - Method In other words: the focuses are the ease-of-use and the degrees of self-evidence and self-explanatoryness, , the user shall be able to answer to following questions: 1. “where am I?” 2. “how did I get here?” 3. “how can I go back?” 4. “what’s the next step?”

07 Expert Evaluation - Results 7. Expert evaluation - Results 7.1 Main obstacles found in the role of a student The following problems were found. The problems are presented according to the order of the use cases and are not ranked according to severity. Task T5C2s

Experts’ notes

“Find the course “Ozeane der Erde XL” Make the loupe icon in the meta navigation interactive and move it to the correct position.

A task of the student role was to read the introduction of the course “Ozeane XL”. The experts used the search field to find the course and realized that the only way to start the search was hitting the enter key. The loupe icon to the left of the search field is not clickable and there is no button to the right of the search. It is becoming a standard in many applications that icons are click-able (in particular the search loupe), hence, this is not implemented according to user-expectations.

07 Expert Evaluation - Results

Task T6C2s

“Read the course’s introduction and save the pdf to your hard-disd

Experts’ notes

Indicate clearly (with a comprehensive error message) if access to a course is only possible after inscription.

Access to the course “Ozeane XL” is only possible after the student has filled in the inscription form in the inscription module. All Experts tried to access the course directly and received an error message «Information: Dieses Kurselement ist nicht zugänglich». The error message did not help resolve the problem, but the experts quickly found out that inscription was necessary by trial and error. In the results of usability testing with test persons we will see that some students needed much longer (see chapter 9) to find this solution.

All Experts tried to access the course directly and received an error message

Principle: Error messages must have three dimensions formulated in a non-technical language (figure 5):

Error message 1.

What is wrong?

How can I remediate?

3. Figure 5:

debug

1. What went wrong, what was the error? 2. What must be done to resolve the error? . A link to the editor-position where the problem can be resolved is offered The second part of the error message (What must be done to resolve the error?) must be a clear instruction containing the next step to take (the next click), and the wording used must be the same as the wording used for the respective controls on the screen.

The 3 dimensions of a good and helpful error message.

07 Expert Evaluation - Results Task T7C2s

Experts’ notes

“Bookmark the course in OLAT” Make it clear how to access the bookmarks set in the OLAT environment.

The experts where asked to set a bookmark for the course. One expert stated that it was unclear afterward how to retrieve the bookmark. If the list of bookmarks was available in the meta-navigation (near the search field), the handling would resemble a browser environment: the bookmark list can be accessed from anywhere. The experts identified a problem of distinguishing the OLATinternal bookmarks from the browser-internal bookmarks. This was confirmed during usability testing with test persons: some compounded the use of these two bookmark lists.

Figure 6:

OLAT’s principle that actions are offered as buttons is broken here: “Hinzufügen” is designed as a link.

Task T8C2s

“Upload the pdf from the CD-ROM to your personal folder in OLAT”

Experts’ notes

Evaluate the interaction concept for file upload. Offer a classical interaction concept in order to support all kinds of users, offer «modern» interaction concept - in addition - for expert users.

The experts identified a problem of distinguishing the OLAT-internal bookmarks from the browser-internal bookmarks.

The experts should upload a file to the personal folder (Persönlicher Ordner). This was accomplished easily. However, the experts stated that they expected a button for the add file (Hinzufügen) transaction rather than an icon/word-combination that fades in three options: upload file, create folder, create document. This was not classified as a severe usability problem; but this interaction design used on the page mentioned is out of concept of what is encountered elsewhere in OLAT: actions are offered as buttons, and the wording contains a verb (figure 6).

07 Expert Evaluation - Results «...metaphors bundle complex relationships. ... The system of cross references provided by the metaphor satisfies the orientation needs...» The navigation pattern found on the upload page mentioned above prompted some discussion among the experts of pros and cons of using a wide or narrow set of interaction/screen design elements and metaphors. They stated that there is a trade-off existing: the elements/metaphors which are best known and recognized by users are usually the older and less stylish ones than more modern solutions, for example, a classic grey button versus a context menu to be called by hovering over or even right-mouse-key click. If the design concept of a system allows a wide set of interaction/screen design elements and metaphors, it offers opportunities to introduce new “modern” solutions without redesigning every existing page. The severe disadvantage is that users must learn and handle these varieties. This problem becomes very vital when interaction based on AJAX-technology is added to a system (see chapter 12).

Figure 7:

Usually, users know the name, not the username of fellow students. Therefore, the name field should be displayed in the first place.

Task T9C3s

“Find your fellow student (and OLAT member) ‘Minder’ and download the pdf he has published”

Experts’ notes

Improve the usability of the «search a member» feature. Offer better support for user search.

The metaphors best known are usually older and less stylish ones than more modern solutions

In task “Find your fellow student (and OLAT member) Minder” the experts searched for another OLAT user to download a file offered in his/her folder. One expert men

Konersmann, R., Noever, P., Zumthor, P. (2006). Zwischen Bild und Realität. gta Verlag, ETH Zürich.

07 Expert Evaluation - Results tioned that the first field in the search form should contain the name and not the username because the search of a username is more rare than the search of a real name; if the searching user doesn’t recognize the label of the first field, he or she fails to find the other person, except if the username is the same as the real name (figure 7). Furthermore, the action “select (wählen)” on the hit list should be a button according to the general principle in OLAT ”actions are executed through buttons; their wording is/contains a verb”. Two sub-optimalities were put on record by two experts: a) The file to be downloaded from the person found could not be located by using the general OLAT-search and b) the subfolders in the public folder of the person found were sorted at random every time a subfolder was clicked. That led to the artefact that, when browsed systematically through them, one could be missed. Task T12C3s

Experts’ notes

Actions are executed through buttons; their wording is/contains a verb

“Book a group meeting in the group calender” Improve the calendar handling / interaction.

In task «Book a group meeting in the group calendar» the use of a group calendar was asked. The expert stated that distinguishing the personal from the group calendar is unclear and that an invitation of members should be automatically offered whenever a group meeting is initiated in the group calendar. In further development of OLAT’s calendar utilities, well-established booking and invitation procedures of existing calendar software should serve as model because users have widely adopted these and are confused if the procedures are aberrant.

The expert stated that distinguishing the personal from the group calendar is unclear

07 Expert Evaluation - Results Principle: Implicit user guidance: GUI common sense

Common sense met by the GUI

Explicit user guidance: on-screen instructions

Using common interaction patterns as a model for OLAT. The ease of use depends on the knowledge brought on by the users. Nevertheless, this knowledge depends on the users’ profiles and culture, and the use of common interaction patterns increases the probability of successful control through users, and supersedes on-screen instructions (figure 8).

Necessity of on-screen instructions Figure 8:

The less self-evident a GUI is (through common sense met) the more on-screen instructions become imperative to settle at least for self-explanatory.

Task T14C4s

“Work through the chapter ‘pacific ocean’ and assess your knowledge in the MC”

Experts’ notes

No problems.

In the course “Ozeane XL” the experts filled in the selfassessment test. There were no severe usability problems found. A linear navigation (next page button) is eligible. Task T15C4s Task T16C4s

Experts’ notes

“Answer Beatrice’s question in the forum” “Post your own question in the forum” Evaluate the properties of the forum related to the users status.

During the use of the forum one expert recognized that the forum content can be read as if not yet registered in the course, but that the buttons (create message, answer a. s. o.) are missing. For unregistered users it is unclear why they can read but not write. Furthermore, there is a thread view missing (one heading per line, follow-up indented), so that long discussions remain well-arranged.

07 Expert Evaluation - Results 7.2 Main obstacles found in the role of an author Authors need some basic information about OLAT to be able to create courses in the system. This information was presented as a flash movie. OLAT itself offers this information in the help section. As reading the help document is too time-consuming, an introduction for novice authors should be easily available on the first screen without interrupting expert users.

Figure 9:

This screen makes a straight forward transaction, filling out and sending a form, intricate: There are 3 titles, 6 labels, 6 buttons, 5 icons.

Task T2C1a

“Create a new course. You can choose any title and description”

Experts’ notes

Offer easily accessible introduction to the author functions. Improve the layout- and interaction concept of the learning resources.

The first overlay (figure 9) asks for course title, description and an optional image. Although this is a form with only three fields, information to be handled by the author is very dense: the page is in an overlay containing a tabbed pane, but with only one tab. There are 3 titles, 6 labels, 6 buttons, 5 icons, two of them clickable, and all of them with text on hover. The word “course (Kurs)” is displayed at 4 different positions. When the optional picture is uploaded before title and description are saved, the latter are lost. The experts stated that it is unclear what the description is for (Where is it displayed? Is it indexed by the search engine?). When having saved title and description, the system changes the color of the save-button from

There are 3 titles, 6 labels, 6 buttons, 5 icons, two of them clickable, and all of them with text on hover

07 Expert Evaluation - Results orange to grey but does not give further instructions. The only way to proceed is to click the next-(Weiter)-button at the bottom of the overlay. Task T3C1a

Experts’ notes

Figure 10: Dispersed status representation: The “Willkommen” page has no problems. This is indicated by the green triangle to which refers the key on the bottom of the page. Furthermore, there is a tick-off mark at the top of the page.

“Add a welcome page to your course” Reconsider the interaction concept for meta-data. The path to be followed in order to create page concept is misleading. Improve the status representations of the system.

Then the expert evaluators were asked to create a welcome page as first element of the course. The meta-data (Beschreibung) of this page has to be recorded in a form asking for a short title, title and description. Again the experts stated that it’s unclear what this data is used for. The page content itself has to be created or imported under the fourth tab, called “HTML-Page (HTML-Seite)”. There, the button “choose file (Datei auswählen)” has to be clicked even if the author wants to create a new page. The experts noted that the wording in this area has to be reconsidered. After having created the welcome page, the experts were confronted with the different status representations of the system (figure 10). They noted that the status representa- The status representation is too complex tion is too complex and too disperse: The welcome page and too disperse is now mentioned in the content navigation (left column) carrying the “single-page” icon, this icon features a green arrow. This green arrow is explained in the key at the bottom of the screen. Furthermore, there is a tick-off mark 25 with the text “No problems or errors found in this course” on the top of the page; when clicked, more text is displayed (“This course does not contain any errors in the editor.”).

07 Expert Evaluation - Results Task T4C1a

“Add a folder to your course. Ensure that you are the only one allowed to upload to this folder”

Experts’ notes

No problems

The authors were able to add a folder, a file and a forum without any obstacles. They stated that the ease of the workflow in this phase was very convincing. Tasks T7C2a and T8C2a

“Publish the course” “Adjust the general course settings: All registered OLAT Users shall have access”

Experts’ notes

Make the transactions necessary for publishing a course more transparent.

The next task was to publish the course. This could be done without any obstacles but led to the misunderstanding that the course was now available to the target students. The evaluating experts had to be exactly instructed that a second transaction must be performed, so that the course is really published. This transaction has to be performed outside the editor. The expert evaluators noted that this is a fourth-grade obstacle which hardly can be overcome by trial and error; rather, special knowledge is necessary (chapter 10.4; ‘Assessing the degrees of self-explanatoriness versus obscurity’). When discussing the secondary publishing transaction with the expert evaluators the problem became obvious that the awareness about “where I am in OLAT” gets lost because they had do close the editor, ending up on a meta-data page of the course, called “Detailansicht”. On this meta-data page, there is indicated that only the owner can access the course, but the edit opportunity of this state is a good mouse movement

A fourth-grade obstacle which hardly can be overcome by trial and error

07 Expert Evaluation - Results away in the toolbox: “Change course settings (Einstellungen ändern)”. Principle: Fourth-degree obstacles are usability problems which can only be overcome by users acquiring specialized knowledge. Such obstacles have to be imperatively avoided (chapter 10.4). Task T9C2a

“Check if your students are allowed to access the course”

Experts’ notes

Improve the course preview.

After having adjusted the course settings (second publishing), the experts tested the visibility of the course for students. The experts highly appreciated the possibility of adopting roles but were disappointed that this preview was not fully functional. Task T12C3a

Experts’ notes

Cartoon 2: By Hans Holzherr.

“Add a wiki to your course” Avoid having to switch interaction within one task.

At this moment the expert evaluators have become OLATconnoisseurs having experience with creating and publishing courses. Based on the new experience and knowledge available, they tried to add a wiki to their course. They did not succeed because the wiki has to be prepared outside the course editor before it can be linked into the course. The experts stated that this change of interaction

07 Expert Evaluation - Results is problematic, and that the context help was not adequate to resolve the problem. Principle: Similar transactions imply similar controls. Users learn the concept of interaction in a system by using it. If they achieved a goal in a certain way, they would expect the system to be functioning like this in general.

Cartoon 3: By Hans Holzherr. There is a phrase in aviation that says, „what can be mistaken will be mistaken“. Good usability avoids this type of error.

Summary of the main obstacles found • Students: Missing button to the right of the search field • Missing instructions when the student is not yet registred • Bookmarks menu is expected to be omnipresent (meta-navigation) • Users did not find the upload-button • When searching for a person, the name should be in the first field in the form, not the (technical) user name • OLAT's full-text search cannot find files in certain folders • Sub-folders sorted randomly • Too many calendars. How to distinguish them? • Entering a group appointment does not invoke invitation e-mails • Users having read-only access to a forum don’t see any buttons and don’t know why • Authors: Missing short introduction for authors.

07 Expert Evaluation - Results Forms with more than one send-button Users must enter titles and description twice Wrong or sub-optimal labels on tabs and buttons To understand symbols, users must consult the corresponding key • The two-step publishing process is obscure • Preview is not fully functional • The wiki must be prepared outside the course-editor • • • •

07 Expert Evaluation - Results 7.3 Interaction design issues The expert evaluators discussed the interaction design with the test executives at a personal computer with focus on the questions “where am I?”, “how did I get here?”, “how can I go back?”, “what’s the next step to take?”: the system should induce the answers to these questions at any time “voluntarily”, i. e. without making deep interrogation by the user necessary. The last question (“what’s the next step to take?”) assumes that the user wants to conduct a certain workflow to achieve a certain goal. If a more open situation is imagined, the question can be re-phrased: “what can be done?” Also, the answer to this question has to be clearly conceivable, it’s a selling proposition.

Where am I? How did I get here? How can I go back? What’s the next step to take?”

First screen This page is OLAT’s etiquette. Offer a short introduction for novice users.

Figure 11: OLAT’s first screen from a novice’s perspective: Does it answer the four crucial questions?

With these premises the first screen of OLAT (figure 11) was discussed from a novice’s perspective. The wherequestion is clearly answered by the page title: “Willkommen bei OLAT”. The fact that one did the “right thing”, login, is implicitly confirmed by the appearance of the welcome screen, and explicitly at the bottom: Eingeloggt What can be done? als... . Going one step back from this position is possible by clicking the log-out link in the top right corner of the 30 screen. “What’s the next step to take?” or “What can be done?” are the complex questions because many options have to be offered the representatives of mainly two roles,

07 Expert Evaluation - Results authors and students. OLAT offers the tree main sections (Home, Gruppen, Lernressourcen), a navigation in each section (at the left screen border) and the contextual toolboxes (at the right screen border). Furthermore, a link to the help document is offered in the meta-navigation. A special section of the first screen is the Quick Start Links (OLAT Schnellstart Links). They clearly indicate the aim to introduce novice users, but in the opinion of the expert evaluators they do not achieve because they are just linking to certain positions in OLAT. They do not offer a didactically arranged introduction which shows the key concept, e. g. the meaning of the three main sections (Home, Gruppen, Lernressourcen), the concept and granularities of courses and course-elements (Lernressourcen) and the concept of having possible transaction offered in toolboxes at the right screen border. The novice user has two ways of finding out about the key concepts: reading the help document or by using the system and interpreting his/her experiences.

The Quick Start Links do not offer a didactically arranged introduction

The expert’s opinion was that direct on-screen instructions in form of short text with standardized tip-off box layout would improve the usability of the system. First screen OLAT-Lernressourcen A consequent graphic concept for icons is missing. Figure 12: First screen when having entered the section “Lernressourcen”; on the very left the content navigation, and on the very right the toolboxes. Both of these areas may be imploded by the respective clicking icon.

On this screen (figure 12), direct on-screen instruction is available. It describes how to find content, but does not indicate that the toolboxes at the right screen border offer possibilities to create content. About this screen, the expert evaluators discussed the icons for menu/toolbox

07 Expert Evaluation - Results

Figure 13: Same as figure 12 with the content navigation auto-hide mode. On mouse-out the menu implodes. The offered close icon cannot be reached by the mouse pointer.

implosion and explosion. The general opinion was that usability would be improved just in this situation if they simply didn’t exist, but it was clear that in courses which do not need these navigation elements, hiding them may make sense. However, four problems with these icons can be stated: • There is no graphic concept as to which icons are clickable and which are not. • If the menus are in the dynamic mode, the mouse pointer can be placed on the menu border, so that they implode and explode in quick succession (vibrating). • The x offered to close a dynamically superimposed menu can never be clicked because the menu fades before the mouse pointers arrives at it. • The function behind the icon to enlarge the central column is more experienced as a function hiding both menus. The respective hover wording could be changed like this: If the menus are hidden with this method, maybe accidentally, the only way to get the menus back is to click this icon: The hover wording now is just “kleiner”. If a user does not achieve fadingin of the menus again, he or she is lost (figure 13) Answering the where-question is central Make the transactions necessary for switching between the views (course, editor, ‘Detailansicht’, preview) more transparent. Furthermore, the back button should be enabled. Since expert evaluators had switched section from Home to Lernressourcen, the “where am I”-question was dis-

07 Expert Evaluation - Results cussed here. The section is indicated by the colored tab The tab metaphor is “Lernressourcen”. Experts stated that the tab metaphor is used graphically wrong used graphically wrong because the tab is not connected to the sheet and should have the same color as the sheet. A second answer to the where-question is the orange-colored “Lernressourcen” in the content navigation.

Figure 14: The back button is the most-used feature of the web browsers.

Here, the «where question» is related to knowing how to go one step back. The back button is the most-used feature on the web browsers. The back button is not supported in OLAT; when clicked, a respective message is displayed (figure 14). In the present situation (just entered the section “Lernressourcen” coming from the Home-Section) ‘one step back’ to the user means clicking the “Home”-Tab. This implies firstly that the user knows where he came from, and secondly, that this section is re-available on the Home-Tab. The job to record a navigation history is shifted from computer memory to human memory. The experts’ opinion was that the back button must be supported although being aware of the technical implications though e. g. when data is written in the data base. However, disabling the back button is very suboptimal, but the OLAT-way to do this is straightforward: we did not find any exceptions, and the user is clearly informed.

The experts’ opinion was that the back button must be supported although being aware of the technical implications though

The where-question was focussed on further by the evaluation experts by setting up a course with a single page. When overlays are used, the “spatial” context is implicitly clear because the page from where it is called is still present in the background. This situation is exemplarily present when a new course is initiated. What is inconsistent

Krug, S. (2005). Don’t make me think. Berkeley CA, New Riders Publishing.

07 Expert Evaluation - Results with the use of overlays is the way they can be closed: some of them carry a close-icon in their top right corner, some have a back link; others display one or more cancelbuttons. The experts’ opinion was that the back-links be dropped, the cancel-buttons should work contextually in forms and the close-icon should always be present.

Some of the overlays carry a close-icon in their top right corner, some have a back link; others display one or more cancel-buttons.

As soon as the course is initiated and the course editor is open (figure 15), answering the where-questions becomes complicated: the tab “Lernressourcen” isn’t colored as selected any longer, a new tab displaying the course name is generated and is active now and the word “edit” is displayed as a watermark all over the page. Furthermore, the function “close editor” is offered in a toolbox. We are in the editor: where can we go from here, and what would be the way back?

Figure 15: First screen in the course editor. The tools not only offer elements to be inserted, but also implicit views. Experts recommend to make the views explicit.

07 Expert Evaluation - Results Action leaving the editor

Position in OLAT after having left the editor

Way back to the editor

Closing the editor with the respective link in the toolbox

course content

clicking the editor link named “Kurseditor” in the toolbox

Clicking the active course tab’s close-icon

Lernressourcen

Go to “Meine Einträge”, click the course title, then the show contentbutton or directly the show content-link in the preceding list, then the editor link in the toolbox.

Clicking another tab, e. g. Lernressourcen

Lernressourcen

Clicking the course tab still open or using one of the ways already described above

Clicking the course preview link in the tool box

preview pre-page, then preview

Closing the preview. If the “Lernressourcen”tab is clicked, the course tab remains and its content stays in the preview mode, meaning, when selecting the course again in “Meine Einträge”, the preview is displayed. It has to be closed to go back to the editor.

Table 1:

A special page is the so called “Detailansicht” containing meta-data of the course. Some experts experienced this page as the first page of the course, which was perplexing with the first content-page of the course, the latter displayable in the course-view, editor-view or pre-view. At this point, the system cannot longer answer the wherequestion appropriately. The meta-data page is optically linked to the “Lernressourcen”-tab, but offers two ways to start displaying the course content (firstly as button, and secondly, as link in the toolbox), and even starting the editor is offered in the toolbox worded astonishingly differently: “Edit content” (Inhalt editieren) instead of “Editor” (table 1). The evaluating experts recommended clarifying the where-question here. They identified certain views which should be declared and switchable as this. The usability problem detected here gets knotty when it comes to the two-step publication process necessary in OLAT. While the first step (publish) is very common, the second (granting access to the course as a whole) is a speciality which implies two problems. Firstly, it must be remembered, and secondly, it must be found. Both problems should be mitigated by usability improvement as mentioned in chapter 10.3.

“Detailansicht”, course-view, editorview, pre-view. At this point, the system cannot longer answer the where-question appropriately

The experts identified certain views which should be declared and switchable as this

Wherever authors go, at some point they need the editor again. The way back is hurdled.

07 Expert Evaluation - Results How much virtual “spatial” information is a user able to handle? Narrow the screen design elements and interaction patterns to a well defined set.

Figure 16: The nested ‘where’ (Background image source: Wikipedia Commons).

We cannot offer an answer, but vivify reflections about user limitations. Imagine a user at a computer with a single screen. His operating system offers menus, and there are applications open, e. g. two browser windows. The browser windows contain several web-pages organized in tabs. One tab is OLAT. In OLAT there are also tabs, a course tab is active. While editing the course, overlays will be opened, and, even more tabs, tabbed panes. When the user forgets to save form data in the tabbed pane (which maybe is in an overlay in the course tab in the browser’s OLAT-tab which may be the second open browser window) a JavaScript-alert window is displayed and must be confirmed (figure 16). «The myriad of inflowing information bits ... can hardly be processed; rather, in order for something like an inner landscape image to form, all unrelated information has to be suppressed or eliminated.» Anyway, if the user wants to consult the OLAT calendar, it will be opened in a pop-up that would now be the third browser window. Of course, all these metaphors are very helpful and mostly used properly in OLAT. But when the introduction of a new design element or even a new metaphor is under

Burckhardt, L. (2006) Warum ist Landschaft schön? Die Spaziergangswissenschaft. Markus Ritter und Martin Schmitz. (Hrsg.)Berlin, Martin Schmitz Verlag

07 Expert Evaluation - Results consideration, the picture of the stressed user should be present as this will perhaps lead to the decision to stick to elements already used. This thought came up during expert evaluation when the calendar opened in a pop-up window. In a rather complex “spatial” context (as this is the case for OLAT) the user can be supported and his or her navigation costs can be minimized as follows:

When the introduction of a new design element or even a new metaphor is under consideration, the picture of the stressed user should be present

Principle: If the probability of a user’s need for a tool on a certain screen is rather high, the tool should be present on the respective screen. The high probability often is implied by the common workflows. The experts noted that this principle is hurt when inserting a new wiki into a course. While course elements such as a forum can be generated and inserted on the fly in the course editor, the wiki must be prepared outside the editor, but linked to the course in the editor. Cartoon 4: by Hans Holzherr.

Answering “Did I do the right thing?”

While course elements such as a forum can be generated and inserted on the fly in the course editor, the wiki must be prepared outside

Besides the existing error messages, introduction of affirmative messages should be considered. Very often, this question is implicitly answered: e. g. when clicking a tab, the relative page comes up front. The experts identified that after saving content, the implicit answer could not be clear enough: the save button changes from orange to grey and furthermore, the overlay (if the form is in-a-such) stays sometimes open and sometimes Usability Testing and Evaluation, and Redesign Recommendations for the LMS OLAT. Report © 2007 University of Bern, Institute of Medical Education (IML)

07 Expert Evaluation - Results it closes. The experts discussed the introduction of short affirmative messages. Answering “What’s the next step to take?” Guide the user with short on-screen instructions and give the step-by-step instruction in the context help.

The experts suggest persistently offering short instructions on screen

The need to do this is evident when a novice user creates a course. The trade-off is that expert users could be annoyed by long textual instructions on-screen. However, the expert evaluators suggest persistently offering short instructions on screen in course creation and editing modes. These tip-off boxes could be switched off by configuration through expert users. Principle: Textual instructions should not be used to overcome usability problems caused by design errors. The experts suggest three degrees of textual instructions ordered by their instruction depth 1. on-screen instructions 2. context help 3. entire help document and tutorials The expert evaluators consulted some context help text and noted three problems: 1. the context help icon does not look like a clickable icon 2. the context help icon is placed outside the fieldset 3. the context help text often is a description not an instruction Usability Testing and Evaluation, and Redesign Recommendations for the LMS OLAT. Report © 2007 University of Bern, Institute of Medical Education (IML)

07 Expert Evaluation - Results The third problem is the most serious one and may hinder the user from solving a problem even after having read the context help. The context help text should exactly focus on the context and not talk about anything else; rather, it should give step-by-step and complete how-to information using the exact wording displayed on the respective screen.

The context help should give step-bystep and complete how-to information using the exact wording displayed on the respective screen

Principle: Context help texts must be gap-free, straight-forward instructions in the exact context of the position where the help icon was clicked Link instead of button

Screen design issues

Non-clickable 3D-buttons (3D implies clickability)

Always consult your existing design concept before considering new screen elements.

Clickable 2D-buttons

The experts assessed the overall screen design as very highly elaborated; mostly a clear concept was recognizable.

Same icon, different meanings Infobox on mouseover on a button Mouse over dialog

Figure 17: Screen design; selection of inconsistencies.

1. Set a kink 2. more informations 3. insert

1. “Command-edit” 2. Adapt the table

A threat for any concept is the incidence of exceptions and specialities (see figure 17). Principle: The look of each design element has to carry out a task, primarily to visually communicate the element’s properties (usability) and secondarily to look nice (emotional design). Elements with similar properties have to be similarly designed. Hardly ever are there good reasons for exceptions.

07 Expert Evaluation - Results Icon clearance Define semiotics for the icon design. Users can process a large amount of graphical information from screens by spatial and semiotic discrimination. If spatial discrimination is disabled (by ungrouping / regrouping the graphical elements) clarity of the semiotics can be verified. Figures 18ab and 19ab are inspired by Wehrli’s art work deconstructing famous paintings as we did it with OLAT pages below. These charts shall enable the viewer to assess the isolated semiotic message of the icons and their implicit formation of groups. The semiotic messages and groupings should represent the functions in OLAT. A respective concept seems to be missing (see figure 19b, clickable vs. non-clickable icons)

Figure 18a: Selection of screen design elements in their context. Figure 18b: Buttons and Icons eluted out of their screen design context of figure 18a

Wehrli, U. (2004). Kunst aufräumen. Königstein, Kein und Aber Verlag

07 Expert Evaluation - Results

Identifying definition gaps in a design concept Gaps in the design concept have to be closed. A concept for a developing and growing system cannot be complete. It is important that gaps in the concept are closed at the moment when they are identified. The moment of identification usually is when a certain design question cannot be answered: Figure 19a: Buttons and icons; semiotic analysis by a first impression sorting.

non-clickable icons

clickable icons

Where should that button be placed? • Where should that icon be placed? • Which icon should be used here? • Which control should be used here? • How should this text be formatted?

Figure 19b: Confrontation of one semiotic dimension, clickability to figure 19a.

07 Expert Evaluation - Results If the concept is not specified, developers and designers tend to create solutions off-concept. Some examples which could belong to this category are stated in the left column (figure 20). Just for the record: it is not a mistake when you hit the limits to go beyond - on the contrary - however, the new limits have to be defined within reasonable time. «Only the fact that not everything can be planned and controlled from start to end makes it possible to materialize an especially good object.»

Figure 20: Selected cut-outs of screens with seemingly off- or non-concept elements. System status message in bold and italic, buttons outside the fieldset-area, icons executing special functions as short cut to the editor (newly added) and menu implode / explode modes.

Frank, G., Herzog & de Meuron, van Lieshout, J. (2006). What moves architecture? (in the next five years). gta Verlag, ETH Zürich.

07 Expert Evaluation - Results Missing typographical optical axis Align text and graphics consistently. The experts often missed the implication of a clear typographical optical axis implicitly drawn by the consistent alignment of elements (figure 21). Typographical optical axis help the user to keep his/her eyes on track and to discriminate a large number of graphical elements perceived from on the screen.

Figure 21: Too many typographical optical axis misguide the eye.

Usability Testing and Evaluation, and Redesign Recommendations for the LMS OLAT. Report ÂŠ 2007 University of Bern, Institute of Medical Education (IML)

07 Expert Evaluation - Results

Complex encoding of information Display status information in a clear and simple way.

Figure 22: The read / write access is represented with a file-system metaphor wrongly used.

The experts recommend giving information to the user directly and as plain text. Screen designers tend to encode information when there is not enough space for the plain wording. The experts selected two examples to be considered for revision (figures 22 and 23). Because there could be even more places in OLAT where (over)-encoded information is displayed, OLAT should be screened for that, applying the following principle: Principle: Information must be clear to users without having them to look up symbols or abbreviations in keys. In this list the access (read) rights are displayed, encoded with four single letters and a dash as NULL, two colors for the whole string and four position slots for the character, making the false impression to represent 1250 possible states (meanings). The experts needed some time to understand this encoding and discovered that the 4 positions have no informational value because each letter is always in the same slot. Furthermore, the color does not permute states because certain letter combinations always have a certain color. Indeed, this pattern encodes only 4 states as displayed in the corresponding key. The experts hold that this encoding is much too complicated, and that the metaphor behind it, the bitwise representation of read/write/execute rights in file systems, is used falsely.

Usability Testing and Evaluation, and Redesign Recommendations for the LMS OLAT. Report ÂŠ 2007 University of Bern, Institute of Medical Education (IML)

07 Expert Evaluation - Results Another problematic encoding of information has been found on this screen (figure 23). An error in a course element is indicated with a red square displayed in the content tree, in the header, and in the corresponding key in the footer. Furthermore, this error state brings up an error message. This kind of disperse error state representation hard to be perceived and interpreted.

Figure: 23: The user must decode the meaning of the red square by recognizing its association with an element in the content tree, then consulting the respective key and heading.

Usability Testing and Evaluation, and Redesign Recommendations for the LMS OLAT. Report ÂŠ 2007 University of Bern, Institute of Medical Education (IML)

08 Usability Testing in the Usability Laboratory - Methods 8. Usability testing in the usability laboratory - Methods Test Room

Observation Room Software Developers

Camera focusing on the user's face

User's workplace with computer and list of the tasks

Test Administrator

Camera focusing on the user's table-top interaction

Recorder: Audio, Video, Events, Remote-Screen 3 Monitor Screens Windows Media stream for distant audience

Figure 24: Usability laboratory and equipment used.

For the usability testing, test persons were invited to accomplish the tasks listed above in our usability laboratory (figure 24) . The Test persons were selected from populations representing typical OLAT-users in the role of the student, and the role of the author: The authors where 12 instructors (professors and assistants) from the University of Berne and 10 students where selected from the Medical Faculty of Berne. Selection criteria were for both roles: gender and computer literacy based on self-declaration; for authors: e-learning experience. Data about e-learning experience was available for 286 instructors of the University of Berne based on a survey conducted in December, 2006. To validate the self-declared computer literacy, a short questionnaire was run in the form of a 3-minute interview before the usability test. The maximum score was 12. These results and further information on the test persons are shown starting on page 49. Further more the column overall ease of use of OLAT displays the self-declaration of these users concerning this subject just after the test.

The Test persons were selected from populations representing typical OLAT-users in the role of the student, or the role of the author

To validate the selfdeclared computer literacy, a short questionnaire was run

Every test person was booked for an individual one-hour time-slot. After arrival and the welcome coffee, the premises of the usability laboratory where briefly toured and subsequently, the test person was briefed in the meeting room about the test procedure and interviewed on the basis of the mentioned computer literacy questionnaire. The tasks to be performed were printed in a booklet and read Usability Testing and Evaluation, and Redesign Recommendations for the LMS OLAT. Report ÂŠ 2007 University of Bern, Institute of Medical Education (IML)

08 Usability Testing in the Usability Laboratory - Methods through by the test person, and questions could be asked. After the briefing, the test person and the test administrators took seat in the laboratory on the observer’s side. The laboratory consists of two rooms connected by a one-way mirror, so that the administrators can see the test person, but not vice versa (see figure 24). The room for the test person is equipped like a standard bureau with a personal computer with web-access. Both screen content and test person are videotaped, and the video signal is transmitted to the screens in the observer room. Both rooms are acoustically isolated; communication is possible through intercom.

Both screen content and test person are videotaped, and the video signal is transmitted to the screens in the observer room

The test person was helped to arrange chair, screen height, keyboard and mouse position and mouse keys (right or left handed) suitable for her/him, then the test person was left alone in the test room, door closed, and he or she surfed the internet freely during a few minutes to acclimatize. Afterwards, the signal was given through intercom to begin with task number one.

Cartoon 5: By Hans Holzherr.

Test persons were requested to solve each task self-handedly in the given order and do some trial and error if necessary. Utilities like OLAT’s context help or full help could be used. The test person was told to ask for help, whenever she or he would not succeed by trial and error, or, the test administrators decided to offer help when a certain amount of time had passed. Help was always given verbally through intercom, and always in two steps: first, a general hint (e. g. “You should search for this function in another section”) was given. If this wouldn’t help, instructions where to click were given.

The test person was told to ask for help, whenever she or he would not succeed by trial and error

08 Usability Testing in the Usability Laboratory - Methods Time

Activity

5 min.

arrival welcome coffee, usability laboratory tour

5 min.

briefing computer literacy, questionnaire (interview)

3 min.

perusal of the tasks by test person

30 min.

test person performs tasks

Location

meeting room

When all tasks were finished, test administrators asked a few questions (through the intercom) concerning the userâ&#x20AC;&#x2122;s experience during the last 30 minutes. Then the test person was owed his/her cooperation and accompanied to the meeting room where he/she filled in an ISO-based questionnaire online. The questionnaire allows to assess the software as it was experienced during the usability test in the fields defined by ISO 9241-10. The test time, all included was in average 55 minutes (table 2).

test room

5 min.

interviewing the test person (experience and debriefing)

10 min.

electronic questionnaire assesment of the software by the test person (web based questionnaire adapted from ISO 9241-10)

Table 2:

meeting room

Test procedure and timing applied.

Usability Testing and Evaluation, and Redesign Recommendations for the LMS OLAT. Report ÂŠ 2007 University of Bern, Institute of Medical Education (IML)

08 Usability Testing in the Usability Laboratory - Methods Overview test persons

* 1982 Person 01: Computer literacy self-declared: abecedarian

often problems

rarely problems

ace

Computer literacy questionnaire: 1

“I wouldn’t say that I had problems with OLAT… I find the fact that the back button doesn’t work disturbing, because I use it all the time.” * 1984 Person 02: Computer literacy self-declared: abecedarian

often problems

rarely problems

ace

Computer literacy questionnaire: 1

“My overall impression: OLAT is on the complicated side. Its design doesn’t look familiar. The placement of the elements on the screen is more scattered, and, as far as I’m concerned, lacks a certain logic.”

08 Usability Testing in the Usability Laboratory - Methods * 1985 Person 03: Computer literacy self-declared: abecedarian

often problems

rarely problems

ace

Computer literacy questionnaire: 1

“OLAT is very friendly and has a fairly clear design. The only problem was that it took some time until I understood that you have to register for the course to be able to access its documents… A hint would have been helpful.” * 1985 Person 04: Computer literacy self-declared: abecedarian

often problems

rarely problems

ace

Computer literacy questionnaire: 1

“I find the way OLAT basically works interesting. I don’t like the layout so much. The design is not really inviting.”. 50

08 Usability Testing in the Usability Laboratory - Methods * 1984 Person 05: Computer literacy self-declared: abecedarian

often problems

rarely problems

ace

Computer literacy questionnaire: 1

“Apart from the fact that it was not clear that you have to register for the course at the beginning, the program isn’t bad at all. I like it… if only the back button were changed…” * 1984 Person 06: Computer literacy self-declared: abecedarian

often problems

rarely problems

ace

Computer literacy questionnaire: 1

“A lot of times you have to look hard to find what you’re looking for; okay, it’s always difficult the first time… I would rather not use OLAT for learning.”

08 Usability Testing in the Usability Laboratory - Methods * 1985 Person 07: Computer literacy self-declared: abecedarian

often problems

rarely problems

ace

Computer literacy questionnaire: 1

“At the beginning I had troubles with orientation: Where am I now? Where to go now? How do I get back? I found it a bit complicated.” * 1987 Person 08: Computer literacy self-declared: abecedarian

often problems

rarely problems

ace

Computer literacy questionnaire: 1

“Generally speaking, I found OLAT thrilling; I would like to work with OLAT… At some point I had troubles to register for a course. They could have given a hint, there.”

08 Usability Testing in the Usability Laboratory - Methods * 1985 Person 09: Computer literacy self-declared: abecedarian

often problems

rarely problems

ace

Computer literacy questionnaire: 1

“I found it thrilling to look into OLAT; I hadn’t seen anything like that before. Tip for the developers: an introduction for first-time users of how everything works.” * 1983 Person 10: computer literacy self-declared: abecedarian

often problems

rarely problems

ace

Computer literacy questionnaire: 1

“It is not really clear at first that you have to register for the course… I would do fine after a number of sessions, though.”

08 Usability Testing in the Usability Laboratory - Methods Overview Test Persons Authors

* 1946 Faculty of Natural Sciences. Department of Chemistry and Biochemistry Person 11: Computer literacy self-declared: abecedarian

often problems

rarely problems

ace

Computer literacy questionnaire: 1

“Learning by trial and error. Instruction would be more efficient.” * 1958 Faculty of Natural Sciences. Institute of Applied Physics Person 12: Computer literacy self-declared: abecedarian

often problems

rarely problems

ace

Computer literacy questionnaire: 1

“Obviously, you have to deal with OLAT for a long time before you accomplish anything reasonable. I think it isn’t really self-explanatory. I wouldn’t use it from what I know after just this hour that I spent on it.”

08 Usability Testing in the Usability Laboratory - Methods * 1972 Institute of Computer Sciences and Applied Mathematics Person 13: Computer literacy self-declared: abecedarian

often problems

rarely problems

ace

Computer literacy questionnaire: 1

“One of OLAT’s problems is that you have to jump out of the course environment to create things. I felt a bit irritated that you must not change pages without saving changes first; otherwise, they get lost. It would be practical to be able to save everything at once.” * 1972 Institute of Computer Sciences and Applied Mathematics Person 14: Computer literacy self-declared: abecedarian

often problems

rarely problems

ace

Computer literacy questionnaire: 1

“The key word index doesn’t get you to the help topics, but just explains the respective term. When I click on Wiki, I’d expect to see ‹How to create a Wiki›.”

08 Usability Testing in the Usability Laboratory - Methods * 1957 Institute of English Languages and Literature Person 15: Computer literacy self-declared: abecedarian

often problems

rarely problems

ace

Computer literacy questionnaire: 1

“It wasn’t clear to me where to enable access for OLAT users. This would probably cause me troubles if I had to do it again.” * 1968 Department of Clinical Veterinary Medicine Person 16: Computer literacy self-declared: abecedarian

often problems

rarely problems

ace

Computer literacy questionnaire: 1

“It seems to me to be a versatile and flexible instrument which of course is an advantage for those who would like and are able to tap the full potential. At the same time, the whole thing gets relatively complex… Perhaps, a short introduction in whatever form would help to understand the basic functions.”

08 Usability Testing in the Usability Laboratory - Methods * 1950 Faculty of Social Sciences Person 17: Computer literacy self-declared: abecedarian

often problems

rarely problems

ace

Computer literacy questionnaire: 1

“There are things that I didn’t find ... somehow, this selfcreated mask needs getting used to.” * 1952 Medical Faculty Person 18: Computer literacy self-declared: abecedarian

often problems

rarely problems

ace

Computer literacy questionnaire: 1

“At the beginning it was hard to recognize the structures at all. I should have paid more attention to the flags on top. It wasn’t clear to me that through them I would have had access to several items.” 57

08 Usability Testing in the Usability Laboratory - Methods * 1970 Faculty of Natural Sciences Person 19: Computer literacy self-declared: abecedarian

often problems

rarely problems

ace

Computer literacy questionnaire: 1

“It wasn’t totally clear whether or not the way back would be irrevocable and I would lose or no longer be able to see something.” * 1951 Person 20: Department of Space Research and Planetology Computer literacy self-declared: abecedarian

often problems

rarely problems

ace

Computer literacy questionnaire: 1

“I wasn’t sure if the students would see the course. I’d log out and log in as a student to check it.” 58

08 Usability Testing in the Usability Laboratory - Methods * 1951 Health and Welfare Direction of the Canton of Bern Person 21: Computer literacy self-declared: abecedarian

often problems

rarely problems

ace

Computer literacy questionnaire: 1

“Before, you should have an idea of what is filed exactly where. You need a manual of some kind to familiarize yourself with the terms.” * 1976 Person 22: Institute of Psychology Computer literacy self-declared: abecedarian

often problems

rarely problems

ace

Computer literacy questionnaire: 1

“Sometimes, I had the feeling that a right mouse click would give me access to further possibilities, such as settings, just because I’m used to that from Windows. ... With publishing, it’s not clear what to do, really.” 59

09 Usability Testing - Results 9. Usability testing – Results In this section we present problems found during usability testing with test persons in our usability laboratory. You find the problems arranged by roles (user and authors) and by task, or in some cases, summarized by the type of problem. The problems are documented on video which can be accessed by surfing to http://usability.unibe.ch/olat/

where you will find the list with the video files in the directories indicated below (Dirname). The following login is required: username: olatstakeholders password: useresults The videos are DivX-encoded, a download-link for this codec is offered on the URL stated above. The video collection also displays the users which did not encounter problems at the respective tasks.

9.1 Students Task T4C1s

“Update your Vcard in OLAT” (figure 25)

Problem found No response after clicking ‘save’. Dirname

updating_personal_settings

Figure 25: Student updating her Vcard.

09 Usability Testing - Results

Task T5C2s

“Go to the course Ozeane XL” (figure 26)

Problem found Problems to find the course. Dirname

googleing_in_olat

Task T6C2s

“Read the introduction to the course” (figure 27)

Figure 26: Student searching for a course.

Problem found Cannot access the course content because inscription is necessary. Some tests subjects need some time to find the solution. Dirname

no_access_without_inscription

61 Figure 27: Student trying to access a course.

09 Usability Testing - Results

Task T8C2s Problems found

Dirname

“Upload a PDF file to your personal folder in OLAT” (figure 28)

Some tests subjects have problems detecting the add-file link. Uncertainties after the upload. Was it successful? upload_to_personal_folder

Figure 28: Student uploading a PDF file.

Task T9C3s Problems found

Dirname

“Download PDF x from student y” (figure 29)

Cannot access the course content. Problem browsing the sub-folders because they re-sort randomly after each click. Full text search cannot find file. download_pdf_x_from_student_y

Figure 29: Student downloading a PDF file.

09 Usability Testing - Results

Task T10C3s Problems found

“Create a study group” (figure 30)

Missing feedback after clicking the save-button. Uncertainties when adding members.

Dirname

Adding owners instead of members. create_a_study_group

Figure 30: Student creating a study group.

Task T12C3s Problems found

“Add a study group meeting to the group calendar” (figure 31)

Uncertainties as to which calendar the meeting should be published in. Unclear feedback after initiating a group calendar. Uncertainties if adding a meeting automatically prompts invitations.

Figure 31: Student downloading a pdf-file.

Dirname

Users expect drag functionality when writing a new appointment. initiate_a_study_group_meeting

09 Usability Testing - Results

Task T14C4s Problems found

“Solve the MC test and the self-test” (figure 32) Disappointing response to answers placed in the MC-test.

Missing linear navigation in the selftests makes the test persons hesitate.

Dirname

The options how to terminate or leave the test are not recognized straight away. mc_test

Figure 32: Student solving the MC test and the self-test.

Task T16C4s

“Write a question in the forum” (figure 33)

Problem found If the inscription for the course is missing, the forum can be read but not written to (missing buttons). Dirname

posting_in_the_forum

64 Figure 33: Student writing a question in the forum.

09 Usability Testing - Results

Action observed

“Using the back button” (figure 34)

Dirname

sorry_back_button_forbidden

Action observed

“Using the right mouse-key” (figure 35)

Dirname

right_mouse_key_use

Problem found The back button of the browser was clicked often, although it is not supported in OLAT.

Figure 34: Student using the back button.

Problems found

When exploring interaction possibilities, some users tried right mouse clicks.

Figure 35: Student using the right mouse-key.

09 Usability Testing - Results Action observed

“Using AJAX/desktop-style interactions” (figure 36)

Dirname

intuitive_mouse_drag

Comment

In the calendar, some users already expected AJAX/desktop-style interaction patterns.

Figure 36: Student using AJAX / desktop-style interactions.

09 Usability Testing - Results 9.2 Authors Task T3C1a Problems found

“Creating a ‘welcome page’ with the HTML Editor” (figure 37) The “choose file button” must be clicked to go to the editor. Form data can be lost. Module description is confounded with module content. Missing feedback after clicking the save button. After clicking the “save button” the “next button” has to be clicked. Some users did not see this button on the spot.

Figure 37: Author creating a welcome page.

Dirname Task T5C1a Problems found

Figure 38: Author uploading PDF files.

Dirname

Title and description have to be recorded twice. There is no indication why and what they are used for. where_is_the_create_page_button

“Uploading PDF files” (figure 38)

Users often do not see the upload link at once. Missing feedback after upload. uploading_pdf_files

09 Usability Testing - Results

Tasks T7C2a and T8C2a Problems found

“Publishing a course and giving access to the students” (figure 39) Users do not understand the fact that after publishing, a second publishing transaction is necessary.

Users cannot find the page where the second publishing transaction has to be executed.

Figure 39: Author publishing a course and giving access to students.

Dirname

Tasks T9C2a Problems found

Dirname

Users are not sure if the students have access now. the_two_step_publishing_hurdle

“Do students see the course?” (figure 40)

Because some course elements are not functional in the preview, the users do not find the confirmation they are looking for, namely, that the students do have access to all course elements. do_students_see_the_course

Figure 40: Author consulting the course preview.

09 Usability Testing - Results

Tasks T10C3a and T11C3a Problems found

Dirname

Figure 41: Author re-editing the course: making the forum accessible depending on the date.

Tasks T12C3a Problems found

“Re-editing the Course: making the forum accessible depending on the date” (figure 41) Users are not always aware of the “views” in OLAT: there is the course and the course editor besides the metadata page (Detailansicht) and the preview. the_view_problem_going_back_to_the_editor

“Adding a Wiki to the course” (figure 42)

Users do not understand the principle why some modules have to be pre-prepared outside the course editor. The error message does not display instructions. Users do not find the way to the page where the wiki has to be pre-prepared. After creating a wiki, there is no clear feedback that it’s done.

Figure 42: Author adding a wiki to the course.

Dirname

Title and description have to be recorded twice. There is no indication why and what they are used for. wiki_to_be_prepared_before_insertion

09 Usability Testing - Results Action observed

“Changing between tabs” (figure 43)

Dirname

unsaved_data_between_tabs

Problems found

Users are accustomed to change tabs without saving data first.

Figure 43: Author changing between tabs without having saved before.

9.3 First steps for novice authors We assume that novice authors, besides the ones just look- There are no direct ing around in and exploring OLAT, have an idea of what on-screen instructions for novice authors short term goal they want to achieve within the system during their first session. This intention is based on their teaching and communication needs and the mental model they have about OLAT, e. g. «OLAT is a learning management system; in such systems I can store learning content for my students». The mental model about OLAT in real life at the University of Zurich is communicated by a short individual demonstration given by OLAT administrators to novice authors; for test authors, that was replaced by a short flash movie.

09 Usability Testing - Results 9.4 The test persons assessed OLAT on an ISO 9142-10 based questionnaire

Figure 44: Authors’ opinion to “The program was easy to operate“.

Figure 45: Authors’ opinion to “I mastered the program during the test”.

The questionnaire was filled in online just after the usability testing session (chapter 8). As reported above, test persons in the role of authors met many obstacles they could not overcome without assistance by the test administrators. Nevertheless, only 4 authors out of 13 said that OLAT was not or partially not easy to operate (figure 44). However, the question of how they mastered operation of OLAT revealed some difficulties: 8 authors voted “not well” or even “bad”, no one said “very well” (5 voted “well”), (figure 45);

The question of how authors mastered operation of OLAT revealed some difficulties

By contrast, all test persons in the role of the student (10) voted that operation of OLAT was easy or rather easy (figure 46) and only one voted for “I did master the software...” with a negative answer (“not well”), (figure 47). Figure 46: Students’ opinion to “The program was easy to operate“.

Figure 47: Students’ opinion to “I mastered the program during the test”.

Here are some statements from authors • “some important menu items should be better marked • “missing feedback” • “at some points, help was necessary” • “too complicated” • “totally unclear what to do in the left and right menu” • “functionality is OK, but the structure is not apparent at once” • “frequent lack of clarity how to proceed”

09 Usability Testing - Results • “the program needs getting used to – although I could imagine to know its obstacles fairly soon” • “more good than bad, but I was very much aware of being observed and was trying hard not to make myself a complete idiot” • “as already mentioned, a view of the page as it appears online would be helpful at least to spot mistakes right away” • “there are many menu items, forcing you to take your time getting the hang of the program” and from students • “at the beginning I didn’t see the big picture, and I was a bit confused” • “sometimes, small hints are missing” • “good program” • “sometimes, I had to find the applications” • “but once you get the hang of it, it’s certainly userfriendly” • “depending on the application, not too good” • “and eventually, maybe even excellent! ;-)” • “by trial and error, I managed tolerably well”

09 Usability Testing - Results

Figure 48: Authors’ opinion to “How did the program react when you entered data or navigated?”

Figure 49: Authors’ opinion to “The icons used are clearly understandable”.

While the problems regarding the dialogs between user and OLAT discussed in the expert evaluation are visible in the votes, users did not mention in the questionnaire that they felt irritated by the icon design (for which rather severe inconstistencies were reported during expert evaluation): Only 4 authors voted that the reaction of OLAT was always clear and easy to understand (figure 48), but 12 authors voted that the meaning of the icons used were (always or most of the time) clearly understandable (figure 49). Again, the test persons in the role of the student encountered fewer problems: 7 out of 10 reported in the questionnaire that the reaction of OLAT was always clear, and 3 that it was sometimes unclear (figure 50). There are many interesting answers in the questionnaire which can help to understand users’ views. Please find the fully featured answers online http://usability.unibe.ch/olat/

Figure 50: Students’ opinion to “How did the program react when you entered data or navigated?”

Figure 51: Students’ opinion to “The icons used are clearly understandable”.

username: olatstakeholders password: useresults The online version offers filtering (show only males or females, show only users with low computer literacy a. s. o.), and furthermore, the answers of each person are available for some cross analysis (what else did this person answer?). The test persons assessed their OLAT experience also in a short interview recorded just after the usability test sessions. Please find excerpts at the URL mentioned above.

10 Redesign â&#x20AC;&#x201C; Reflection and Recommendations 10. Redesign - Reflection and recommendations 10.1 About redesign dimensions and redesign depth When evaluating OLATâ&#x20AC;&#x2122;s usability, it is apparent that much human brain power has been invested in defining and realizing a usable and appealing design and interaction patterns, leading to success: it works. If the existing usability problems can be reduced it will work even better. In other words: there are users not having much experience with web-based interfaces; for them, the detected usability problems are obstacles impossible to overcome. Once even these users are successful OLAT-users, this will constitute, as mentioned in the introductory chapters, an economic factor, and a sales argument.

Cartoon 6: By Hans Holzherr.

For low computer literacy users the detected usability problems are obstacles impossible to overcome

We have detected three sources of usability problems: 1. Exceptions: Some design patterns used are off-concept 2. Missing concept: Seemingly, there is no stringent design concept for certain elements 3. Inherent necessities: technological circumstances become manifest in the user interface 74

Usability Testing and Evaluation, and Redesign Recommendations for the LMS OLAT. Report ÂŠ 2007 University of Bern, Institute of Medical Education (IML)

10 Redesign – Reflection and Recommendations The third usability problem generator may have the following backgrounds: • For developers the manifestation of background technology (e. g. database structures) in the front end often is not an error but a logical consequence • Disruptions in the interaction design are often found in places where a newer module meets older program elements • Disruptions in screen design often reveal which team in which stage has developed the respective feature We approach the creation of redesign and design suggestions as follows: 1. Removal of the obstacles detected during the usability test 2. Implementation of the suggestions from the expert evaluation 3. Applying the general gestalt rules 4. Implementation of the general suggestions from the respective ISO standards and other publications and in dedicated chapters: 5. Thinking about possibilities to enable a low level entry for novice authors 6. Thinking about the proper use of AJAX When thinking about removing the obstacles detected during usability tests, the question of redesign depth becomes acute. Is it just giving the missing information to the user by an on-screen message, or changing some controls, or changing the order in the workflow, or even changing the concept behind the workflow?

When thinking about removing the obstacles detected during usability tests, the question of redesign depth becomes acute

Screendesign Technology

Macro usability Micro usabilty

10 Redesign – Reflection and Recommendations

Interactiondesign

Workflows (based on Use Cases)

When the redesign “only” touches the way an underlying concept is realized as screen and interaction design, we call it ‘changes in micro usability’. When the underlying concept itself has to be adjusted, we call it ‘changes in Suggestions for usabilmacro usability’ (see figure 52). ity improvement are Suggestions for usability improvement are only as good as their feasibility. If they end up in a drawer, users won’t benefit. Hence we make a point of suggesting adjustments mainly on the micro usability level in section 10.3. Nevertheless, some more complex interventions in the workflow area must be considered in order to resolve all problems. Then, in chapter 11 we will propose an extension of the existing concept to a wizard-like meta-layer for an important use case of novice authors.

only as good as their feasibility

In chapter 11 we will propose to a wizardlike meta-layer for novice authors.

Basic Concept (features, goals, models)

Figure 52: Usability is influenced by different levels of concepts, technology and design. Redesign approaches can concern all levels.

10 Redesign – Reflection and Recommendations 10.2 Identified redesign areas Redesigning icons

Figure 53: Which icon is clickable?

• The icons shall clearly be identifiable as clickable or non-clickable (figure 53) • The icons shall show understandable symbols (figure 54) • When the meaning of icons have to be learned by the user, learning shall be supported by the appearance of the icons Using common patterns If there are several possibilities in design or interaction, the most common in contemporary western usage of screen media shall be supported.

Figure 54: Which metaphor is more common?

10 Redesign – Reflection and Recommendations Examples:

Module 4 Module 4

Module 3

Module 2 Module 2

Module 1

• A form has input fields and one button to save/proceed; mandatory fields are asterisked; • The back-button brings up the previous page • The wording used shall reflect the user’s experience with other software. E. g. fieldset-titles are section titles (mostly nouns), neither instructions nor actions • Metaphors should be used correctly Removing gaps in the workflow arising from system unit junctions and boundaries • Keeping the workflow uninterrupted: The user should be able to initiate a transaction where and when it is implicated by a common workflow without changing modules (figure 55) • Avoiding redundancies: When opening a new course, title and description can be recorded twice. Input of redundant data should be either explained to the user or eliminated • Keeping data consistent: Where redundancies remain, modifications should be triggered

Figure 55: Gaps in the workflow have to be closed on the GUI-level.

10 Redesign – Reflection and Recommendations User guidance • The user is principally guided by on-screen instructions • Users’ transactions are affirmed by a positive feedback or an error message • Error messages must offer a solution • These messages have a consistent layout • The context help texts are gap-free step-by-step instructions in the very exact context • Where the same content can be displayed in various views, the views have to be declared clearly and must be controllable by the user Screen design principles • Elements on screen shall be arranged in thematic groups. No elements shall be placed in the space outside of implicit or explicit groups • The groups shall constitute typographical optical axis helping the user to direct his view • Appearance of controls has to be consistent with their function (e. g. links vs. buttons)

79 Cartoon 7: By Hans Holzherr.

10 Redesign – Reflection and Recommendations 10.3 Redesign recommendations On the following pages, we show possibilities to adjust screen and interaction design both on the micro and macro usability level in a way that the problems found during expert evaluation and usability testing are corrected. In other words: we hypothesize that with the design solution suggested, all test persons could have performed all tasks obstacle-free without any help by the test administrators. Furthermore, we adjust the design patterns in all areas identified and listed in Section 10.2.

With the design solution suggested, all test persons could have performed all tasks obstacle-free

Adjusting the tab metaphor Figure 56: Current design (screen 1), redesign recommendation (screen 2): Correct use of the tab metaphor.

? Figure 57: Current design (1st line), redesign recommendation (2nd line): Clickable icon are in 3D-shape.

From this redesign recommendation a general principle can be derived: metaphors used should be close to their brick-world original. That’s why we suggest to connect the active tab to the active page, and to make the active tab the same color as the active page. Furthermore, the three tabs of OLAT’s three main sections should be disunited because they are individually clickable. As they are the main-section, they are drawn larger than the other tabs (figure 56). Making clickability of icons visible Both icons in 2D and 3D-shape are used in OLAT, but this shape does not refer to their clickability. Because a 3Dshape communicates clickability this should be applied as a design principle (figure 57).

Both icons in 2D and 3D-shape are used in OLAT, but this shape does not refer to their clickability

10 Redesign – Reflection and Recommendations Unambiguous use of icons

1. set a link

2. more information

2. adapt the table

3. insert

1. “command-edit”

Figure 58: The same icon has different meanings.

Identical icons are used in different circumstances. This difference should be expressed by using different icons (figure 58). Use of self-explanatory icons, e. g. for the chat status The chat status icon is a yellow star and variations of it. The meaning of a yellow star-icon often is used as icon for a bookmark section in browsers. That’s why we suggest a redesign to a series of smilies. Even though the different chat-states are expressed in only slight changes of the smily-icon, humans are able to recognize the meaning of these variations because of their sensitivity to details in faces (figure 59).

Figure 59: Current design (1st row), redesign recommendation (2nd row): The usual meaning of the star icon is ‘bookmarks’.

Avoiding keys for icons

Mein Kurs

Einführung Forum Aufgabe Wiki

Die Konfiguration dieses Kursbausteines ist fehlerhaft Figure 60: Current design (on the left), redesign recommendation (on the right): Disassembling two icons and replacing the key with mouse-over text.

The cluster of icons makes interpretation more difficult because besides the meaning of each single icon the extended meaning of the grouping has to be understood. Displaying a key is helpful to the user but it is more paper than screen style, and because of that fact users might overlook the key. In the redesign suggestion in the left column you see slight changes: the status icons are placed next to the document type icon instead of on top of it. We have omitted the caption and replaced it with a mouseover text because test persons were intuitively trying to read the accessory icons by means of mouseover (figure 60).

Displaying a key is helpful to the user but it is more paper than screen style

10 Redesign – Reflection and Recommendations The search field in the meta navigation We suggest to make the loupe icon clickable, and to place it as a button to the right of the search field (figure 61). That’s according to the common use of input fields: they can be sent by hitting enter or clicking a button or icon.

Figure 61: Current design (screen 1), redesign recommendation (screen 2): Making the loupe icon the button executing the search.

10 Redesign – Reflection and Recommendations (0/0)

Home Gruppen Lernressourcen Home Einstellungen

Suche

Hilfe ? Log out

Willkommen bei OLAT

Kalender Benachrichtigungen Bookmarks Persönlicher Ordner

Diese Startseite konfigurieren

Notizblätter Leistungsnachweise Andere Benutzer

OLAT Schnellstart OLAT Einführungsfilm Expresskurs anlegen Kurs finden

OLAT’s first screen for first time users The first screen is OLAT’s showcase for first time users. If it is tidy and helpful, these are the first attributes memorized by the users. We recommend reducing it to the maximum, displaying only the portlets that are absolutely necessary, the most important one being “OLAT Quick start” with three buttons (besides the main- and metanavigation), (figure 62).

If OLAT’s first screen is tidy and helpful, these are the first attributes memorized by the users

«... the effective use of visual and auditory modalities is directly linked to efficient instruction.»

Figure 62: OLAT’s first screen reduced to the max with a new quickstart section.

• Clicking the button “OLAT, introductory movie” prompts an overlay presenting two buttons: “Introduction for students”, and “Introduction for authors” • Clicking the button “Create Express Course” prompts a new wizard discussed in chapter 11: a medium-literacy novice author is enabled to set up a standard course within 5 minutes and only about 7 clicks, a staggering sales argument! • Clicking the “Find a Course”-button prompts an overlay offering the possibilites to find a course: fulltext search, list, catalogue, my courses

Clark, R., Nguyen, F., Sweller, J. (2006). Efficiency in Learning. Evidencebased guidelines to manage cognitive load. San Francisco, CA, Pfeiffer

10 Redesign – Reflection and Recommendations Overlay: Creating an new course The current version of this tabbed pane consists of two forms and a next-button which is off the content area of the pane (and sometimes was overseen by the test persons). We suggest redesign to a classic form: first, the fields have to be filled in, and then, one of the save-button has to be clicked (technically, the upload of an image doesn’t have to be handled separately). We introduce two save buttons to enable the user to save and select the follow-up screen with one click; this approach is seen in many desktop applications and is commonly accepted.

(0/0)

Home Gruppen Lernressourcen Lernressourcen Katalog

Scuhmaske Meine Einträge Kurse Tests Fragebogen

Importieren

Lernressourcen Jetziger Text:Benutzen Sie den OLAT Katalog, um zu einem Lerninhalt zu gelangen. Alternativ können Sie eine allgemeine Suche über den Menüpunkt 'Suchmaske' starten oder Sie wählen eine der vordefinierten Suchen im Menu auf der linken Seite. Hier muss der Text überdacht werden. Vorschlag: Kurze Erklärung, was eine Lernressource ist. Der Hilfe-button gibt dann Handlungsanleitungen

Lernressourrce: Kurs

CD-Lerninhalte SCORM-Lerninhalte Ressourcenordner Wikis Glossar

Hilfe ? Log out

Suche

Titel und Beschreibung

Kurs CP-Lerninhalt SCORM-Lerninhalt Test Fragebogen Wiki Glossar Andere Formate

Herstellen

Informationen zum Eintrag

Kurs

Die mit * bezeichneten Rubriken müssen ausgefüllt werden.

? Test

Fragebogen Ressourcenordner Wiki Glossar

Titel des Kurses * Kursbeschreibung * Wikisyntax Bild zum Kurs (jpg, png, gif)

durchsuchen... speichern und Editor öffnen

speichern und schliessen

abbrechen

Id: ID-Nummer Urheber: Urhebername Typ: Kursbausteintyp

Figure 63: Current design (screen 1), redesign recommendation (screen 2): The new form has a clear typographical optical axis and offers the common interaction pattern: Filling out fields then clicking a button.

We introduce two save buttons to enable the user to save and select the follow-up screen with one click

On this overlay we introduce a redesigned close icon (3Dshape = clickable). We suggest using it consistently in every overlay in the top right corner. Other controls to close overlays as a back- and close links should be eliminated. The cancel buttons on overlays as antipoles to the savebuttons should remain even if their function is similar to the close icon. On the tabbed pane we introduced a redesigned help icon Context help must (3D-shape = clickable). Because it brings up the context be placed inside, not help it must be placed in the context, meaning inside, not outside the fieldset outside the fieldset. Furthermore, we specify the wording on the tab and change the asterisking of mandatory fields from an orange bullet to the more self-explanatory asterisk. Meta information which is not imperative for common users is grouped in the bottom left corner and formatted smaller. If you compare the old screen with the redesigned one you may observe that the overall appearance is more settled and the grouping is clearer. This is achieved by a

10 Redesign – Reflection and Recommendations typographical optical axis created by the uniform alignment of input field and first button, and the left alignment of labels. Furthermore, we have removed the course-type icon (figure 63). The redesign of this screen simplifies the interaction (figure 64). Screen: Course editor

Figure 64: Interaction schema before (left) and after (right) recommended redesign.

The main problem on this screen is answering the “where am I”-question. To guide the users not only with the editwatermark on the background, we suggest identifying the editor clearly as a view. Therefore, the toolbox “Views” (Ansichten) is introduced and according to the existing concept for the content navigation, the active view is colored orange (figure 65).

To guide the users we suggest identifying the editor clearly as a view. Therefore, the toolbox “views” is introduced

The theoretical rationale why the simultaneous presentation of all possible views can be found in the temporal contiguity principle: «When corresponding portions of narration ... are presented at the same time, the [user] is more likely to be able to hold mental representations ... in working memory.» The new view-toolbox makes the link “close editor” which was formerly placed as a tool obsolete: the user can just change views. Technically, it might be a challenge to enable the view-links to close the existing view on one hand, and open the new one on the other hand. 85

Mayer, R. E..(2001). Multimedia Learning. New York, Cambridge University Press.

10 Redesign – Reflection and Recommendations «You can’t make everything self-evident» Home Gruppen Lernressourcen Mein Kurs

(0/0)

Mein Kurs

Hilfe ? Log out

Suche

Ansichten

Tipps

Kursinhalte anzeigen Kurseditor Kursvorschau anzeigen Detaileinstellungen anzeigen Systemmeldungen einstellen

Im Kurseditor können Sie mit den Werkzeugkasten rechts u. a. Kursbausteine einfügen und/oder verändern. Die eingefügten Kursbausteine erscheinen links.

Editorwerkzeuge Ablageordner Publizieren

Kursbausteine einfügen

Home Gruppen Lernressourcen Mein Kurs

(0/0)

Mein Kurs

Tipps Im Kurseditor können Sie mit den Werkzeugkasten rechts u. a. Kursbausteine einfügen und/oder verändern. Die Einstellung der Systemmeldung Tipps eingefügten Kursbausteine erscheinen links. Tipps immer ausblenden

Tipps nur hier ausblenden

Suche

Struktur Einzelne Seite Externe Seite CP-Lerninhalt Hilfe ? Log out SCORM-Lerninhalt Forum Wiki Dateidiskussion Ansichten Ordner Kursinhalte anzeigen Bewertung Kurseditor Aufgabe Kursvorschau anzeigen Test Detaileinstellungen anzeigen Selbst-Test Ablageordner Fragebogen Systemmeldungen einstellen Einschreibung Kontaktformular

Kursbausteine einfügen Struktur Kursbaustein ändern Einzelne Seite Löschen Externe Seite Vershieben CP-Lerninhalt Kopieren SCORM-Lerninhalt Forum Wiki Dateidiskussion Ordner Bewertung Aufgabe Test Selbst-Test Fragebogen Einschreibung Kontaktformular

Figure 65: Redesigned first screen in the editor; the first toolbox offers all available views. These are directly switchable, and the active one is highlighted. Furthermore, user guidance is improved by tip-off box which can be deactived (for experts).

Self-evidence is the best user guidance, self-explanatoriness the second best. On this screen we introduce user guidance the second way: yellow tip-off boxes. We suggest realizing these on all authoring screens containing short directions what should or can be done on this screen. The tip-off box displays the uniform close button which prompts the question if this tip shall be closed permanently. The overall configuration of these messages can be maintained in the “views”-toolbox where the control “configure system messages” is introduced. Even though the configurations available there are valid globally for this author, it makes sense to offer this link in the courseeditor context (see Chapter 10.2).

Kursbaustein(e)... Löschen Vershieben Kopieren Publizieren

Krug, S. (2005). Don’t make me think. Berkeley CA, New Riders Publishing

10 Redesign – Reflection and Recommendations Screen: Create a single page; tab: title and description «Error situations are critical for usability for two reasons: First, by definition they represent situations where the user in trouble and potentially will be unable to use the system to achieve the desired goal. Second, they present opportunities for helping the user understand the system better.» On this redesigned screen you see the new shape of error messages (figure 66): • they are straight forward instructions • the title is no longer a link, but a debug-button is offered • the context help button is offered Home Gruppen Lernressourcen Mein Kurs Willkommensseite

(0/0)

Mein Kurs

Unvollständiger Kursbaustein

jetzt beheben

Hilfe ? Log out

Suche

Ansichten

Definieren Sie Inhalte für den Kursbaustein „Willkommensseite“. Klicken Sie auf den Reiter „Seiteninhalte“ und erstellen Sie eine Seite im Wysiwyg-Editor, binden Sie eine schon erstellte Seite ein oder laden Sie eine Seite hoch.

Kursinhalte anzeigen Kurseditor Kursvorschau anzeigen Detaileinstellungen anzeigen Ablageordner Systemmeldungen einstellen

Kursbaustein: einzelne Seite Kursbausteine einfügen Titel und Beschreibung

Seiteninhalt Sichtbarkeit Zugang

Informationen Die mit * bezeichneten Rubriken müssen ausgefüllt werden.

Seitentitel * Deskriptiver Titel Seitenbeschreibung

speichern Link auf diesen Kursbaustein setzen

Struktur Einzelne Seite Externe Seite CP-Lerninhalt SCORM-Lerninhalt Forum Wiki Dateidiskussion Ordner Bewertung Aufgabe Test Selbst-Test Fragebogen Einschreibung Kontaktformular

Kursbaustein(e)... Löschen Vershieben Kopieren Publizieren

Figure 66: Current design screen 1 (cut-out), redesign recommendation screen 2 (entire screen): On the lower screen all redesign recommendations are implemented, the focus here is the error message: better wording (what is wrong?, what is to do?); debug button instead of linked title; context help icon; close button instead of hide button.

The red square indicating incomplete course modules is now a red point (traffic light metaphor) and only displayed in the content navigation. On the old screen it was displayed there too (covering the module symbol partly), in the heading and in the caption (key). We did away with the caption because the red point can be hovered over, and the new error message replaces the heading. The tab giving access to the content has a new wording: “Seiteninhalt”, because the old Wording “HTML-Seite” was unclear for many test persons. This tab could be positioned as second tab, but this change is not imperative. 87

Frese, M., Brodbeck, F., Heinbokel, T., Mooser, C., Schleiffenbaum, E., and Thiemann, P. (1991). Errors in training computer skills: On the positive function of errors. Human-Computer Interaction 6, 1, 77-93.

10 Redesign – Reflection and Recommendations We redesigned the third toolbox: it now displays all actions referring to course modules. Now, the action “publish” is available there. Nevertheless, it is in the last position and can easily be discovered (better than if it were in the middle of a list).

The third toolbox displays all actions referring to course modules

Screen: Create a single page; tab: title and description

Home Gruppen Lernressourcen Mein Kurs Willkommensseite

(0/0)

Mein Kurs

Unvollständiger Kursbaustein

jetzt beheben

Hilfe ? Log out

Suche

Ansichten

Kursinhalte anzeigen Kurseditor Kursvorschau anzeigen Detaileinstellungen anzeigen Ablageordner Systemmeldungen einstellen

Kursbaustein: einzelne Seite Kursbausteine einfügen Titel und Beschreibung

Seiteninhalt Sichtbarkeit Zugang

Auswahl der HTML-Seite

HTML-Seite

erstellen, auswählen oder editieren

Status: Keine Datei ausgewählt oder erstellt Darstellung der HTML-Seite

Mit Sartseite:

Nein, Inhalt bitte direkt starten

In «i-frame»:

Nein

Zuletzt besuchte Unterseite bleibt bestehen:

Nein

Struktur Einzelne Seite Externe Seite CP-Lerninhalt SCORM-Lerninhalt Forum Wiki Dateidiskussion Ordner Bewertung Aufgabe Test Selbst-Test Fragebogen Einschreibung Kontaktformular

On this screen again (figure 67), we show the option to move the content-tab from the fourth to the second position. However, most important is the new wording of the button which has to be clicked to create, choose or edit a page. Because clicking this button really offers three things, the wording has to communicate this fact (test persons did not click this button, but searched for a ‘create page’ function). Furthermore, the wording of the fieldset-title has been changed from action description to section nouns (common sense in desktop applications). The layout and position of the status-message has been optimized (no special font weights, positioning below the button).

Because clicking this button really offers three things, the wording has to communicate this fact (test persons did not click this button, but searched for a ‘create page’ function)

Kursbaustein(e)... speichern

Löschen Vershieben Kopieren Publizieren

88 Figure 67: Current design (screen 1), redesign recommendation (screen 2): Besides general rearrangement (typographical optical axis, icons, a. s. o.) the orange button is now labeled correctly (test persons didn’t understand the former button).

10 Redesign – Reflection and Recommendations Screen: Create a single page; overlay: module single page Some improvements already introduced are present in this overlay (figure 68): the back-link has been replaced by the close-icon, each fieldset-section contains a context help (button placed inline), and all controls are aligned left.

Home Gruppen Lernressourcen Mein Kurs Willkommensseite

(0/0)

Mein Kurs

Unvollständiger Kursbaustein

jetzt beheben

Hilfe

Suche

Log out

Ansichten

Kursinhalte anzeigen Kurseditor Kursvorschau anzeigen Detaileinstellungen anzeigen Ablageordner Systemmeldungen einstellen

Editor

Kursbausteine einfügen

Titel und Neue Beschreibung Seiteninhalt Sichtbarkeit Zugang HTML-Datei im Wysiwyg-Editor erstellen Dateiname: Auswahl der HTML-Seite ?

erstellen HTML-Seite erstellen, auswählen oder editieren Ablageordner

Status: Keine Datei ausgewählt oder erstellt Eine HTML-Datei aus dem Kurs-Ablageordner wählen Darstellung der HTML-Seite wählen Upload

Mit Sartseite:

? Nein, Inhalt bitte direkt starten

Nein im Dateien im Zip-Paket in den Kurs-Ablageordner hochladen In «i-frame»: Eine HTML-Datei oder mehrer

ZuletztDatei: besuchte Unterseite bestehen: Ziel: /bleibt Ablageordner

dursuchen...

Nein

übermitteln

Struktur Einzelne Seite Externe Seite CP-Lerninhalt SCORM-Lerninhalt Forum Wiki Dateidiskussion Ordner Bewertung Aufgabe Test Selbst-Test Fragebogen Einschreibung Kontaktformular

Kursbaustein(e)... Löschen Vershieben Kopieren Publizieren

löschen

Information zur Belegung Ihres Ordners 0% (0 / 300000 KB)

Figure 68: Current design (screen 1), redesign recommendation (screen 2) Note that each fieldset contains a context help icon.

10 Redesign – Reflection and Recommendations Screen: Create a single page after having created page On this screen (figure 69) we introduce the message type „positive affirmation“. Users feel more comfortable if they know they have made the right thing. Very often, OLAT says just nothing, e. g. after saving form data, which made users insecure.

Users feel more comfortable if they know they have made the right thing

We have removed the preview button because it was placed outside the content area of the tabbed pane. We suggest making the status line “selected file” a link. As mentioned before, the publish-tool is placed in the third tool box, we are clicking this link now. Home Gruppen Lernressourcen Mein Kurs Willkommensseite

(0/0)

Mein Kurs

Kursbaustein „Willkommensseite“ erfolgreich hinzugefügt

Ansichten

Den Kursbaustein sehen Sie links in der Navigation

Kursinhalte anzeigen Kurseditor Kursvorschau anzeigen Detaileinstellungen anzeigen Ablageordner Systemmeldungen einstellen

Kursbaustein: Willkommensseite Titel und Beschreibung

Hilfe ? Log out

Suche

Seiteninhalt Sichtbarkeit Zugang Kursbausteine einfügen

Auswahl der HTML-Seite

Datei in Editor öffnen

Datei auswechseln

Status: Gewählte Datei: Ablageordner/willkommen.html Sicherheitseinstellung

Verlinkung im gesamten Ablageordner zulassen

Nein speichern

Darstellung der HTML-Seite

Struktur Einzelne Seite Externe Seite CP-Lerninhalt SCORM-Lerninhalt Forum Wiki Dateidiskussion Ordner Bewertung Aufgabe Test Selbst-Test Fragebogen Einschreibung Kontaktformular

Mit Sartseite:

Nein, Inhalt bitte direkt starten

In «i-frame»:

Nein

Zuletzt besuchte Unterseite bleibt bestehen:

Nein

Kursbaustein(e)... Löschen Vershieben Kopieren Publizieren

speichern

Figure 69: Current design (screen 1), redesign recommendation (screen 2) Inter alia affirmation in green boxes are introduced.

10 Redesign – Reflection and Recommendations Overlay: Publish For the test persons, making the modules accessible to the students was a tough nut to crack. They thought publishing just does that, but in OLAT a second step, changing the general access parameters, has to be taken. However, these parameters cannot be found in the course editor. Out of 13 test persons, no one succeeded in making the course accessible. That’s why we have to make a suggestion here for usability improvement, which is easy to draw on a mock-up screen, but touches the macro usability level (figure 70). However, realizing the solution shown would be a big step forward in OLAT’s usability in authors’ perspective. Home Gruppen Lernressourcen Mein Kurs Willkommensseite

(0/0)

Mein Kurs

Keine Probleme oder Fehler in diesem Kurs gefunden

Ansichten Kursinhalte anzeigen Kurseditor Kursvorschau anzeigen Detaileinstellungen anzeigen Ablageordner Systemmeldungen einstellen

Kursunterlagen Kursforum Mein Kurs Publizieren der Änderungen „Mein Kurs“ Titel und Beschreibung Seiteninhalt Sichtbarkeit Zugang Wichtig: Auswahl der HTML-Seite Beim Publizieren von gelöschten Kursbausteinen werden dessen untergeordnete Kursbausteine ebenfalls gelöscht. Analog werden beim Publizieren von neuen Kursbausteinen dessen untergeordnete Kursbausteine ebenfalls publiziert.

Datei in Editor öffnen

Kursbausteine einfügen ?

Datei auswechseln

Tipp: Stellen Sie den Zugriff für den Kurs ein und wählen Sie die Kursbausteine aus, welche Sie publizieren möchten. Status: Gewählte Datei: Ablageordner/willkommen.html Weitere Einstellungen der Lernressource können Sie nach dem Publizieren rechts über die Toolbox „Ansichten“ über den Eintrag „Detaileinstellungen“ tätigen. Sicherheitseinstellung

Verlinkung gesamten Mein im Kurs Ablageordner zulassen Willkommensseite Kursunterlagen Kursforum

Hilfe ? Log out

Suche

Nein speichern

Alle auswählen Darstellung der HTML-Seite

Struktur Einzelne Seite Externe Seite CP-Lerninhalt SCORM-Lerninhalt Forum Wiki Dateidiskussion Ordner Bewertung Aufgabe Test Selbst-Test Fragebogen Einschreibung Kontaktformular

? Zugriff haben alle registrierten OLAT-Benutzer

Mit Sartseite: publizieren In «i-frame»:

Kursbaustein(e)... Löschen Vershieben Kopieren Publizieren

abbrechen

speichern

Figure 70: Current design (screen 1), redesign recommendation (screen 2): OLAT’s two-step publishing process is brought together in the new overlay.

In the general course parameter section which can be found outside the editor in the “Detailansicht”, there are many configurable values. We chose the critical one to be displayed and being configurable in the publish dialog overlay. What seems rather odd to OLAT-developers (configuring a single parameter here is off system architecture) makes sense for users: they will successfully access the content in one step. The technical challenge will be to make this inter-module communication possible.

Authors thought publishing just does that, but in OLAT a second step, changing the general access parameters, has to be taken

What seems rather odd to OLAT-developers makes sense for users: they will successfully access the content in one step

Besides this fundamental adjustment, here are some minor changes: The content tree metaphor was optimized by removing a title (the content tree is not a sub-item of this title), and the links “select all” and “deselect all” are changed to a select all-check box as commonly used in web-mail application to perform global selections.

10 Redesign – Reflection and Recommendations Screen with tabbed pane: Create Wiki, title and description Besides the optimizations suggested above (redesign of the error message section with a debug button, error message wording, typographical optical axis, “real” asterisking of mandatory fields a. s. o.) again, just as an option to think about, we moved the fourth tab to the second position. We used the wording “the wiki” to make it very clear what the user can expect from this tab (figure 71).

Home Gruppen Lernressourcen Mein Kurs Kurswiki

(0/0)

Mein Kurs

Unvollständiger Kursbaustein

jetzt beheben

Hilfe ? Log out

Suche

Ansichten

Definieren Sie Inhalte für den Kursbaustein „Kurswiki“. Klicken Sie auf den Reiter „Das Wiki“ unm ein bestehendes Wiki in den Kurs einzubinden oder ein neues zu erstellen.

Kursinhalte anzeigen Kurseditor Kursvorschau anzeigen Detaileinstellungen anzeigen Ablageordner Systemmeldungen einstellen

Kursbaustein: Kurswiki Kursbausteine einfügen Titel und Beschreibung

Das Wiki

Sichtbarkeit Zugang

Informationen Die mit * bezeichneten Rubriken müssen ausgefüllt werden.

Titel des Wikis *

Kurswiki

Deskriptiver Titel Beschreibung des Wikis *

Das Kurswiki hat zum ziel ein Wissenspool zu sein, welches von allen Studierenden kollaborativ editiert wird.

speichern Link auf diesen Kursbaustein setzen

Struktur Einzelne Seite Externe Seite CP-Lerninhalt SCORM-Lerninhalt Forum Wiki Dateidiskussion Ordner Bewertung Aufgabe Test Selbst-Test Fragebogen Einschreibung Kontaktformular

Kursbaustein(e)... Löschen Vershieben Kopieren Publizieren

92 Figure 71: Current design (screen 1), redesign recommendation (screen 2): Create Wiki.

10 Redesign – Reflection and Recommendations

Home Gruppen Lernressourcen Mein Kurs Kurswiki

(0/0)

Mein Kurs

Unvollständiger Kursbaustein

jetzt beheben

Hilfe ? Log out

Suche

Ansichten

Definieren Sie Inhalte für den Kursbaustein „Kurswiki“. Klicken Sie auf den Reiter „Das Wiki“ unm ein bestehendes Wiki in den Kurs einzubinden oder ein neues zu erstellen.

Kursinhalte anzeigen Kurseditor Kursvorschau anzeigen Detaileinstellungen anzeigen Ablageordner Systemmeldungen einstellen

Kursbaustein: einzelne Seite Kursbausteine einfügen Titel und Beschreibung

Das Wiki

Sichtbarkeit Zugang

Auswahl des Wikis

Wiki

erstellen

auswählen

Status: Kein Wiki ausgewählt oder erstellt

Struktur Einzelne Seite Externe Seite CP-Lerninhalt SCORM-Lerninhalt Forum Wiki Dateidiskussion Ordner Bewertung Aufgabe Test Selbst-Test Fragebogen Einschreibung Kontaktformular

Kursbaustein(e)... Löschen Vershieben Kopieren Publizieren

Figure 72: This redesign recommendation brings the possibilities to either select an existing wiki or creating a new one together in one tabbed pane (in the current version the user has to leave the course editor to create a new wiki).

Screen with tabbed pane: Create wiki, the wiki On this screen (figure 72) the central message is directly given on-screen: “choose an existing wiki or create a new one”. A recommendation on the macro usability level is the introduction of the button “create wiki”. With this, surfing to the Lernressourcen-section is no longer necessary, and an on-the-fly generation is now possible for a wiki as users experienced it in the usability test when adding a single page or a forum. We recommend making a principle out of this: everything potentially being added to the course in an editor session should be producible there.

An on-the-fly generation is now possible for a wiki as users experienced it in the usability test when adding a single page or a forum

Nevertheless, we are aware of what a challenge it will be to develop this inter-module communication in OLAT; we emphasize to realize it because no test person could overcome this hurdle. And it is worth making a second principle out of this: Module borders should not be manifest as gaps in the user’s workflow; the graphic user interface should offer smooth bridges.

10 Redesign – Reflection and Recommendations Recording redundant data

Figure 73: (These forms are referred to as “A” and “B” in the text). Recording redundant data.

When a new course is added the system requires a title and a description in the form “A”; then, the initial course record is written in the database, and the course editor opens. In the course editor, again, a title, and furthermore, a long title and another description can be recorded in form “B”. While the title entered in “A” is automatically copied to form “B”, the description there is empty (figure 73). For the user it is not clear why this data can be recorded and edited in two different forms. Likewise, when this data is edited later in one of the mentioned forms, this modification will not be adopted by the other. The user asks him-/herself: “Which title is displayed in the navigation? Which title is displayed in the course’s main- tab? Where are my descriptions displayed?“. These multiple meta-data make sense when looking at the fact that SCORM-content can be imported. However, authors get confused by these redundancies when creating content directly in OLAT. A low-level usability improvement would be a complete hand-over of the form data between the forms (including description). A more complex approach would be to keep this data synchronized by triggering modifications.

For the user it is not clear why some data can be recorded and edited in two different forms

10 Redesign – Reflection and Recommendations Making over-encoded information plain text As described in the results of the expert evaluation, currently 4 possible states are represented by a falsely utilized metaphor (inspired by file system right management) with the potential to represent 1250 states. We change the encoding to short plain text making a corresponding key obsolete. The new values in the table should be clickable and connect to a dialog window enabling modification of this value (figure 74).

Currently 4 possible states are represented with the potential to represent 1250 states

Lernressourcen 4 Einträge Typ >

< Titel der lernressource > Frau Dr. Blumenthal mein OLAT-Kurs mein Ordner Ozeane der Erde - XL

< Urheber > boog boog boog mschuessler

< Zugriff > Nur Besitzer Besitzer + OLAT-Autoren OLAT-Benutzer ohne Gäste OLAT-Benutzer inkl. Gäste

< Erstellt > 17.04.07 13:08 01.05.07 13:20 19.04.07 14:54 26.10.06 15:12

Figure 74: Current design (screen 1), redesign recommendation (screen 2): Encoded information is changed to plain text.

10 Redesign – Reflection and Recommendations Screen: Create a group. Tabbed pane position: Members

Home Gruppen Lernressourcen Meine Gruppe

(0/0)

Meine Gruppe

Suche

Hilfe ? Log out

Meine Gruppe

Mitglieder Administration

Name und Beschreibung Mitglieder

Werkzeuge

Mitglieder

Masseneinschreibung

Mitglied hinzufügen

2 Einträge Wahl >

< Benutzername > boog usabilitystudent

< Vorname > Béatrice Peter

< Nachname > Boog Boog

There is a phrase in aviation that says, „what can be What can be mistaken mistaken will be mistaken“. That‘s exactly what we will be mistaken found to be true in usability testing by students when they had to add members to their work group: quite a few added group owners. We locate the cause for that in three problems in the current screen design: 1. The fieldset „Besitzer“ is the first to be shown; it is more dominant because there is already a person there (the owner); and in the fieldset „Besitzer“, as well as in the fieldset „Teilnehmer“, the add button is called „Benutzer hinzufügen“ instead of “Besitzer hinzufügen” resp. „Teilnehmer hinzufügen“. Please find in the left column our proposed solution which resolves these problems (figure 75). We did away with the distinction between „owners“ and „members“ on the fieldset level and instead created a global list of group persons, and introduced a new column called „Rolle“ in the table. So, there is only one ‚add’ button left.

< Rolle > Besitzer Teilnehmer

Alle auswählen

entfernen Anzeige Mitglieder

Mitglieder sehen Besitzer

Mitglieder sehen Teilnehmer

Nein

speichern

Figure 75: Current design (screen 1), redesign recommendation (screen 2) Changing ‘add owner’ and ‘add user’ to only one function: ‘add member’. The roles are assigned later.

10 Redesign – Reflection and Recommendations Home Gruppen Lernressourcen Meine Gruppe

(0/0)

Meine Gruppe

Ansichten

Mitglieder Administration

Name und Beschreibung Mitglieder

Werkzeuge

Mitglieder

Masseneinschreibung Mitglied hinzufügen Welchen Benutzer möchten Sie der Gruppe hinzufügen?

usabilitystudent Muster Alle auswählen Auswahl löschen

Nachname:

Peter

Boog

Teilnehmer

Vorname: entfernen Benutzernamen:

E-Mail-Adresse: Anzeige Mitglieder

Mitglieder sehen Teilnehmer suchen speichern

Ja Nein abbrechen

Kursinhalte anzeigen Kurseditor Kursvorschau anzeigen Detaileinstellungen anzeigen Ablageordner Systemmeldungen einstellen

Kursbausteine einfügen

2 Einträge Geben Sie > den Nachnamen, den Namen,>den Benutzernamen an>und weisen <Sie dem>neuen Mitglied Wahl < Benutzername < Vorname >oder die Mail-Adresse < Nachname Rolle eine Rolle zu! boog Béatrice Boog Besitzer

Rolle: Benutzer Mitglieder sehen Besitzer

Hilfe ? Log out

Suche

Struktur Einzelne Seite Externe Seite CP-Lerninhalt SCORM-Lerninhalt Forum Wiki Dateidiskussion Ordner Bewertung Aufgabe Test Selbst-Test Fragebogen Einschreibung Kontaktformular

In this overlay (figure 76), we have placed the name as the first search field instead of the user name because it conforms more to the line of thought of the searching person. The roles of the members are assigned here when looking them up. In the design of this overlay (current and redesigned) the overlay title is set against a grey background which wasn’t the case in other overlays. We recommend to uniformly apply this to all overlays.

Kursbaustein(e)...

Figure 76: The roles of the members are assigned when looking them up.

Löschen Vershieben Kopieren Publizieren

10 Redesign – Reflection and Recommendations Home Gruppen Lernressourcen Meine Gruppe

(0/0)

Meine Gruppe

Suche

Hilfe ? Log out

Meine Gruppe

Mitglieder Administration

Name und Beschreibung Mitglieder

Now, we would like to show you two redesign propositions for the mass subscription wizard; the difference between them lies in the handling of the distinction between the roles (members/owners).

Werkzeuge

Mitglieder

Masseneinschreibung Mitglied hinzufügen Mehrere Mitglieder zur Gruppe hinzufügen Schritt 1 / 3: Benutzernamen angeben. 2 Einträge

Wahl >

< Benutzername > boog Teilnehmer hinzufügen usabilitystudent

Wizard abbrechen

< Vorname > Béatrice Peter

Benutzernamen:

< Nachname > < Rolle > Boog Besitzer hinzufügen Besitzer Boog Teilnehmer

Benutzernamen:

minder Alle auswählensalvi Auswahl löschen initiator

In the first proposition (figure 77) that distinction is achieved by means of two text areas, in the second one (figure 78), the roles are assigned in the table only at the end by means of a selection menu.

entfernen

Anzeige Mitglieder

Mitglieder sehen Besitzer

Mitglieder sehen Teilnehmer

Nein

speichern

(EIn Name pro Zeile eingeben. Bsp.: administrator usabilitystudent1 user7 etc.)

Overlay, 3-step wizard, design option 1: Add several members to a group

(EIn Name pro Zeile eingeben. Bsp.: administrator usabilitystudent1 user7 etc.)

weiter

Home Gruppen Lernressourcen Meine Gruppe

(0/0)

Meine Gruppe

Suche

Hilfe ? Log out

Meine Gruppe

Mitglieder Administration

Name und Beschreibung Mitglieder

Werkzeuge

Mitglieder

Masseneinschreibung Mitglied hinzufügen Mehrere Mitglieder zur Gruppe hinzufügen Schritt 2 / 3: überprüfen und fertigstellen 2 Einträge

Wizard abbrechen

3 Einträge Benutzername > minderAlle auswählen salvi entfernen initiator

Auswahl löschen

< Vorname > Stefan Giorgio Auguste

< Nachname > Minder Delmonte Leduc

< Rolle > Teilnehmer Teilnehmer Teilnehmer

Anzeige Mitglieder

weiter Mitglieder sehen Besitzer

Mitglieder sehen Teilnehmer

Nein

speichern

Figure 77: Mass subscription redesign version 1; the roles are distinguished by two respective text areas.

10 Redesign – Reflection and Recommendations Home Gruppen Lernressourcen Meine Gruppe

(0/0)

Meine Gruppe

Suche

Hilfe ? Log out

Meine Gruppe

Mitglieder Administration

Name und Beschreibung Mitglieder

Werkzeuge

Mitglieder

Masseneinschreibung Mitglied hinzufügen Mehrere Mitglieder zur Gruppe hinzufügen Schritt 2 / 3: Rolle zuweisen und fertigstellen 2 Einträge

Wizard abbrechen

3 Einträge Benutzername > minderAlle auswählen salvi entfernen initiator

< Vorname > Stefan Auswahl löschen Giorgio Auguste

< Nachname > Minder Delmonte Leduc

< Rolle > Besitzer Teilnehmer Teilnehmer

Anzeige Mitglieder

weiter Mitglieder sehen Besitzer

Mitglieder sehen Teilnehmer

Nein

speichern

However, the second proposition would introduce an interaction design hitherto not found in OLAT: direct mutations in tables without save button (AJAX-style solution: save on change). We took care that the screen design of the redesigned wizard conforms to the principles already discussed. We condensed the information contained in the header (steps textual and graphical) into a block and changed the symbols indicating step number to 2D-icons (because non-clickable).

Figure 78: Mass subscription redesign version 2; the roles are directly assigned in the table.

10 Redesign – Reflection and Recommendations Making course access unambiguous

(0/0)

Home Gruppen Lernressourcen Lernressourcen Katalog

Scuhmaske Meine Einträge Kurse Tests Fragebogen CD-Lerninhalte SCORM-Lerninhalte Ressourcenordner Wikis Glossar

Importieren

Lernressourcen 4 Einträge Typ >

< Titel der lernressource > Frau Dr. Blumenthal mein OLAT-Kurs mein Ordner Ozeane der Erde - XL

Hilfe ? Log out

Suche

< Urheber > boog boog boog mschuessler

< Zugriff > Nur Besitzer Besitzer + OLAT-Autoren OLAT-Benutzer ohne Gäste OLAT-Benutzer inkl. Gäste

< Erstellt > 17.04.07 13:08 01.05.07 13:20 19.04.07 14:54 26.10.06 15:12

Kurs CP-Lerninhalte SCORM-Lerninhalte Test Fragebogen Wiki Glossar Andere Formate

The current listing of courses (after full text search, in the ‘my courses’ section a. s. o.) offers two ways of entering the course content: 1: clicking the course title, or 2: clicking the link “show course content”. The former brings up the meta data page (Detailansicht) on which a button and link “show course content” is offered. The latter goes directly to the first course page. As we have recommended above rearranging the meta-data page (Detailansicht), the course itself, the editor and the preview in a set of views, accessing the meta data page via course list becomes obsolete. Therefore, we recommend removing the respective column in the course list (figure 79). Clicking the course title then will be the unambiguous way to access the course content (first course page); all other views are available in the view set in the tool box.

Rearranging the metadata page (Detailansicht), the course itself, the editor and the preview in a set of views, accessing the meta data page via course list becomes obsolete

Herstellen Kurs Test Fragebogen Wiki Glossar

Figure 79: Current design (screen 1), redesign recommendation (screen 2): Making course access unambiguous. The link ‘Inhalt anzeigen’ has been removed. The first page of the course can be accessed by clicking the corresponding title in the list. The so called ‘Detailansicht’ formerly associated with this link is available through the toolbox ‘views’ introduced in figure 65.

100

10 Redesign – Reflection and Recommendations Making context help texts straight forward instructions As experts mentioned, the context help text they consulted weren’t helpful enough to resolve the current problem. We have redesigned the context help for “linking in a wiki” (figure 80), (this task could not be resolved by any test person; some of them read the existing context help). • The new help text is a straight forward instruction with the exact wording of the controls: the button “Wiki wählen” is clearly mentioned. • The new help text takes into account that there are two possible situations: either a wiki is ready to use in the Lernressourcen-Section, or a new one has to be generated. • The fieldset box has been removed because it’s unnecessary. • The title is a summary of the instructions. • The windows title declares that the window is a help text (and what for) which enables the user to recognize the respective button in the (windows-) taskbar. • Finally, we denoised the window appearance which makes the text more readable.

Help text must be straight forward instruction with the exact wording of the controls

Please note that the new context help text shown here is exemplary. If you can realize the redesign suggestion made for the add-wiki-procedure, the request to go to prepare the wiki outside the editor in the Lernressourcensection becomes obsolete. Figure 80: Context help reworded and denoised.

101

10 Redesign – Reflection and Recommendations 10.4 Self-evidence, self-explanatoriness, obscurity

How to start a programm? Welcome! click the button

Welcome! start

Degree 1: self-evident (button) and self-explanatory (instruction and button-wording)

Welcome!

> Degree 2: self-evident (through common sense)

Welcome!

Either, knowing how to interact with controls on screen is common sense, or it has to be learned by the user for the specific situation. The first case always should be the primary goal of designers (making the screens self-evident); however, it’s not always clear what knowledge belongs to common sense; also, it changes over time. Therefore on-screen instructions can help users (making screens self-explanatory). We suggest to grade every interaction pattern (especially the ones to be newly introduced) on a scale of 4 from self-evident including a self-explanatoriness backup (1) over only self-evident (2) and only self-explanatory (3) to obscure (4). See figure 81 in the left column. Of course, a grade 4 hurdle can be overcome by users by just knowing it (the supporter told me on the telephone!) or reading the help text. However, grades 1 and 2 are worthwhile.

We suggest to grade every interaction pattern on a scale of 4 assessing self-evidence and self-explanatoriness

click the blue then the red square to start the programm. Degree 3: not self-evident but self-explanatory through instruction, instructions absolutely necessary

Degree 4: obscure

Figure 81: A theoretic approach to classification of the severity of usability problems.

102

10 Redesign â&#x20AC;&#x201C; Reflection and Recommendations 10.5 Redesign recommendations â&#x20AC;&#x201C; Epilogue Expert evaluation and usability testing went along the path of tasks derived from the use cases formerly identified by the MELS-Team. Nevertheless, this elaborate containment of use cases ensured that quintessential screens and interactions were in focus, only a small part of OLAT was tested. To draw a valuable return on investment out of the consolidated findings, it is important to use them (including the redesign recommendation) as general principles applicable everywhere in OLAT, both for existing features and for upcoming ones. An advanced goal is the creation of revised or retrofitted guidelines and checklists for designers and developers to effectuate a sustainable impact.

To draw a valuable return on investment out of the consolidated findings, it is important to use them as general principles applicable everywhere in OLAT, both for existing features and for upcoming ones

Such guidelines can be based on the principles shown in this report; some are marked with a red line.

103

Usability Testing and Evaluation, and Redesign Recommendations for the LMS OLAT. Report ÂŠ 2007 University of Bern, Institute of Medical Education (IML)

11 - Wizard for Authors 11. Doing successful business with novice authors with a wizard (0/0)

Home Gruppen Lernressourcen Home Einstellungen Kalender Benachrichtigungen Bookmarks Persönlicher Ordner Notizblätter Leistungsnachweise Andere Benutzer

Suche

Hilfe ? Log out

Willkommen bei OLAT Create Express-Course Step 1 / 6: Welcome

Welcome to the Express Course Creator. OLAT Schnellstart

With this function you can setup a standard course where you can upload your lectures’ documents and offer a discussion forum to your students. OLAT Einführungsfilm Expresskurs anlegen Kurs finden

cancel

Diese Startseite konfigurieren

It might cost OLAT-connoisseurs and – developers quite an effort to offer a tool which reduces all the features to a tiny selection. But when novice authors with medium or low computer literacy are doing their first steps in OLAT, the tool “Create Express Course” enables them to have success within the first minutes, which could make them go for more (figures 82-87). «I’ve always found it useful to imagine that every time we enter a web site, we start out with a reservoir of goodwill. Each problem we encounter ... lowers the level of that reservoir.»

When novice authors with medium or low computer literacy are doing their first steps in OLAT, the tool “Create Express Course” enables them to have success within the first minutes

The core of the “Create Express Course” tool is that absolutely no configuration is necessary, nor is it offered in the wizard dialog. Everything has to be set up with standard parameterization ensuring that students can access the content. Of course, everything can be re-parameterized, and the classic way extended after the express setup.

The user clicks the button “Create Express Course” in the portlet “OLAT Quick Start” on OLAT’s first screen the following steps occur in an overlay that opens:

1. «Welcome to the Express Course Creator. With this function you can setup a standard course where you can upload your lectures’ documents and offer a discussion forum to your students.»

Krug, S. (2005). Don’t make me think. Berkeley CA, New Riders Publishing

104

11 - Wizard for Authors

Home Gruppen Lernressourcen

(0/0)

Suche

Hilfe ? Log out

Home Home Gruppen Lernressourcen

(0/0)

Suche

Hilfe ? Log out

Einstellungen

Home Kalender Einstellungen Benachrichtigungen Kalender Bookmarks Benachrichtigungen Persönlicher Ordner Bookmarks Notizblätter Persönlicher Ordner Leistungsnachweise Notizblätter Andere Benutzer Leistungsnachweise Andere Benutzer

Willkommen bei OLAT

Create Express-Course Willkommen bei OLAT Step 2 / 6: Title of the express course Create Express-Course

Diese Startseite konfigurieren

Step 3 / 6: Welcome message

Diese Startseite konfigurieren

Please recordOLAT the title of your express course: Schnellstart Please recordOLAT a welcome message / synopsis attn. your students: Schnellstart

2. «Please record the Title of your Express Course» 3. «Please record a welcome message / synopsis attn. your students» (A simple text area, but not the HTML-Editor is offered to keep it simple!) 4. «Please record the title of the folder, to which you will upload the lectures’ documents»

OLAT Einführungsfilm OLAT Einführungsfilm Expresskurs anlegen Expresskurs Kurs finden anlegen Kurs finden

back

(0/0)

Home Gruppen Lernressourcen Home Einstellungen Kalender Benachrichtigungen Bookmarks Persönlicher Ordner Notizblätter Leistungsnachweise Andere Benutzer

Suche

Hilfe ? Log out

Willkommen bei OLAT Create Express-Course Step 4 / 6: Create folder for documents

Diese Startseite konfigurieren

Please recordOLAT the title of the folder, to which you will upload the lecutres’ documents Schnellstart OLAT Einführungsfilm Expresskurs anlegen Kurs finden

back

105

11 - Wizard for Authors

(0/0)

Home Gruppen Lernressourcen Home Einstellungen Kalender Benachrichtigungen Bookmarks Persönlicher Ordner Notizblätter Leistungsnachweise Andere Benutzer

Suche

Hilfe ? Log out

Willkommen bei OLAT Create Express-Course Step 5 / 6: Create discussion forum

Diese Startseite konfigurieren

Do you want to offer a discussion forum to your students? OLAT Schnellstart yes

OLAT Einführungsfilm Expresskurs anlegen Kurs finden

back

6. «Your Express Course is now ready to use. We sent you an e-mail containing two important internet addresses: Do you want to upload files now?»

(0/0)

Home Gruppen Lernressourcen Home Einstellungen Kalender Benachrichtigungen Bookmarks Persönlicher Ordner Notizblätter Leistungsnachweise Andere Benutzer

5. «Do you want to offer a discussion forum to your students?» If yes is selected, the forum is generated with standard parameterization, and no further questions concerning the forum setup are asked. The forum title is generated from the course title, there is general read/write access, the Express Course creator is the only owner and automatically subscribed (e-mail) to forum messages.

Suche

Hilfe ? Log out

Willkommen bei OLAT Create Express-Course Step 6 / 6: Feedback and Upload

Diese Startseite konfigurieren

Your ExpressOLAT Course is now ready to use. We sent you an e-mail containing two important internet Schnellstart addresses: • Where your students can directly access the course: OLAT Einführungsfilm

http://www.olat.unizh.ch/7657665759 Expresskurs anlegen • Where you can directly access the page to upload your files:

http://www.olat.unizh.ch/3565365888 Kurs finden

print this page Do you want to upload files directly after clicking finish? yes

finish

Figures 82 - 87: Wizard for novice authors.

106

11 - Wizard for Authors More situations in OLAT have to be identified where wizards could contribute to improved usability. Inspirations where wizards can be useful: «Problem: The user wants to achieve a single goal but several decisions need to be made before the goal can be achieved completely, which may not be know to the user Use when: A non-expert user needs to perform an infrequent complex task consisting of several subtasks where decisions need to be made in each subtask. The number of subtasks must be small e.g. typically between ~3 and ~10. The user wants to reach the overall goal but may not be familiar or interested in the steps that need to be performed. The task can be ordered but are not always independent of each other i.e. a certain task may need to be finished before the next task can be done. To reach the goal several steps need to be taken but the exact steps required may vary because of decisions made in previous steps. Solution: Take the user through the entire task one step at the time. Let the user step through the tasks and show which steps exist and which have been completed.»

107

Welie, M. (accessed 05.2007) Patterns in Interaction Design. http://www.welie.com/patterns/showPattern.php?patternID=wizard

12 Opportunities and Dangers Using AJAX 12. Reflections about the opportunities and dangers in using AJAX The AJAX-technology opens new possibilities of interaction design in web applications because page content can be incrementally refreshed and altered without initiating a new page to be loaded. Each mouse click and key stroke can evoke database requests in the background, so that a web application is enabled to react as a desktop application could, namely, with on-the-fly generated contextual responses. In other words, wherever the user clicks with whatever mouse key (of course, thatâ&#x20AC;&#x2122;s true for any keyboard action, too), with JavaScript any event can be caught, and thanks to its use in an AJAX-framework, any imaginable reaction can be produced. This huge diversity of possible interaction patterns makes it necessary to make a narrow selection out of it. Not everything possible is useful; many preposterous eventreaction combinations can be easily identified as such: imagine setting off a save-data-procedure on a mouse wheel action. Nevertheless, it is not easy to make a selection of new interaction patterns out of the subset of the seemingly expedient ones.

The huge set of possible AJAX-style interactions must be broken down into a narrow set containing only commonsense event-action pairs

The new possible interaction patterns must be implemented according to a strict concept. In the AJAX-focus the concept defines these interaction patterns on a small level of granularity: the event-reaction pairs. Usability Testing and Evaluation, and Redesign Recommendations for the LMS OLAT. Report ÂŠ 2007 University of Bern, Institute of Medical Education (IML)

108

12 Opportunities and Dangers Using AJAX The huge set of possible AJAX-style interactions must be broken down into a narrow set containing only commonsense event-action pairs. The concept has to define the following aspects: 1. In which situations/positions which events are offered as interaction possibility 2. How the situation/position is indexed, so that the user knows that that he/she can interact, causing a certain event: " explicitly " implicitly " commonsense " metaphorical 3. What happens when a certain event is triggered 4. How a user can undo a triggered event 5. How a user is informed that an execution of a transaction is finalized 6. What alternative mouse and/or keyboard actions should be available to cause the same behaviour of the system 7. How accessibility is granted to disabled users with or despite the new event-action pairs Letâ&#x20AC;&#x2122;s characterize these aspects by showing examples: 1. Wherever you click with the right mouse key on a Windows PC, in the majority of cases there is a context menu appearing, offering tools and functions in the context of the clicked object. This clear concept has been internalized by many users. Using it in web applications means defining a set of rightUsability Testing and Evaluation, and Redesign Recommendations for the LMS OLAT. Report ÂŠ 2007 University of Bern, Institute of Medical Education (IML)

109

12 Opportunities and Dangers Using AJAX

3. 4. 5.

6. 7.

clickable objects, e. g. course elements in the content navigation and offering contextual tools as open element, edit element, delete element, move element a. s. o. When offering this context menu (see 1.) , the user has to know that he/she can invoke e. g. edit something on a right mouse click, especially if that were the only way to do so. And: in what does the look of a right-clickable item differ from a non-right-clickable one? The same is valid for keyboard shortcuts: the desktop-way to index them is displaying them adjacent to the classic menu entries. When defining that a right-mouse click brings up a context menu, this event is “sold out” and cannot be used with another consequence. When does the called-up context menu disappear? (Close icon, on mouse-out, on time-out, on a longer mouse-out, on outside click…). The use of AJAX can make send / save buttons obsolete. E. g. when a selected menu is changed, the new value can directly be saved (on change) to the database. But many users will search for the send/ save button because this is still common sense in web forms. For example, when tools are available on right click, there should still be a classic steady on-page menu with the same left clickable tools. Imagine a calendar where a new appointment can be added graphically by drawing a box with the left mouse key held down. As a matter of fact, that is

110

12 Opportunities and Dangers Using AJAX what many test persons already expected to be possible in the OLAT calendar. See videos at: http://usability.unibe.ch/olat/

username: olatstakeholders password: useresults To grant accessibility to disabled users with helper software, there must be a “classic” click-by-click way to do it. The use of AJAX-style interaction patterns must stick to a precise concept and the concept must stick to commonsense interaction patterns. When introducing AJAX-style interaction in an established environment it can be done step by step, or a new interface can be launched as a whole (e. g. see Yahoo Mail). In the first approach, there is the problem, that old style and AJAX-style interaction patterns will be mixed, and users do not know what to expect. The second approach takes an enormous effort. Patch-working AJAX-style interaction patterns with old style interaction patterns baffle users and decreases the quality of usability

Figure 88: Examples of right mouse click actions triggered with AJAX (Screenshots taken from the calendar application ZIMBRA).

However, the AJAX-technology could be used as a helper to enable inter-module communication in rather rigid web application frameworks without introducing new interaction patterns. Although users would interact classically (videlicet by left clicked links, icons and buttons), (figure 88) , AJAX-technology could act in the background and integrate distributed controls. The obvious example in OLAT is the consolidation of the two-step publishing procedure for courses.

The use of AJAX-style interaction patterns must stick to a precise concept and the concept must stick to common sense interaction patterns

Patch-working AJAXstyle interaction patterns with old style interaction patterns baffle users and decreases the quality of usability

111

12 Opportunities and Dangers Using AJAX «... perception requires a cultural package of pre-existing designs because only in exceptional cases can a human being perceive something that hasn’t already been conveyed to him visually or in written form.» However, AJAX-style interaction patterns could become common sense in web applications and are, therefore, a sales argument. Hence, some interaction examples appear below as a source of inspiration. See video at: http://usability.unibe.ch/olat/

username: olatstakeholders password: useresults

Burckhardt, L. (2006) Warum ist Landschaft schön? Die Spaziergangswissenschaft. Hrsg. von Markus Ritter und Martin Schmitz. Berlin, Martin Schmitz Verlag.

112

Acknowledgments Acknowledgments For our usability investigation, the object of examination, OLAT, is stimulating, enticing, and last but not least, challenging thanks to its richness in features and the interplay between the roles of author and student. Century-old forms of cooperation between teacher and student are being successfully mapped to and practiced within the merely 12-year old medium called the internet. We took much pleasure in the fact that the University of Zurich commissioned the Institute of Medical Education to conduct the usability investigation. The usability laboratory team members would like to extend their thanks to:

Cartoon 8: By Hans Holzherr.

The OLAT team of the University of Zurich; especially Marion Weber and Dr. Hans-Jörg Zuberbühler for the most welcome assignment, the stimulating cooperation, and for their untiring willingness to work out the details of the test. The Usability Experts of the IML; Prof. Dr. Sissel Guttormsen for the direction of the project; Peter Frey, MD, and Dr. Samuel Schluep for their appreciated collaboration in processing the expert evaluations. The staff members of the Education and Media Unit (Abteilung für Unterricht und Medien); notably Hans Holzherr for the translations and the cartoons, Andreas Clemann for the Usability Lab infrastructure, Lukas Kaiser for the collaboration in reporting, and

113

Acknowledgments Giovanni Ferrieri as our consultant in video processing. The Institute of Educational Sciences (Institut für Erziehungswissenschaften); Martin Studer and Ruedi Wettstein for the conduction of concept and interaction comparisons with the LMS ILIAS. IML’s Curriculum Coordination Unit (Abteilung für Studienplanung); Heidi Christen for the editing of the questionnaire set. The test persons: teachers and students of the University of Bern who for 60 minutes each utilized OLAT intensely and recorded their appreciated opinions on video and questionnaires. Thank you! Your IML Usability Laboratory Team

Béatrice Boog

Stefan Minder

Béatrice Boog

Stefan Minder

Certified designer for graphic art-work FH, and e-learning developer, adult education trainer HF

Biologist MSc, grammar school teacher, and e-learning developer

114

Masthead

Publishing of photos and videos of the test persons only with written consent of the Institute of Medical Education.

Prof. Dr. Sissel Guttormsen University of Bern Institute of Medical Education (IML) Education and Media Unit (AUM) Inselspital 38 3010 Bern Switzerland http://usability.unibe.ch usability@iml.unibe.ch ÂŠ IML 2007

Usability Testing and Evaluation, and Redesign Recommendations for the LMS OLAT. Report ÂŠ 2007 University of Bern, Institute of Medical Education (IML)

115