engage: University of Michigan Clinical Research Usability Report Team Envolve (Pratibha Bhaskaran, Liz Blankenship, Tammy Greene) SI 622 • Assignment 7 • March 27, 2008
Team Envolve • Engage U - M Clinical Research
Table of Contents
Introduction
3
Methodology
4
Overview
6
Key Findings
6
Conclusion
13
Appendix
14
!
Team Envolve • Engage U - M Clinical Research
2
engage: U - M Clinical Research
“Engage: Your Gateway to Clinical Research throughout the University of Michigan”
Introduction Engage is an initiative at the University of Michigan that matches volunteers for clinical research with study teams. It provides general information about clinical research, a way to search for studies and a way for potential volunteers to enroll by providing personal and medical information that is matched with studies seeking volunteers. The target population for Engage consists of two groups - researchers looking for participants in their studies, and people interested in volunteering for clinical research. This project focuses on the latter population, and aims to improve the user experience of the web site for them. The aim of conducting the usability tests was to observe how actual users - both current Engage users, and new users - interact with the site, and identify areas of difficulty. Through the process of creating a generalized transition network, personas and scenarios, conducting comparative and heuristic evaluations, and administering a survey, we had identified certain key issues. Apart from confirming these findings, we aimed to discover new issues through our usability tests. Our recurrent observation that the text-heaviness of the Engage site impeded navigation was confirmed by one user commenting on the amount of text on the website, and two users spoke of the length of the enrollment process. A new finding was that the extremely similar visual design and layout of the Engage site and its parent site, the University of Michigan Health System (UMHS), caused confusion for users as they frequently ventured out of the Engage site (and onto the UMHS site), and didn’t realize that they were no longer on Engage. Users also commented on the lack of organization of the display of studies they might be eligible for in both the registry and on the find studies, and felt this could be presented with more information, represented in a easily scannable form. Team Envolve • Engage U - M Clinical Research
3
engage: U - M Clinical Research Methodology Overview For our usability testing, we first piloted the test with a single user. After refining our test script and changing to a different type of screen capture software, we tested four users with the Engage site using a think aloud procedure to capture their thoughts and intentions as they were using the system. Setup and Logging We used a MacBook Pro laptop running Snapz Pro X to record the user’s voice and capture screen movement in a video during our testing. We also used Mozilla Firefox for the browser. The location of the test varied. A moderator sat a few feet away from the test subject, reading the test script and interacting with the user as necessary. A note taker, sitting several feet back from the user off to the side, used a form to record information about tasks (see Appendix A). This information included the start and stop time for the task, whether or not assistance was needed, how the task was accomplished and if the task was completed. Participant selection The pilot tester and usability test subjects were recruited primarily by word of mouth with the exception of one experienced Engage user who responded to our recruitment e-mail (see Appendix B). We tried to recruit a variety of users and we were moderately successful, with one undergraduate student, one University of Michigan technical research staff, a graduate student, and a clerk at the hospital. We unfortunately were not able to recruit any subjects who had used the site previously as a patient interested in studies about his or her condition. Pre-Test Procedures Before completing any tasks, we asked the users to sign an informed consent form (see Appendix C). We also asked several demographic questions, some verbally as part of the pre-test script, and some on a paper pre-test questionnaire (see Appendix D).
Team Envolve • Engage U - M Clinical Research
4
engage: U - M Clinical Research Tasks We had our users try several tasks using the Engage website while thinking out loud about what they were doing. First, they were asked questions to prompt them to seek information about clinical research in general. We asked them to find information about why clinical research is needed and what they should know about the general process of clinical research before participating in a study. These questions, intended to lead them to the “About Research” and “Questions/Answers” sections, assessed the salience of these items as relevant to understanding what clinical research is. Next we had users search for studies, first for a hypothetical friend who was diagnosed with lung cancer, and then for a study that they qualified for and were interested in. We asked questions after they pulled up specific studies to see that they understood what information was available about eligibility, procedures, risks, and benefits for participation. These tasks were intended to assess two different styles of searching, that common to individuals who already have a disease, and that common to healthy individuals who come to the site. Our final task involved enrolling in the Engage registry. We asked them to apply to make their information available for researchers to review and contact them. Once registered, we asked them what they would do next and what they thought they could do from within the registry. These questions aimed to assess their understanding of the registry. Post-Test Procedures After the final task, we asked users to fill out a Post-Test Questionnaire (see Appendix E, F) and asked several debriefing questions. Our strategy was to focus upon usability, learnability, and aesthetics, asking questions about what the users liked, what they found difficult, and asking them to rate the system on a number of different qualities.
Team Envolve • Engage U - M Clinical Research
5
engage: U - M Clinical Research Overview Through usability testing, we were able to verify and expand upon many of the findings from our previous research. We also discovered some issues with the site that had not yet drawn our attention. The major findings are • Engage is cumbersome to navigate • Engage lacks a visual identity distinct from that of UMHS • Engage lacks clear organization and presentation of information
Key Findings 1. Engage is cumbersome to navigate The usability tests confirmed several prior findings and revealed new findings that deal with the navigability of the Engage site. Observations of the users by the study team as well comments from users as they navigated through the site to complete the tasks confirmed or introduced the following issues: • The Engage site is too text heavy diminishing the prominence of its key navigational structures. • The Engage site contains too many intermediate pages complicating the accomplishment of simple tasks by increasing the number of clicks needed to get to the destination. • The inline text links on the Engage site are buried or “hidden” within the page. They are also used too frequently and often direct users away from the task they are trying to accomplish within Engage. As noted in previous reports, these types of issues can intensely complicate navigation on the Engage site by preventing users from finding useful content or accomplishing minute tasks such as finding the search page or the registry enrollment screen. During testing, users often became frustrated or disinterested when they were directed outside of the Engage site by links or did not
Team Envolve • Engage U - M Clinical Research
6
engage: U - M Clinical Research complete the task when expected due to intermediate pages. This is captured in the following screenshots and quotes:
Figure 1.1 “Engage Home Page” where user task to find studies begins
Figure 1.2 “Find Studies Page” - one click from homepage
Figure 1.3 “Quick Questionnaire Page” - two clicks from homepage Team Envolve • Engage U - M Clinical Research
7
engage: U - M Clinical Research
Figure 1.4 “Find Studies Search Page” - three clicks to reach destination from homepage
Relevant User Quotes: Describing the most difficult task, User 4 said, “Finding studies, which is the purpose of the site.” User 04 was redirected outside the Engage site due to links on the find studies page that hinted that she might be able to find studies there. When unexpectedly directed to the mandatory “question about you” survey, User 3 said, “I don’t wanna answer questions about me. I just wanna look [for studies].” Describing her first attempt to register User 2 said, “I had a hard time before getting to the registry page when I tried to do it myself, this actual registry thing, or else I would have already been registered.” Recommendations These issues should be addressed by removing all unnecessary content, inline text links and intermediate pages to simplify navigation and create an experience that the user expects and enjoys. Specifically, the following strategies could be implemented: • Remove redundant inline text links on the homepage • Remove “Start my search page” and immediately direct the user to the find studies page from homepage • Include a help link and external resources link in the sub-navigation on the find studies page • Reduce the content on the Engage registry page and increase the prominence of the login and friends account links Team Envolve • Engage U - M Clinical Research
8
engage: U - M Clinical Research 2. Engage lacks a visual identity distinct from that of UMHS The University of Michigan Health System (UMHS) (www.med.umich.edu) website is a parent site of Engage, and the two have an almost identical visual design and color scheme, barring the differences in text, the navigation links and the banner. The “Find Studies” page on the Engage site links to a page on the UMHS site (see figure 2.2), which provides a list of U-M organizations that might potentially be in need of volunteers.
Figure 2.1: “Find Studies” page on Engage; “check this page” redirect to a UMHS page (figure 2.2)
Team Envolve • Engage U - M Clinical Research
9
engage: U - M Clinical Research
Figure 2.2: “UMHS page” listing groups that might be in need of volunteers
The almost negligible lack of distinction between the two sites, however, is a source of confusion. Two of the three new users wandered outside the Engage site and did not immediately realize that they were no longer on Engage. They had to be guided back to the Engage website. Relevant User Quotes: After spending a significant amount of time browsing through external links, User 04 said, “If you keep rerouting people through various links, they are not going to read. They are going to get bored. You know very few people are committed to signing up for any research study. They are going to think, why [the hell] do I have to do this and leave.” Recommendations It is important for Engage to depict its association with UMHS, for purposes of credibility and trust; however, certain strategies could be employed to make the site distinctive: • Create an additional banner for Engage that could appear right below the current main banner • Ensure that external (non-Engage) web pages open up in a new tab or browser window Team Envolve • Engage U - M Clinical Research
10
engage: U - M Clinical Research 3. Engage lacks clear organization and presentation of information Especially when displaying large quantities of information, the pages of the Engage website can be difficult to digest because of having so much information and little organization. Users found that the display of studies on the site was confusing and not very useful, both on the part of site where users can search for studies and also later when the user looks for studies in the registry. Similarly, the Questions/Answers page is overloaded with information and would benefit from a clearer presentation. When you search the Engage database, results are organized only by condition regardless of the type of search you perform. This simple organization structure does not provide much aid to the user in filtering through the studies to find ones that they qualify for or might be interested in. User 3 expressed interest in being able to search or view by a number of factors surrounding the volunteer’s responsibilities: “If I could search it for what’s required of me...in terms of procedures and time frames and clinic visits, compensation...” Similarly in the registry, when you look for studies, all you see is an uncategorized list with the title of the study and condition. The lack of any organization is confusing, and one user commented that it was odd that the studies were not categorized again by condition like on the rest of the Engage site.
Team Envolve • Engage U - M Clinical Research
11
engage: U - M Clinical Research
Figure 3.1 “Engage Registry Possible Studies Page” - uncategorized list of studies
Relevant User Quotes: Describing what she expected to find once registered User 4 said, “Right now I’m a person who wants to be of any help to any research...I’m looking for a page that lists studies going on [for healthy volunteers] that need volunteers that I can pick from.” Giving his initial impressions of the find studies interface, “The content is not well organized for the purpose of reading. Some of the content is not helpful. The medical terms are not helpful to figure out if you qualify. And it doesn’t say who benefits or why it’s needed...It would be nice if the title were explained like an abstract.” Recommendations: • Make filtering searches by multiple criteria more intuitive, or present the information about studies in a way that is easily sorted and understood by the user. This could include additional columns, but be wary of cramming too much information on the screen. • In the Questions/Answers section, group questions and answers into categories, so it is easier for the user to find what they are looking for. e.g.: Payment, Duration, Risks, Safety. Team Envolve • Engage U - M Clinical Research
12
engage: U - M Clinical Research Conclusion Conducting usability testing was valuable in terms of confirming the existence of previously identified issues, introducing more specific examples and highlighting new findings in our evaluation of the Engage site. In addition to the key findings, additional findings and recommendations are listed in the appendix (See Appendix G). Our observation of the users revealed specific instances detailing their frustrations with the site as well as their contentment with it. Generally the issues fell into three overarching themes: navigation, identity, and organization and presentation of information. Based on the results of the usability test, specific recommendations were suggested to address the source of the users’ frustrations in an effort to create an environment for the users to easily accomplish their typical goals.
Team Envolve • Engage U - M Clinical Research
13
engage: U - M Clinical Research Appendix A. Engage Usability Observation Sheet: Task 1: Find information Intro/Task description: Suppose you are considering participating in clinical research and you are interested in finding more information about clinical research at the University of Michigan. You would like to know why anyone should participate in studies. First subtask: Use the Engage site to find out why clinical research is needed. [Task can be completed by finding information in two ways, one from information under the volunteer for research tab or under the Q&A tab] Second subtask: You now have some specific questions about the process of participating in clinical research. Use the Engage site to find what you should know prior to participating in a study. [Task can be completed by finding questions to ask on the Q&A page] Observations:
Start Time: ______________________
End Time: _______________________
Percentage Completed: ____________
Assistance Needed: ________________
Team Envolve • Engage U - M Clinical Research
14
engage: U - M Clinical Research Task 2: Find a study Intro/Task description A. Imagine a friend of yours was recently diagnosed with lung cancer. You would like to find studies that he or she would qualify to participate in. How would you go about it? --> What are the eligibility requirements for participating in this study? --> What procedures or risks are involved in this study? B. You are interested in participating in a study yourself. How would you find one that you are eligible for and that interests you? --> Why would you qualify for this study? --> What are the risks and benefits to participating (if any)? --> What would your next step be (to participate in the study)? (Once they find a study, ask:) What would your next step be? Observations:
Start Time: ______________________
End Time: _______________________
Percentage Completed: ____________
Assistance Needed: ________________
Team Envolve • Engage U - M Clinical Research
15
engage: U - M Clinical Research Task 3: Enrolling in the registry Intro/Task description You are interested in being notified when studies become available which you may be eligible for. Your goal is to go through the application process to make your information available for researchers to review and contact you. (Note: carefully avoiding the words enroll, register, registry) First subtask - finding the registry More detail given to the user if they need more than the goal given above: There are new studies being posted all the time on Engage. Rather than coming to check the site every day for new studies that you might qualify for, you want to have researchers contact you. Can you find some feature of the site that allows you to enter information in and save it so that researchers can see it, and contact you if they think you might qualify for a study? Second subtask - Consent forms Third subtask Please enter information as you would to normally register for the site. If there is any information you would like to omit or falsify because you do not wish to share it with us today, feel free to do so. 1. Register who? - Please register yourself. 2. Interested in which kind of study? - Answer whatever you would prefer, use the site as you would under normal conditions. 3. Which locations? - Any that you would like. 4. Compensation? - Use the site as you normally would. 5. Information made available to/preferred contact method/interested in an interview? - Answer as you normally would on the site, but since this is a test site, you will not actually be contacted by researchers or interviewers. 6. Personal info? - Please enter your personal information - you may omit or falsify information if you do not
Team Envolve • Engage U - M Clinical Research
16
engage: U - M Clinical Research Observations:
Start Time: ______________________
End Time: _______________________
Percentage Completed: ____________
Assistance Needed: ________________
Team Envolve • Engage U - M Clinical Research
17
engage: U - M Clinical Research B. Recruitment e-mail Dear Engage user, Thank you for recently taking our survey! You indicated toward the end of the survey that you might be interested in helping us with additional studies for this project. As our next phase we will be conducting a usability test, where we observe a participant using the website in order to gain an insight into how users interact with the site. This helps us understand user needs and identify problems in the system or ways in which we might be able to improve the ease of use. We are willing to work with your schedule if you would be willing to take some time to participate in our study. We are also offering cookies (or a healthy alternative) to compensate you for your time. We need your help! If you are interested in participating, please reply with the following information: 1) Are you a Windows or Mac user? (Or are you comfortable with both?) 2) What browser do you use? (Internet Explorer, Firefox, Safari, don’t know, comfortable with all?) 3) What are two or three times when you would be available between March 17th and March 24th? Thanks once again for filling out our survey, and we hope to hear from you regarding our usability testing! Sincerely, Pratibha Bhaskaran, Liz Blankenship and Tammy Greene SI 622 Team working to improve the usability of Engage
Team Envolve • Engage U - M Clinical Research
18
engage: U - M Clinical Research C. Informed Consent: Study Administrator: Pratibha Bhaskaran, Liz Blankenship, and Tammy Greene First and foremost this is a test of the website—we are not testing you! We want to find out what aspects of the website are confusing so that we can make it better. This is a study about the University of Michigan’s Engage website (http://umengage.org). The Engage website is primarily intended to help volunteers get involved in clinical research at the University of Michigan. The site includes general information about research, ways to search for studies, and a way to let researchers contact you about studies. Our goal is to make the website appealing, intuitive, and user-friendly. Your participation will help us accomplish this goal. In this session, you will be working with the live version of the website as well as a test site. We’ll ask you to try several things that people typically do on the site, such as locate a study based on a keyword search or subject search. One or two members of the usability team will sit in the same room, quietly observing the session and taking notes. A session facilitator will sit near you and help you if you experience difficulties or have questions. All information we collect concerning your participation in the session belongs to the SI 622 Engage project group and will be used for our internal business purposes. With your permission, we plan to audio tape the session and record your screen movements. We may publish our notes from this and other sessions in reports, but all such observations will be confidential and will not include your name.
Statement of Informed Consent I have read the description of the study and of my rights as a participant. I voluntarily agree to participate in the study. Print Name: _______________________________ Signature: ________________________________ Date: ____________________________________
Team Envolve • Engage U - M Clinical Research
19
engage: U - M Clinical Research D. Ask out loud after Pre-Test Questionnaire: What is your primary occupation? (questions in maroon to be asked out loud) o Student o University Staff o University Faculty o Other, describe: Is your occupation in any way related to medicine or health? o Yes _________(explain briefly) o No What is your primary (current or former) affiliation with the University of Michigan, if any? o Undergraduate student o Graduate student o Faculty o Staff o No affiliation (Notetaker should record M/F) Pre-test Questionnaire 1. Which age group do you belong to? o Under 18 o 18 to 25 o 26 to 35 o 36 to 50 o 51 to 65 o over 65 2. How much time do you spend on the internet every week? o Less than an hour o 1-5 hours o 6-10 hours o 11-25 hours o More than 25 hours weekly
Team Envolve • Engage U - M Clinical Research
20
engage: U - M Clinical Research 3. How often do you use engage? o first-time o Almost daily o 1-3 times a week o 1-3 times a month o 1-3 times a year o Less than once a year 4. How would you rate your web browsing ability? Low……….1.……2……3……4……5……High
Team Envolve • Engage U - M Clinical Research
21
engage: U - M Clinical Research E. Post-Test Questionnaire Please answer the following questions to the best of your ability based on your experience with today’s task. We appreciate your time and your participation will help us to better understand how we might improve the Engage website.
Team Envolve • Engage U - M Clinical Research
22
engage: U - M Clinical Research 13. Using the rating chart below, please circle the number that closely matches how you feel about the Engage website. Complex……….1.……2……3……4……5……Simple Unreliable..........1…….2……3……4……5……Reliable Difficult to Use…1…….2……3……4……5……Easy to use Unfriendly………1…….2……3……4……5……Friendly Unprofessional…1…….2……3……4……5……Professional Low Quality…1…….2……3……4……5……High Quality Dislike…………..1…….2……3……4……5……Like 14. What did you like most about the website? __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ 15. What did you like least about the website? __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ 16. Is there anything you expected to be able to do on the website and couldn’t? __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ 17. Is there one thing that you would like the designers to change? What is it? Why? __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ 18. Would you use the website again? __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ 19. Additional comments. __________________________________________________________________________ __________________________________________________________________________
Team Envolve • Engage U - M Clinical Research
23
engage: U - M Clinical Research F. Questionnaire Data Verbal pre-test
User #
Is your occupation in any way What is your pri- related to medimary occupation? cine or health?
Gender
What is your primary (current or former) affiliation with the U of M, if any?
3
M
University Staff
2
F
Student
yes - pharmacology research student/staff undergraduate no student
4
F
Student
Yes - remotely; research dealing with drug technology, healthrelated Graduate student
1
F
University Staff
No
Staff
Pre-test questionnaire
How much time Which age group do you spend on How would you do you belong the internet every How often do you rate your web to? week? use engage? browsing ability?
User #
3
18-25
11 to 25
first-time
3
2
18-25
11 to 25
first-time
4
4
18-25
1-3 times a year
3
1
36-50
11 to 25 More than 25 hours weekly
Team Envolve • Engage U - M Clinical Research
1-3 times a week
5 Average: 3.75
24
engage: U - M Clinical Research I felt the Initially upon task of I think that I using the Finding would like website, I InformaUser to use this felt it was tion was # system easy to use. easy
After completing a few I felt the tasks and task of learning I felt the Enrolling more about task of Find- in the reg- the website, ing a study istry was I felt it was was easy. easy. easy to use.
3
2
3
4
3
1
1
2
5
4
3
5
5
4
4
3
2
3
4
3
3
1
4 3.5
4 3.25
4 3.5
3 3.75
2 2.75
4 3
I felt the amount of explanation on each screen was adequate for User performing # the tasks.
I felt the website organization and section titles were easy to understand.
I would imagine that I think this most people I would will be a would learn recomgood re- to use this mend this source for system very site to volvolunteers. quickly. unteers.
I think volunteers will feel comfortable using this site.
3
3
1
1
2
4
3
2
5
5
5
4
5
5
4
2
3
3
4
5
3
1
3 3.25
4 3.25
5 3.5
5 3.75
5 4.75
5 4
Team Envolve • Engage U - M Clinical Research
25
engage: U - M Clinical Research Rate from 1 to 5
Average rating
Complex ‌. Simple
2.75
Unreliable...Reliable
4.5
Difficult to use...Easy to use
3.5
Unfriendly...Friendly
4.25
Unprofessional...Professional
4.5
Low Quality...High Quality
3.75
Dislike...Like
3.5
Free response questions from questionnaire Us er
Is there anything Is there one thing you expected to that you would like What did you like What did you like be able to do on the designers to Would you use F most about the least about the the website and change? What is the website website? website? couldn't? it? Why? again? It's linked to a very extensive database of studies. I didn't think Its often short on I wanted to sign No. I would deany studies were information and up for an e-mail cide it was too missing from the contains hidden notification of new complicated to 3site. links. studies. do studies. I had a hard time The amount of finding my way to explanation of the actual regisresearch trials tration page, and what inwhich (when I Absolutely, I volvement in previously went to really appreciate them entails, as Engage b/c of an the "find studies" well as accounts, email I got) I had Put a more direct link and all it that it offers for come to the page link to the registra- leads to, and I'll people new to the specifically in tion sequence be returning to it 2experience. search of. No. (reason above). regularly! Look for an appropriate study due to lack of Routing people enough time to The listing of cur- through a million spend on learning 4rent studies links the website Do not be wordy Maybe
Team Envolve • Engage U - M Clinical Research
26
engage: U - M Clinical Research
1Ease of use
New site isn't laid out/planned well. Studies are not grouped with like studies No.
Team Envolve • Engage U - M Clinical Research
Add number of pages to "registry" so users aren't frustrated and overwhelmed with information needed. Maybe tell users how long to expect registry to take and that CPI number will be needed. Give option to come back without losing info. Yes
27
engage: U - M Clinical Research G. Other General Findings
Finding
Evidence
Enrollment process is
User 1: “Some pretty in depth screening. I think it might be off
too long
putting for some people. Like, it’s, you know, too many pages,
Severity High
it’s too long, I just want to put my name in and they can contact me.” The concept of what
User 3 kept wondering where or how all of this was going to
the registry is sup-
help him with getting onto an e-mail list.
posed to do is unclear
User 1 did not seem to understand what the registry was and
from users’ interac-
called it the “new site.”
High
tions with the site Engage is too text
User 3: “I don’t usually read pages...I’m finding for this site
heavy, users stop read-
I’m not reading them close enough to get the right things I
ing every word and
need.”
Medium
miss links. When users are search-
Several users encountered studies they didn’t
ing or in the registry,
qualify for, either because of the age range or even
they are shown studies
because they didn’t have a condition required for
they do not qualify for
eligibility.
Visual identity and
Several users navigated off of the Engage site, to realize several
indication of site
clicks later they had left the site. In addition, User 3 said, “...
structure/context is
organization is not consistent...sometimes I may have to click
lacking
tabs up top, sometimes I may have to find a text link, some-
Medium
Medium
times I may have to click a button at the bottom... there is no one system to it, so every time I had to read carefully whatever instructions were there, and so that just makes the whole thing take much longer”
Team Envolve • Engage U - M Clinical Research
28
engage: U - M Clinical Research Finding
Evidence
Enrollment process
User 3 knew his height in centimeters but was not sure about
asks for information
feet. User 1 and User 2 didn’t know their CPI, but were willing
users may not have
to give it.
accessible, but are will-
User 1 while on CPI page: “I don’t think that people in general
ing to give
are going to have this information, or know it all. Or be willing
Severity Medium
to give it.” Engage contains too
Navigating through extra pages takes longer than if there were
many intermediate
a more direct path. The mandatory survey before searching,
pages
for example, was found to be intrusive
Inline text links are
On the find studies page, several users clicked inline links that
confusing
took them outside of the Engage site, rather than clicking the
Low
Low
“Start my Search” link
Recommendations Displaying studies Use the detailed information in the Engage database to help users filter studies. The existing Keyword search is a great start that users don’t discover easily and it should be highlighted. Enrolling in the registry The enrollment process should at a minimum tell the user how long the process will take, and ideally, it should make it possible to save, exit, and return at any point without having to fill in all of the required fields of every step of the process. Navigation and general site design Clean up the site design by providing a clearer structure, with less text and clearer emphasis on helping users accomplish their goals.
Team Envolve • Engage U - M Clinical Research
29
engage: U - M Clinical Research H. Detailed Script Intro script: So, let's begin! As I said before, my name is __ and I'm part of a team working on recommendations for improving Engage. Thanks for your time today. As you may know, Engage has been working on a project to improve their website. The Engage website is primarily intended to help volunteers get involved in clinical research at the University of Michigan. The site includes general information about research, ways to search for studies, and a way to let researchers contact you about studies. Today, I’ll have you go through a few tasks with these features, and then answer some questions at the end. Although we’ll be observing you, please remember that we’re testing the system, not you. Just so you know, when the tasks start we will record your screen movements and your spoken comments. We do this to keep a more accurate record of our evaluation of this product. Here is an informed consent form we’d like you to read and sign that says we’re recording you and that you will remain anonymous. <INFORMED CONSENT> We also have a quick pre-test questionnaire for you to take. <PRETEST QUESTIONNAIRE>
Ask out loud after Pre-Test Questionnaire: * What is your primary occupation? (questions in maroon to be asked out loud) o Student o University Staff o University Faculty o Other, describe: * Is your occupation in any way related to medicine or health? o Yes _________(explain briefly) o No * What is your primary (current or former) affiliation with the University of Michigan, if any? o Undergraduate student Team Envolve • Engage U - M Clinical Research
30
engage: U - M Clinical Research o Graduate student o Faculty o Staff o No affiliation (Notetaker should record M/F)
While you’re doing the task, we’ll ask you to ‘think aloud’. This means that as you are working, we’d like you to say what you are thinking and doing.
Feel free to take as long as you need to in completing the tasks. Also, as you move through the tasks, we ask that you try your best without our help and may not offer you help right away even if we notice you struggling on something. Please don’t take any offense, and if you really do need help to move to the next task, don’t worry, we’ll help! Do you have any questions before we begin? <Start recording software, open the website for them, http://www.umengage.org, using their preferred browser> Tasks: Task 1: Find information Intro/Task description: Suppose you are considering participating in clinical research and you are interested in finding more information about clinical research at the University of Michigan. You would like to know why anyone should participate in studies. You want to know what you would have to go through if you decided to participate in a study. First subtask: Use the Engage site to find out why clinical research is needed. [Task can be completed by finding information in two ways, one from information under the volunteer for research tab or under the Q&A tab] Second subtask:
Team Envolve • Engage U - M Clinical Research
31
engage: U - M Clinical Research You now have some specific questions about the process of participating in clinical research. Use the Engage site to find what you should know prior to participating in a study. [Task can be completed by finding questions to ask on the Q&A page]
Task 2: Find a study Intro/Task description A. Imagine a friend of yours was recently diagnosed with lung cancer. You would like to find studies that he or she would qualify to participate in. How would you go about it? B. You are interested in participating in a study yourself. How would you find one that you are eligible for and that interests you? (Once they find a study, ask:) What would your next step be? Task 3: Enrolling in the registry Intro/Task description You are interested in being notified when studies become available which you may be eligible for. Your goal is to go through the application process to make your information available for researchers to review and contact you. (Note: carefully avoiding the words enroll, register, registry) First subtask - finding the registry More detail given to the user if they need more than the goal given above: There are new studies being posted all the time on Engage. Rather than coming to check the site every day for new studies that you might qualify for, you want to have researchers contact you. Can you find some feature of the site that allows you to enter information in and save it so that researchers can see it, and contact you if they think you might qualify for a study? *****When they reach the registry: say “Ok, great, now before you enroll, I’m going to change over to the test site- that way you don’t have to worry about any real consequences coming about from your actions today. No researchers will contact you. Please do use the site as you normally would, however.”**** Second subtask - Consent forms
Team Envolve • Engage U - M Clinical Research
32
engage: U - M Clinical Research Third subtask Please enter information as you would to normally register for the site. If there is any information you would like to omit or falsify because you do not wish to share it with us today, feel free to do so. 1. Register who? - Please register yourself. 2. Interested in which kind of study? - Answer whatever you would prefer, use the site as you would under normal conditions. 3. Which locations? - Any that you would like. 4. Compensation? - Use the site as you normally would. 5. Information made available to/preferred contact method/interested in an interview? Answer as you normally would on the site, but since this is a test site, you will not actually be contacted by researchers or interviewers. 6. Personal info? - Please enter your personal information - you may omit or falsify information if you do not
Now that you're enrolled in the Engage registry, what would you do next? Could you please talk about what you think you could do from in here? <post-test questionnaire> De-briefing How do you feel about participating in clinical research after your experience on Engage? What did you find difficult about the tasks? Do you have any other comments?
Team Envolve â&#x20AC;˘ Engage U - M Clinical Research
33
engage: U - M Clinical Research I. Time Estimates for Tasks User
Task 1
Task 2
Task 3
User 01
Didn’t Complete: No assistance: 1: 1:00 mins (1) 2: 0:56 mins (1)
Completed: No assistance: 1: 1:39 mins (8) 2: 0:57 mins (6)
User 02
Completed: Assistance needed: 1: 3:00 mins (12) 2: 0:45 mins (2)
Completed: No assistance: 1: 1:53 mins (11) 2: 1:20 mins (10)
User 03
Didn’t Complete: Assistance needed:
Completed: No assistance:
Completed: No assistance: 1: 2:01 mins 2: 1:30 mins 3: 1:45 mins 4: 6:23 mins Completed: No assistance: (2) 1: 0:13 mins 2: 1:38 mins 3: 1:04 mins 4: 4:02 mins Completed: No assistance:
1: 5:35 mins (1) 2: 3:03 mins (0)
1: 5:29 mins (10)
Didn’t Complete: Assistance needed: 1: 1:02 min (2) 2: 2:41 mins (7)
Didn’t Complete: Assistance needed: 1: 2:10 mins (9) 2: 5:32 mins (20)
User 04
2: 3:03 mins (8)
Team Envolve • Engage U - M Clinical Research
1: 1:07 mins (3) 2: 2:27 mins 3: 1:18 mins 4: 6:18 mins
Completed: No assistance: 1: 0:46 min (3) 2: 2:07 mins 3: 2:00 mins 4: 9:15 mins
34