HEREWORCS Phase 1 Soft Launch Report

PATIENT PORTAL PHASE ONE USER TESTING

SOFT LAUNCH

REPORT

1

CONTENTS

INTRODUCTION Report outline PHASE ONE RECAP Prior feedback Iteration to date SOFT LAUNCH TESTS Methodology Participants Diary insights PHASE TWO Diary summary Survey insights Recommendations

2

3

INTRODUCTION

REPORT OUTLINE This report has been compiled in order to update the Worcestershire and Herefordshire NHS Integrated Care System (ICS), in relation to the ongoing user centred research and design process employed by Mindwave for the regional patient portal.

The key intent of the report is to update on the process and progress in design and iteration to date, and to outline the insights derived from the latest batch of “soft launch” user testing, consisting of both quantitative and qualitative outcomes.

It is recommended that these latest soft launch findings are translated into user stories and combined with the priority one items on the roadmap for phase two of portal development.

4

INTRODUCTION

“I think this platform will help carers organise, strategise and deliver good quality, informed care for patients” ICS clinical staff participant

5

PRIOR FEEDBACK In the journey to realisation of the phase 1 platform solution deployed, Mindwave worked closely with the Worcestershire Health and Care trust, Older Adult Mental Health Service (OAMHS) to produce a specialist area, base platform for the most digitally excluded groups, prior to the merger with the regional ICS portal.

Whilst working with the OAMHS service, an extensive discovery phase to establish the base needs of the user groups was carried out. This assisted us to deploy a minimum viable product (MVP) platform for the OAMHS service on a user acceptance testing platform (UAT).

In the journey to this point, we undertook a range of alpha workshops with OAMHS participants using the platform screens in prototype (non functioning) visual format to refine and develop ready for beta deployment.

When the OAMHS MVP was live, we were then able to carry out beta user testing through live interviews with participants in the OAMHS service in order to evaluate usability.

In the approach to the OAMHS / Integrated Care Systems (ICS) regional platform merger, the ICS also carried out a small number of user engagement workshops and interviews.

The process for each stage of user engagement and evaluation to date, prior to the current soft launch phase is explained in the pages that follow.

6

PHASE ONE RECAP OAMHS DICOVERY PHASE January 2020 - March 2020 During this initial period of user engagement for the Health and Care trust, Mindwave facilitated 11 qualitative workshops and consultation meetings with clinicians, admin staff, patients, carers and community members in order to fully understand their current experiences of the Older Adult Mental Health service, and consider how a digital platform might best support the needs of the least digitally engaged/enabled groups.

This was supported by 2 questionnaires with carers, all findings were corroborated with 10 online sources relating to the local Health and Care landscape and its digital vision, to help us prioritise user stories/needs and suggest possible digital features to satisfy related acceptance criteria.

A detailed report of user needs was compiled, providing a high level overview of user insights and contextual priorities to inform the scope of the project and define prioritisation for the alpha prototyping phase, first sprints and subsequent user testing.

7

OAMHS ALPHA BETA USER TESTING PHASE August 2021 Following the deployment of the OAMHS MVP to UAT, Mindwave carried out some user testing with participant users of various roles in the OAMHS service in the Health and Care trust. The purpose of this period of testing was to observe how users interacted with the site when given various tasks to complete in an interactive format..

We provided users with the UAT URL and asked them to screenshare their activity through individual 1:1 remote interviews. We were then able to screen record their responses and draw conclusions related to platform usability accordingly.

The result was a alpha > beta report which documented outstanding actions from alpha phase prototype and beta deployment testing. Aseries of “quick wins” and “longer term solutions” were compiled and documented as potential solutions, in order to enhance usability.

8

PHASE ONE RECAP

ICS ENGAGEMENT SESSIONS July 2021 - August 2021 Prior to the OAMHS merger into the regional portal, the ICS were carrying out their own discovery sessions in conjunction with the OAMHS alpha>beta user testing for a base platform being developed simultaneously.

Three workshops were carried out with service users and five 1:1 interviews were facilitated with clinical staff. The purpose of these sessions was to discover the digital health priorities and concerns of patients and staff in the ICS and capture the insights and unique perspectives of this wider section of the region.

A short summary of the outcomes were delivered in summer 2021, in a google slidedeck to update the stakeholders on user needs.

9

ITERATION TO DATE The three prior series of user discovery, testing and engagement sessions, provided a wealth of insight and suggestions for improving future iterations of the platform and scoping later phases. The key areas identified in each series and the subsequent outcomes implemented to date are described next:

10

PHASE ONE RECAP OAMHS DISCOVERY PHASE The key patient, clinical and non clinical requirements for the OAMHS service were as follows: • Strong dementia focus required due to high local prevalence • Make the site accessible to users not currently in service to reduce admissions • • • • • • • • • • • •

Focus on BAME groups / learning / language and technology barriers Include access to advance care planning, named GP’s and specialist advice and support Provide more ongoing carer support and sense of community through accessible local activities Have a dedicated clinical, human point of contact for building a long-term relationship Provide reliable and trusted information as a single truth Give support with technology use and assist wider social needs Must be interoperable with existing system use especially Carenotes Must be simple to use We must avoid duplication of workload and save time Must provide full patient information Must focus on consent and capacity/governance constraints Must work offline with trust devices

11

This resulted in producing the following features for OAMHS phase 1 sprints: • OAMHS relevant “skin” or aesthetic (based on discovery recommendations) • Sign up / login with consent / capacity flow • Access to care plans • Patient to Care team messaging • Advance care planning / respect docs • Bereavement support • Patient / carer ability to customise tools and resources based upon choice or identifiers such as geolocation / diagnosis / specified needs • Overview of patients explicit communication needs (basic and enhanced view) • Tracker overview including goal setting • Fluid and medication trackers • Lifestyle trackers (diet / exercise etc) • Notifications dashboards • Patient Health history • Google page translate • Admin Dashboard • Clinician Dashboard • Information redaction with best interest decision making • Clinician to clinician messaging • Clinician ability to review patient recorded aspects and leave comments or input from Evie / Carenotes / Admin portal • Video conferencing (including break out rooms and waiting rooms)

12

PHASE ONE RECAP

OAMHS ALPHA>BETA PHASE The OAMHS quick wins that were suggested by both clinical and non clinical users, and subsequently implemented during the merger with the regional ICS platform were as follows:

Welcome / dashboard and navigation •

Better descriptions of the purpose of each page from a clinical perspective and functionality tooltips for new features or areas of detail

•

Ensure that the dashboard and home page are the same thing with the most relevant features in the most prominent areas and differentiate better between the caseload and patient overview The dashboard should say “Welcome <name of clinician>” so you know that the portal is providing you with what’s relevant to you.

There should also be a back button for ease of navigation Remove “get started” button for clinicians as they do not require customised onboarding Combine the dashboard and home page and move the home icon to the top bar for clarity Make it possible to hide the welcome video / initial information.

Clarify the purpose of the home page / dashboard for patients through the descriptions Create and implement a “back arrow” Sign up and login should be next to each other Logout needs to be clearer Include the word “menu” under the bento Clarify how to “get started” with onboarding on the home page. This should explain what the two options are, apply to become a carer or customise your profile by connecting your NHS number Add a patients carer (and contact information) to patient dashboard

• • • • • • • • • • •

•

13

Resources • • • • • •

•

•

•

Create a faster way to get to resources from the dashboard.

Make it clear that all the resources are patient focussed and sharable, include an explanation as to how to achieve this Include a pop-up confirmation which resource has been saved and where to find it The save icon should be changed from a heart to a floppy disc or bookmark – it could even just be a “save” button The saved resources section on the clinician dashboard may need more explanation Saved resources needs to be made clearer and the icon for saving should be changed to something more recognisable for older adults such as a bookmark or floppy disc When the item is saved, there should be a clearer way to demonstrate this visually, the filled heart is unrecognisable to most older adults as it currently stands Sending resources needs to be clarified so that people are clear that the resource can be sent and that the method is via messaging in the circle, consider using an envelope icon rather than the paper plane Include a resource saved / sent pop up

Best interest decisions •

• •

•

14

Change the description to make it clear for clinicians that the best interest decisions here relate to proxy access to data viewing and edit rights, specifically through the portal The process for BID making in relation to portal use needs explaining There needs to be a review and some adjustments made to the “traffic light” colour coding system of BID status so that its clear what each status means and what actions need to be taken There also needs to be additional information such as the carers name and the context of the decision to be made alongside the patients name and any clinicians involved in their care

Appointments •

Improve the description in the appointments section to explain what is being shown currently

Advance care and respect • •

Provide links to “the life book” by Age UK Explain what the respect doc is and how to complete it

Caseload / appointments •

Create a fast user journey from Clinician dashboard to caseload and patient summary

•

In clinical caseload, it needs clarifying that we are looking at a list of clinicians in the first instance and then we click on each clinician to view their caseload of patients Clinicians don’t want an appointment summary, just highlight the upcoming appointment per patient on the relevant patient overview page and perhaps include a link to the clinical appointment calendar in Carenotes for clinicians

•

Circle and messaging • • • • •

•

Clarify the purpose and functionality of circle in description.

Add a pop up message for when a message has been sent Add the words “send message” in a button or similar instead of just using the paper plane icon Change section language, consider an alternative to “Circle”.

Create a pop up screen for messaging to warn patients and carers that the messages aren’t regularly checked and this should include a place to go for urgent help My contacts in the circle needs clarification through description and an indication of how to differentiate contacts from those of the carer role and the patient role

15

Patient info/care • •

Content of care section needs clarification in description.

The flow needs to be clearer when accessing a patients

•

overview from the external browser login, access to caseload should be obvious from the dashboard and the description in caseload should specify that patient summaries are accessible by clicking on them in the caseload list.

Clarify that the number beneath the patient profile picture is their age Ensure that certain field in the “about me” section are mandatory so that the carer has at least the basic information related to the patient. This should also include the GP surgery

•

Look and feel •

16

Double check on trust logos and branding to be used going forward and ensure that the ident at the top is the same as the one at the bottom of the page

17

ICS ENGAGEMENT SESSIONS In the ICS engagement sessions, 6 important emerging themes were highlighted for special attention in portal development as below. The upgrades we have implemented thus far as a direct response to user need are further detailed below each header.

1.

2.

3.

4.

5.

6.

Transparency and access to ‘the truth’ Security and consent Communication Accessibility Personalisation and management Clinical aims and concerns

Transparency and access to ‘the truth’ • •

•

Mindwave is working hard with 3rd parties to provide people with access to the shared care record.

The platform empowers by providing a single shared area of visibility over a patients health for patients, clinicians and carers to access Clinician approved resources are provided through the portal and made available to all users

Security and consent • •

•

18

Patients have complete control over what data dan be accessed and by whom.

If a carer is granted full proxy access (patient capacity dependent and subject to clinical approval) they can make decisions about data and security on behalf of the patient.

Medical professionals can see health and care data that is clinically produced in the trust environment but they cannot access the patient generated data in the portal without advance permission from the patient or nominated proxy

Communication •

•

Patients can use the portal to contact mutually approved circle members about their health and wellbeing and send attachments, images and resources Clinicians can contact colleagues using a clinical directory

Accessibility • • • • • • •

•

Mindwave have included a text size slider functionality to increase the size of the font on all pages.

A range of UI colour palettes with various contract intents are made available to assist with issues affecting vision and recognition.

The design team have adhered to WCAG AA standards for contrast, layout font size and weight.

The platform uses a typeface (poppins) well known for readability The platform has a global language translation facility.

The platform is built on a responsive frameworks so that the layout automatically adapts to various devices and browsers.

The dev team are working on implementation of WCAG AA standards in the coding (pending testing) this includes the ability for the platform to be compatible with screen readers (including image alt tags) and keyboard tab navigation.

Our video production partners have implemented written and audio description of video content produced for the platform.

19

Personalisation and management • • • • •

20

A users resources can be customisable by geolocation, self recorded diagnosis and service A patient can customise their dashboard content manually.

Users can select the colour palette, font size and language most appropriate for them.

A user can manually populate their “about me” section A user can select which trackers they wish to engage with and hide the remainder

Clinical aims and concerns • • •

• • • • •

• • • •

•

A diary facility has been made available for patients to capture their day to day needs and share with clinical teams Messaging looks clearly different to live chat or chat bots so people don’t think they are talking to someone live Strong starts on signposting to bereavement services, advance care planning, end of life, respect documentation provision and social prescribing have been made, though these still require further development Explanatory video animations are in production currently and will assist with explanations of use to different groups “What’s important to me” and “patients explicit communication needs” are already present in the portal Care plan provision is in the pipeline for deployment Patient medications and allergies are available to view in the portal Patient account closure and/or data management is in development for deployment soon, related to when a patient dies or becomes incapacitated.

Clinicians have access to patient trackers and an overview of related data generated Clinicians have an in context view of the patient summary in an i.frame accessible directly from Carenotes Covid-19 resources have been made available in the portal Patient ability to create their own health history by archiving outdated versions of their own health is supported by the portal currently Clinical admins have access to a CMS platform to upload and manage available resources

21

METHODOLOGY

After building the My Health and Wellbeing portal, the next stage was to test it with users to provide insight into how the portal can be developed further.

The users were placed into 5 groups which specified their role on the portal so that the questions they were asked were relevant to their role as follows: Group Group Group Group Group

1: Patients connected to carenotes & proxy 2: Patients not connected to carenotes & proxy 3: Carers connected to carenotes & proxy 4: Clinicians 5: Admin

With each group we delivered a baseline survey, two sets of tasks to carry out using the portal (to be reported back as diary studies) and then a final survey as a comparison to guage difference in user perception of healthcare management.

22

SOFT LAUNCH TESTS

Survey 1 •

A typeform survey link was sent to users to complete to provide insights into their current thoughts regarding the management of their healthcare data (pre use of the portal)

Diary Studies 1 •

• • •

Next, users were then sent their first set of diary study tasks, which the users wouldcomplete within the portal, logging their user experience in a diary format An example task was to sign up We encouraged them to record their responses in a way that best worked for them e.g. writing notes and taking a picture.

The tasks varied depending on which group the users were in and whether they were connected to the Carenotes software.

Diary Studies 2 • •

Afterwards, the users were sent a second set of diary tasks and subsequent questions.

These tasks were different to the previous set of diary studies e.g. Patients were asked to add someone to their circle and send them a message

Survey 2 •

•

At the final stage, the users were sent their comparison survey asking them about their perception of their healthcare data, following use of the portal.

From the responses we were able to illustrate their perceived value of how the portal assists them to manage their health and wellbeing at this time.

23

PARTICIPANTS

Initially, there were 21 participants invited to participate in this user testing phase.

The invited participants were divided into groups based on the role they would undertake in the portal as follows: Group 1 - Carenotes connected pateints - Alan Group 2 - Other patients - Stuart L, Marcus, Sylvia, Denise, Chris, Simon, David T Group 3 - Carers - Viv, John Group 4 - Clinicians - Phil, David S, Tonia, Donna, James, Stuart J Group 5 - Admins - Jo, Marg, Andrew, Alex, Aaron Unfortunately, many of the participants withdrew, did not make any attempts at the tasks or only partly engaged.

The reasons for the drop out were not always stated, but some were due to personal circumstances, users having a lack of time, or users not receiving the communication as planned.

24

SOFT LAUNCH TESTS

Consequently, our insights were drawn from 2 partly engaged participants and 5 fully engaged participants resulting in a total of 7 users for our data.

This resulted in our sample size being small and this mean that at this stage of development, we have reduced confidence in the generalisation of the data we were able to collect.

The users that were both fully and partly engaged throughout the user testing were: • • •

Group 2 - Sylvia, Denise, Simon, David T Group 4 - David S, Donna, Group 5 - Aaron

Participant Limitations •

•

•

Due many users withdrew at different stages of the process, we have incomplete survey comparisons from users as some participants completed one survey but not the other.

We had no patients that participated from Group 1 which meant that we were unable to test the portal with patients connected to carenotes & proxy.

We had one carer that participated from Group 3, but they were only able to provide insights on the sign up process for themselves and their partner and did not complete the second set of diary studies tasks.

Future Recommendations •

•

For better results in the future, testing with more participants that are able to complete the user testing from start to finish could improve our insights and overall feedback.

We recommend a professional participant recruitment company to be subcontracted to assist in future testing.

25

DIARY INSIGHTS Diary Studies Overview The qualitative research tasks set were framed in the form of diary studies.

Diary studies consisted of participants engaging with a series of task to complete through use of the portal. They then were asked to answer questions in relation to those tasks.

Only one “Carenotes and proxy connected” patient was invited to participate, he unfortunately did not fully engage, as such, we were unable to test these aspects of the portal from a patient perspective.

• •

26

SOFT LAUNCH TESTS

Patient diary tasks From the patient group, the participants were asked to complete the following tasks: • • • • • •

Sign Up and create an account Customise your dashboard Send or accept a circle request Deny someone access to view your documents Input that you have a health condition Send a message to someone

User Feedback •

After gathering the information that the users submitted, we gathered the strengths and weaknesses of their experience of the portal for each of the tasks.

Sign Up and Create and account •

•

•

•

Generally, the patients were able to sign up to the portal with little or no help. User 2 mentioned that “signing up was fine.” However, they also added “are older people without knowledge of apps.

User 3 had a hyphen in his surname and when he attempted to sign up, the system would not approve of his attempt until he removed the hyphen from his surname.

User 4 was using a screen reader due to her visual impairment and had trouble signing up. There were some problems with the labelling of items on the site and her software, “Completely impossible to sign in using Edbrowse.” Although she identified some gaps in the portal, we are looking to improve this for future use.

User 4 further mentioned that “when I go to LogIn and get to the Box where I input my email It automatically goes to these little drop-down boxes, which reads “this is the text size slider...” I can navigate down to the email box and password box but then am unable to put anything in them. “Using Firefox I can’t login, enabling JavaScript I was able to login.”

27

Customise your dashboard The ability to customise the dashboard as a user, appeared to be a likeable feature. The various opportunities for customisation provided a unique experience for the users.

Nevertheless, some users raised some difficulties that they experienced. The language drop down menu allows the users to change the language on the site, but most users mentioned that they were unable to identify the language the site was currently using.

•

• • •

•

• •

28

User 1 - “If you start in English and then change it again, “english” has disappeared from the list. I was able to get english back by selecting dutch.”

“I found the layout confusing.”

“The text boxes describing the sliders etc appear, then disappear and then appear again” User 2 - “I think I was expecting more to work from the Dashboard, and my eye didn’t naturally look to the top right of screen for the menu.”

User 3 - He had different experiences across two devices.

He also wanted to use certain functionalities that the portal currently does not have.

“On the Mac with a mouse the text slider works fine. On the iPad and iPhone it appears to be disabled.”

“First time round I pulled all across but then decided they weren’t in the right order. I couldn’t grab and place. So had to remove all and start again.”

• •

SOFT LAUNCH TESTS

•

•

•

User 4 - She seemed to be able to do this task with some ease, but had some problems with certain features for customising. “I’ve tried out the feature to change the colour backgrounds and that seems to work.”

The cross in the top right hand corner would not read out properly.

When I finally got rid of it I couldn’t move onto the next stage because I kept getting annoying little messages that I didn’t want”

Send or accept a circle request •

• •

User 4 was the only user to leave feedback on this section. She was able to complete the task but was unaware about whether it worked or not.

“The recipient we were trying to add using Firefox did get a message the following day” “But I wouldn’t class this as a success because we had no idea they had been Invited.”

Deny someone access to view your documents • •

•

One user found this task difficult whilst the other found it useful.

User 2 found this task “very difficult and time wasting to find where the section is to deny someone access to see my documents. I’m not sure I have achieved this!”

User 4 felt that “it would be useful” to have as a feature.

29

Input that you have a health condition • • •

•

User 3 - He was able to complete this task but found the layout quite confusing.

“I added to my existing diagnosis” “I think the layout needs to be reconsidered. Each diagnosis may be more than one word and some words will be long.

I don’t think it would be uncommon for a patient to have multiple ailments especially over time.”

User 4 - found this task “impossible” so seems not to have been able to input any information.

• •

30

SOFT LAUNCH TESTS For these sections, User 1 made no submissions leaving Users 2, 3 and 4 to provide their feedback.

Add entries to your pain and sleep trackers •

•

• • • • • •

User 2 found this quite difficult and mentioned “the amount of days I entered on the sleep tracker didn’t always register, despite me adding 30th March, it never appeared on the graph.”

She also highlighted that “the line showing tracker dashboard at the top of the page, and then sleep etc is lost in sight because it’s not distinguished from the other blue colours at the top of the screen.

If it was a yellowish colour, like the boxes which say, add edit etc. it would work so much better. Should the text be bolder?”

User 3 found it difficult to find the trackers, so we gave him so guidance on how to do so.

“This page may seem obvious but I didn’t read ‘Inactive Trackers’ as add tracker.

User 4 “could find a sleep tracker” but had some challenges once she found the trackers.

“It seems impossible to add anything to the graph” This user also experienced difficulties due to the lack of labelling as she used the NVDA screen reader.

She also encountered the same problem as User 3 where the active and inactive trackers did not seem straight forward on separate pages. “It would be so much easier to have all the items in one list”

31

Compare both trackers on the graph • •

User 2 seemed confused by this task and asked “How do I know I need to toggle the trackers to compare them?”

User 4 was unable to complete this task using NVDA and mentioned “who is going to have the time or energy to put in all this information especially if they are tired and or suffer from chronic pain?”

Set up a goal of drinking 8 glasses of water from 10am - 5pm •

•

•

32

User 2 found this task quite difficult as “when completing the fluid intake on my computer, the page kept moving up and down on its own which meant I couldn’t add info easily. In fact it did for all tasks which is unnecessary and frustrating!”

The page moving up and down did not seem to happen to other users, but perhaps we could investigate this problem she experienced further.

“If you have drunk various small and large bottles, that info doesn’t seem to add in the records. It’s almost that one has to drink the same size containers for it to work.”

• •

•

SOFT LAUNCH TESTS

• • •

User 4 was able to select fluid However, she mentioned that “could not add anything to the graph” There were some more problems with labelling, which we will need to revisit for improvements.

She also made the suggestion that “It would be better to write 3 litres rather than 3 L. The L can sometimes look like a one.”

View the resources section • • • •

This task seemed straight forward and easy for users to achieve.

User 2 found that they could “view the resources section successfully”.

User 4 “was able to get into resources page in Apple IOS.” She did have difficulty when selecting the “Read more button of the first two items” as she was presented with inaccessible PDFs”.

She recommended HTML’s or other options in case a screenreader cannot access the version available.

Save a useful resource and send a resource to a circle contact • •

• • • •

•

This section seemed generally simple to complete.

User 2 - “The idea of saving resources by clicking the ‘Star’, isn’t a good one. The instruction to do so is in text away from it. Why not just have ‘Save’ under the star?

User 3 - “Very useful to be able to share” “I’m not too sure if you can add in a resource of your own and share it” “When sending a resource it would be useful to be able to send a message at the same time.”

User 4 - “As there was no feedback in IOS I have no idea whether I was successful in sending a resource video to someone I tried to add in my circle.”

She was unable to use NVDA to ensure she had completed the task successfully.

33

Carer Diary Tasks •

• • • •

•

There was only one user for this part of the portal (user 5).

This user had to withdraw for genuine reasons and so only completed the Sign Up task.

The carer struggled with the sign up process, the link didn’t work for her the first time.

She then tried to access her husbands notes by connecting as a carer but the process of how to do this was not clear to her.

When she eventually discovered how to do this (with video call support) the NHS number did not connect to her husbands notes because an account hadn’t been created by him and a dormant version was not created by the trust for him so there wasn’t anyone for the carer to connect to.

This experience and lack of explanation around the process meant that the carer got frustrated and “gave up”

• •

34

35

SOFT LAUNCH TESTS

Admin Diary Tasks There was only one user for this section (User 6).

He did not leave any comments for signing up but seemed to find the portal fairly simple and useful.

Patient Overview He provided a mostly positive experience of the patient overview feature for the one admin participant.

He found it generally “intuitive to use” with the format of the patient summary page being “easy to navigate”.

He commented on the helpfulness of the patient filtering tool and was very positive about the use of first person language when presenting the patient details.

One improvement was suggested to patient summary by the admin, to be able to reverse the order of patient sorting when using the filtering button

Team caseload The user found the caseload feature “easy to navigate.”

He did request for the ability for filtering of patients by clinician however.

36

• •

SOFT LAUNCH TESTS Questionnaire The user found the questionnaire feature easy to use and commented on its flexibility to suit differing clinical workflow preferences <quote>.

“It allows the user to have quite a lot of flexibility in how they record and gather the data from the patients.”

He found the share facility via the circle perfect for simple distribution as it “allows access to those in your circle, also makes distribution of the questionnaire nice and simple.”

37

Explore all the platform features •

Overall, the admin seemed to be really pleased with the intuitive navigation and the visual accessibility

•

Some further comments were made by email relating to typos “driving and dimentia” and checking that video links ] are correct.

The admin also expressed some confusion with the prioritisation of certain conditions in the MVP such as prostate cancer and suggested that some features needed to be more service specific. For example, caseloads should have the ability to sort by service e.g. OAMHS and resources should be tagged for OAMHS users for fast finding.

Final comments related to being unable to access the end of life resources and app, when the icons are clicked nothing happens.

User 6 “had a bit of trouble completing all of the tasks for this week as I appear to not have any allocated patients in my clinical caseload.”

•

• • •

• •

38

SOFT LAUNCH TESTS Save a Resource • •

The admin was impressed with the platform resource saving feature He did suggest however that the saved resources section might be evolved to include the ability to create folders to organise the saved resources.

“I wonder if it could be considered to have folders in the favourited resources.”

39

Clinician diary tasks Signing Up •

The clinician initially struggled with accessing the platform, she wasn’t clear what the URL was or where to find it.

•

She tried to run the name of the platform through a search engine retrieving obscure results and was unable to access at work using her desktop browser, as she was attempting it using Internet explorer which the platform doesn’t support.

•

She also had trouble registering and creating an acceptable password. A clear need for education and training has been identified for staff as an outcome.

•

When using her phone instead, she found it difficult as the vertical scrolling was excessive to access the appropriate information. It might be fruitful to consider alternatives such as expandable tabs of digestible information and / or horizontal scrolling

•

“Found the whole process of logging in difficult, had to resort to I phone in the end”

40

• •

SOFT LAUNCH TESTS

Navigate around the platform • •

• • •

The fixed menu bar on mobile was found to be a useful feature to avoid unnecessary scrolling.

The clinician thought that the top of the page descriptions were helpful for context and she was very positive about the “text size slider, language options and colour themes.”

On mobile, the clinician found the guide tool to be a distraction on the right side of the screen She suggested being able to move it to the top or bottom of the screen so that it is out of the way.

She also emphasised the importance of a “home” button to get back to the dashboard, she didn’t realise that the logo at the top left took you there and was pressing the back button repeatedly instead to access the dashboard

41

Look around the Clinical Caseload •

The clinician found the caseload page easy to use but was perplexed that it was not populated with their patients, this also meant that they were unable to view a patients health data vis the caseload function.

View a patient’s health information •

She also expected all of her patients to be auto-enrolled into her circle so that she could message and interact with them immediately, the same is true of her colleagues.

Send a message to a patient •

She did figure out that the clinician could be messaged directly from the directory but also wanted to see the message and the clinical contacts in her circle.

Send a message to a colleague •

The user generally felt satisfied with the process of adding and messaging people.

42

• •

43

SOFT LAUNCH TESTS

Compare your patient’s trackers under one graph •

User 7 found it easy to access a patients trackers. The suggested that separating the headings on the menu would be helpful to differentiate and that we may wish to complement these pictorally.

•

In the tracker section, the clinician felt that “blood” as a title for the blood pressure tracker was misleading.

•

The clinician participant felt that in the tracker section, their related goal should be at the top of the page to clearly indicate what should be tracked and why.

•

The clinician noticed an issue with the user journey. As she was using her “back button” to navigate repeatedly, she was finding that it was taking her to circle and then to the patient as opposed to the previous page.

View your patients summary •

The patient summary was easy for the clinician to follow once they were able to find it.

44

• •

SOFT LAUNCH TESTS Access your patient’s information •

The clinician commented that it was comprehensive and held lots of useful information about the patient, but suggested that the section on communication needs should take primary focus in the clinician view of the patient summary and should be visible also in the circle prior to messaging them to ensure that the communication is effective when attempted.

•

“I do feel that this should either be at the top or in patient summary to see before any details as this is very important to know,”

View your patient’s diary •

The clinician felt that the diary function would be very useful for a clinician “to understand thoughts, deterioration in both mental and physical health or indeed stability in both”

•

However, she was concerned that it would not be well utilised by the patient due to data security.

45

DIARY SUMMARY Once the users completed the diary studies, we analysed their feedback and have provided some actionable insights that we should look to create user stories from and implement as new and upgraded features in future phases by agreed prioritisation.

PATIENTS Sign in and create an account •

•

Providing useful and appropriate labels will help to make the portal more accessible for those who use additional softwares to navigate the site.

Ensuring that users are able to sign up smoothly with a hyphen or some uncommon characters within their name, or providing a pop up that indicates which characters the system cannot accept at that stage could be solve this user problem.

Customise your dashboard •

•

46

Allowing users to reorder the items on their dashboard by grabbing and dropping them could make the user experience simpler and less tedious.

Two users mentioned that they disliked the help boxes that appear and disappeared.

PHASE TWO

•

An alternative method could be that when users press the “?” help button, the system could provide a pop up summary with pictures of each feature so that users could navigate to the section that they require help on.

Send or accept a circle request •

A user mentioned that they were not notified when their recipient received their circle request. A pop up that the request has been sent could reduce concerns about whether it has been done or not.

Deny someone access to view your documents • •

This user journey appeared to be slightly long for user 2 and they did not seem to know if the function had worked.

We could ensure they are notified and attempt to make the user journey shorter so that users can access that feature smoothly.

Input that you have a health condition •

•

•

User 3 suggested that “the table should have cell lines” to make it easier to understand the content and divide the features so that it is easier to provide the correct information.

User 4 had a lot of difficulty completing this task as there were some unlabelled items that prevented her from being able to complete the task. This is something we will be revisiting in the next phase.

There also seemed to be some inconsistencies in language that we will need to review. “Why was it called my messages in one place and my messaging in another?”

47

Add entries to your pain and sleep trackers •

• •

The user journey appears to be frustrating, both user 3 and 4 recommended that instead of having a separate page for inactive and active trackers, all of the trackers could be visible on the same page.

The toggles could be used to visually demonstrate which trackers were active to simplify the overview There appears to be a some difficulty regarding saving entries to trackers as two users had the same issue. In the next phase, this will be reviewed so that it can operate smoothly for users.

Compare both trackers on the graph • •

48

As most users how to compare both trackers, we may need to provide a help function on this section.

User 4 was unable to complete this task due to NVDA being unable to find the data. We will need to review how screen readers interpret the data on the portal to increase ease and accessibility.

PHASE TWO Set up a goal of drinking 8 glasses of water from 10am - 5pm •

•

User 3 suggested that there should be a goal that users work towards so that these comparisons have a clear objective. This would be a useful addition to the feature to reinforce users to achieve their goals.

User 4 also suggested that we change the units in this section so that 3L is 3 Litres to improve clarity as she mentioned that the L could look like a number 1.

View the resources section •

Ensuring that the resources are in a HTML format as opposed to PDF’s will make it more accessible for screen reading.

Save a useful resource and send a resource to a circle contact • •

User 3 suggested that if users could send a message at the same time as sending a resource, this could be useful.

User 4 was unsure whether she completed the task successfully, so perhaps a pop up with labels that explain that the resource has been saved or sent would provide clarity on the actions taken.

49

CARERS •

• •

50

Based on the feedback from one user, we need to implement a simple “sign up” process. Engagement materials need to be created by the trust with step by step “easy to follow” instructions of the key features and benefits of use.

The method for a carer to connect to a patient needs to be more prominent and better explained Dormant accounts need to be autogenerated for all patients in a service so that a carer can connect to them by NHS number regardless of capacity status

51

PHASE TWO

ADMINS The administrative staff member did not give feedback on the sign up process.

Patient Overview •

To make the overview easier to navigate, a filter reversal button could help users to find the patient they are looking for.

Team caseload •

Adding a “sort by clinician” filter to team caseload could simplify finding staff members.

All features • The prioritisation of conditions in each MVP should be made •

•

52

clear in the onboarding Providing options to filter patients and caseloads by service is something we could do to reduce the user journey when users trying to find patients by service, for example, if an OAMHS clinician arrives on resources, they should be able to see colorectal cancer for example We will need to review typos and validity of links

PHASE TWO Access your patient’s information •

The user was unable to complete many of the week 2 tasks as he did not have any allocated patients connected to his caseload

•

Therefore the first five tasks relating to viewing patient generated information was difficult for him to access Clinical caseloads need to be populated with patients so that testing is more accurate

•

Saving a resource •

•

The admin was impressed with the platform resource saving feature but suggested that the saved resources section might be evolved to include the ability to create folders to organise the saved resources Creating folders or a way to organise information in the saved resources page could make access to resources more efficient

53

CLINICIAN Signing Up • •

Staff will require basic training on how to sign up We could also consider alternative horizontal or collapsible layouts for pages with large amounts of information requiring lengthy scrolling on mobile

Navigate around the platform • • •

We could change the help guide from the left of the screen and provide the option for it to be “moved” to a position of choice Including a “home” button near to the logo at the top left of the screen could be a more appropriate position for users.

Look around the Clinical Caseload • •

•

54

The clinicians currently need to add patients to their circle to have them populated in their caseload.

The original design intention was that the patients be autoenrolled to the relevant clinical caseloads by connecting via Carenotes at the back end.

Recommend revisiting original intent and auto-populating caseload with relevant patients, as caseloads change in the clinical systems the portal would instantly update the caseload.

PHASE TWO •

•

This would mean patients in all services should be auto enrolled with inactive accounts and sit idle in the clinical caseload. The clinician could then request to add the patient to their circle from there.

Clinicians should also have the ability to add anyone in the directory to their circle for fast messaging, if they message form the directory the clinician should then be auto added to the clinical circle and the message sent should be added to history.

Add a patient and carer to your circle •

• •

Some considerations we could make could be to add “colleague” and “carer” to the dropdown menu for adding someone to the circle Change the radio buttons in data sharing selection to on/off switches or change the wording Recheck links in email invite confirmations work

55

Compare your patient’s trackers under one graph •

Make the menu headers more distinct

View your patients summary •

Communication needs of patients should be at the top of patient summary

View your patient’s diary •

The security of the diary should be made more explicit in the description

View your patients coping strategy •

Explain better what coping strategies are in the description and/or consider changing the title

Send a resource to another clinician •

56

Consider adding the resources section to the clinician dashboard

PHASE TWO

•

57

SURVEY INSIGHTS

For the baeline submission, we asked users to complete a survey in Typeform that examined their experience of how their healthcare data was managed generally. The survey questions varied based on the group that the users were in to capture the most relevant information, the data collected was quantitative and allowed us to guage satisfaction with “pre-portal” provision..

PATIENTS •

•

•

58

From the patient portal, there were 3 users that attempted the first survey questions, however, for the second survey series, 1 of the users did not make an attempt. Therefore, we drew conclusions from the two users.

We drew comparisons from each question by each user to determine how using the portal in between the surveys could increase their scores and have a positive impact on the way that their healthcare information is managed.

The first 2 questions are not included here as they included personal details about the user allowing us to identify them for demographic purposes.

PHASE TWO The following highlights the questions from survey 1 and 2 and the score outcome percentage, demonstrating whether the user’s score increased, decreased or stayed the same.

Q3. Security of the user’s data • •

•

•

Both user’s scores stayed the same with user 1’s score being 5/7 and user 3’s score being 7/7.

User 1 scored slightly lower suggesting that the they had a decent level of faith in the security of the system, but the portal was unable to increase their feeling of security at this stage.

This suggests that using the portal may not have provided an opportunity to increase the user’s perception of the security of their data.

User 3 did score highly on this question so this indicates that they already felt that their data was secure.

Q4. Clarity of healthcare information received • • •

•

User 1 scored nothing in survey 1 but went onto score 5/7 in survey 2.

User 3 initially scored 5/7 for survey 1 and the 3/7 for survey 2.

Although user 1 scored nothing in survey 1, as he scored quite highly on the clarity of his data, it may demonstrate that the portal has instilled some confidence in him about the clarity of his healthcare data.

Nevertheless, as user 3 was the only one to offer scores for both surveys and his score decreased, it may indicate that he did not understand how the portal provides clarity in how his healthcare information is received.

59

Q5. Personalisation of information • •

• •

Both users scored lowly on this feature in survey 1. User 1 scored 3/7 and user 3 scored 2/7 Both users evidently felt that their information was not personable and required some adjustments to make this possible.

In the second survey, user 1’s scored stayed the same at 3/7 whilst user 3’s score went up to score 7/7 This may infer that the portal was able to provide a sense of personalisation to user 3

Q6 - Ease for accessing data • • •

60

User 1 scored quite highly in survey 1 with 5/7 but went down to 4/7 User 3 scored 1/7 and went up to 3/7 It appears that although user 3 was not yet confident that he had easier access to the data, he felt a bit more assurance after using the portal. Perhaps if he becomes more familiar with the portal in practice, that may instil more confidence in him about what the portal offers.

PHASE TWO

Q7 - Importance of monitoring healthcare information • •

• •

User 3’s score for this question on survey 1 was 7/7 and remained the same for survey 2 As their score was already at the highest, it is difficult to be able to determine whether the portal provided more insight into this goal or not. Nevertheless, as their score did not decrease, it may suggest that the portal upheld this standard for them.

User 1 did not provide a score in survey 1, but in survey 2, he scored 5/7.

This may suggest that the portal may have emphasised the importance of his healthcare information being monitored.

Overall Score percentages: • •

We drew conclusions about the influence of the portal on the user by gathering how much their survey scores increased.

We summarised that the higher the increase in their scores compared to the first survey, the higher the positive influence that the portal had on their perception of the management of their healthcare data.

User 1’s satisfaction scores: • • •

Increased by 44% Decreased by 33% Stayed the same by 22%

User 3’s scores: • • •

Increased by 63% Decreased by 13% Stayed the same by 25%

This suggests that for these users, the portal is having a positive overall impact on their perception of healthcare management

61

STAFF Q3 - Security of patient’s data when using the portal • • •

User 6’s score went down from 5/7 to 4/7 User 7’s score went down from 7/7 to 5/7 As both user’s scores went down, this may suggest that we need to revisit the security of the patient data on the portal

Q4 - Clarity of healthcare information patient receive • • •

User 6’s score went up from 4/7 to 7/7 User 7’s score went up from 5/7 to 6/7 As both user’s scores increased, this may suggest that the portal helped to provide clarity on the healthcare information the patient receives

Q5 - Personalisation of information for patient and carer • •

Both users’s scores went up from 4/7 to 7/7 This may suggest that both staff members could agree that the portal offered a great sense of personalisation of information for the patient and carer

Q6 - Ease of accessing data for patient and carer • • •

User 6’s score went up from 5/7 to 7/7 However, user 7’s score went down from 5/7 to 3/7 Since user 7 is a clinician, this may suggest that they did not feel that they were able to access data from their patient and carer easily

Q7 - Importance of monitoring healthcare information for patient and carers • • •

62

User 6’s score went up from 3/7 to 5/7 User 7’s score went up from 5/7 to 7/7 This suggests that perhaps the portal helped to highlight

PHASE TWO

Q8 - Amount of healthcare info provided to patient and carer • • •

The scores for both user’s stayed the same User 6 stayed at 4/7 whilst user 7 stayed at 5/7 This could infer that the portal did not affect their perception of the amount of healthcare information provided to the patient or carer

Q9 - Quality of healthcare info provided to patient and carer • •

User 6’s score went up from 4/7 to 6/7, suggesting that the portal may have highlighted the quality of healthcare information provided to the patient and carer User 7’s score stayed the same at 5/7 suggesting that the portal did not affect their perception of the quality of healthcare information provided to the patient or carer

Q10 - Patient and carer’s involvement in healthcare planning • • •

User 6’s score went up from 3/7 to 6/7 As this is quite a big gap, it may suggest that the portal aids the patient and carer’s involvement in healthcare planning User 7’s score went up from 5/7 to 6/7 also suggesting that the portal highlights the patient and carer’s involvement in healthcare planning

Q11 - Duplication in daily workflow • • •

For this question the desirable outcome is that duplication decreases, so a score decrease is what we were hoping to achieve User 6’s score went up from 5/7 to 6/7 suggesting that the portal may still not help with duplication in the user’s daily flow User 7’s score however went down from 6/7 to 5/7 suggesting that the portal may help to slightly reduce duplication

After using the portal, user 6’s satisfaction scoring: • • •

Increased by 67% Decreased by: 22% Stayed the same by: 11%

After using the portal, user 7’s satisfaction scoring: • Increased by: 56% • Decreased by: 22% • Stayed the same by: 22% • This suggests that for these users, the portal is having a positive overall impact on their perception of healthcare management

63

CONCLUSIONS FROM SURVEYS Patient and Staff comparisons • • •

•

• • •

64

From the patient surveys, one patient’s scores increased by 63% whilst the other patient’s scores did not increase at all The staff’s scores went up by 67% for user 6 and 56% by user 7.

Both staff’s scores mostly increased after using the portal, whereas for the patient’s, only user 3’s score mostly increased.

User 1’s scores did not increase but rather mostly stayed the same suggesting that the portal did not seem to increase their confidence about their healthcare data.

Nevertheless, user 1 did not answer some of the questions which meant that only 5/8 of the questions were answered.

This also meant that only 5 comparisons could be made out of 8.

In summary, this information tells us that the staff seem to have had a more positive outlook on the way that the portal aids the user’s experience compared to the patients

65

PHASE TWO

RECOMMENDATIONS

Recommendations related to prior research and testing Any outstanding or unaddressed items from the OAMHS alpha phase testing have been combined with longer term solutions from the OAMHS alpha>beta report and suggested improvements from the ICS engagement slidedeck.

These user needs have been translated into user stories and added to the backlog for My Health and Wellbeing phase 2 scoping alongside the findings in this report form the soft launch user testing.

66

PHASE TWO Recommendations from the soft launch user testing Our key recommendations for drafting phase 2 user stories (and prioritising accordingly) would be as follows:

Patient feature improvements • • • •

Labelling items on the system to increase accessibility Changing the way current format of the help function on the dashboard page Providing simpler headings and user journeys for patients to access items Providing goals for the tracker section and rethinking the layout of active and inactive trackers

Carer feature improvements • •

Ensure the method for a carer to connect to a patient is more prominent and better explained Producing a simpler sign up process

Clinician improvements • • •

Consider alternative horizontal or collapsible layouts for pages with large amounts of information requiring lengthy scrolling on mobile Populating the clinician portal with patients once they have signed up to better test functions in the portal More colour, and pictures to make the experience more engaging

Admin improvements • •

Providing folders in the favourited resources. For example, Favourited dementia and driving resources folder Ensuring language is clear, consistent and free from typos

67

ILSA PARRY UX RESEARCH LEAD e: ilsa@mindwaveventures.com t: 07974186677 LIZZIE OLURONTOLA JUNIOR UX/UI DESIGNER e: elizabeth @mindwaveventures.com t: 07944602831

mindwaveventures.com