Learnings from the Virtual MultiDisciplinary Team (vMDT) Project Phase 2 (2016-2019)
December 2019 1
Acknowledgements There are many people who brought this project together including: • The Project Team at the Christie: Wendy Makin, Ben Heyworth, Alison Reddicen and Chelsey Gilmore • The Macmillan team including Lesley Smith, Dany Bell and Adrienne Bentley • Members of the Project Board, Clinical Reference Group and Patient Reference Group
• All of the experts that gave their time voluntarily to respond to patient cases The project evaluation was undertaken by Georgina Wiley, Laura Parry and Alison Reddicen with feedback from all of the project working and governance groups
2
Who should use this document • This report intends to provide lessons learnt from the investment Macmillan made in delivering this service. It was created to provide considerations for staff who both develop, fund and provide governance for Macmillan funded programmes and projects • There are six key areas of learning which are transferable for other projects at Macmillan; each of the key learnings has summary detail of what happened (the activities), the impacts (the changes) and the lessons learnt (these are aspects of the project that went well and areas that would be approached differently if the programme was undertaken again) • The document summarises the vMDT service, incorporating information and opinion from the Project Co-ordinator and Project Manager 3
How this document should be used • The overall learning is the importance of contracting, delivering and reporting to intended actions and outcomes – which should be outlined in a clear a service design. At point of inception this service had a Logic Model which outlined the key actions and changes that the service was to deliver. Several outcomes and impacts for patients and healthcare professionals were provided, but with the low numbers of referrals and use of the service it felt more appropriate to report on the actions Macmillan took, what happened and what we have learnt – opposed to reporting back on stated outputs, outcomes and impacts in the Logic Model
• This is not an evaluation report or analysis of how successfully the service delivered on intended outputs, outcomes and impacts. Instead this is a reflection and learning report, which draws on opinion of those involved in delivering and overseeing the vMDT service, with references to data where available 4
Key Areas for Learning 1. Programme Team (Resource) 2. Software Development (Resource) 3. Engagement (Referrers and Experts) 4. Referral 5. Resolution – with expertise and advice
6. Sustainability and changes in the environment
5
Background • The virtual Multi-Disciplinary Team (vMDT) concept was proposed by Professor Alastair Munro as a potential model for managing people with complex Consequences of Treatment. (BJC 2013). The vMDT concept was to produce an online platform that would allow healthcare professionals referring to the system, access to written expert opinion within a proposed period (2-3 weeks) on the management of complex consequences of treatment for which local expertise may be lacking • Macmillan awarded Professor Munro’s team at Dundee University a grant to develop the vMDT concept and test a prototype platform (Phase 1, 2013-14). Prof Munro created the vMDT system on the University of Dundee blog site and accepted referrals from University College Hospital (UCH), haematology-oncology team for people who had complex consequences, because of cancer treatment as a child or young person, with a specific focus on bone marrow transplant patients. Overall, the feedback from referrers and expert panel members was positive and the concept well received. The results from the 12 patients referred showed a reduction in outpatient appointments, saving patients’ time and NHS resources • With the encouraging results from Phase One, Macmillan proposed to proceed with the testing of Phase Two and develop a new version of the vMDT system for survivors of adult cancers. This process kicked off in September 2015 with a Stakeholder Engagement event. Following positive feedback from this event Macmillan began the process of finding an NHS partner, creating the software, recruiting experts, recruiting a vMDT coordinator, as well as planning for promotion, marketing and evaluation. Macmillan also established strong governance for the project including the project board, reference group for people affected by cancer and the clinical reference group • Phase Two of the vMDT project was in partnership with The Christie NHS Foundation Trust. Macmillan awarded The Christie a 2.5year grant to deliver Phase Two testing. The pilot would be tested and conclude January 2019
6
What was the vMDT? The vMDT allowed referrers (e.g. oncologists, GPs) to access on-line expert advice within 2-3 weeks on the management of complex consequences of treatment (CoT) where local expertise may have been lacking. The vMDT aimed to: • facilitate a co-ordinated approach to supporting the care of people for rare, complex and/or severe CoTs • transform how health professionals (especially cancer care teams) access and use expert advice that ensures that people have all their CoTs recognised and managed, regardless of how long it has been since their cancer was treated • support the long-term integration of CoT care into cancer follow up in primary and secondary care by also raising awareness of less severe CoTs, which will then be better managed/prevented • evaluate if the concept could be extended in future to other areas of cancer care/health& social care for non-urgent conditions • act as an educational resource For phase two of the programme referral criteria for the system was established which included; people who had cancer as a child or young adult, people who had a bone marrow transplant, and people who had any treatment for any pelvic cancer. Referrals were set up to come only from secondary care. Due to low referral numbers the scope of the project was extended to include all tumour types and to accept referrals from Primary Care (April 2017)
7
Aim of the Phase 2 pilot • The aim of the Phase 2 project was to test the vMDT concept with enough people (people affected by cancer and health professionals) to enable a robust evaluation to inform a decision by Macmillan on how to proceed with the vMDT concept following this project. • The service (Phase 2) ran for 3-years solely funded by Macmillan, and collaboratively working with a key partner, The Christie at Manchester where the physical offices with staff was based. The Christie was chosen through an application for tender process. The service was intended to be accessible for all cancer cases across the UK for virtual referrals. • Macmillan estimates that there were 2.5 million people living with cancer across the UK in 2015 with this projected to rise to 4 million by 2030. Around 625,000 people experience long term consequences following their cancer treatment; some of these are complex in nature and may benefit from expert advice from a range of specialities not available at a local level. • In recognition of this being a pilot project the intended target of referrals for the whole project was set at 100. Across the project’s lifetime from launch in November 2016 to end of January 2019 the total number of referrals for this service was 15. The contract between Macmillan and The Christies ran from December 2015 to January 2019, to run a 3-year programme. Due to the low referral numbers the contract was not renewed, and a robust evaluation was unable to be undertaken.
8
1. Programme Team Staffing structure and scope of the roles
Overview • Phase 2 of the programme was confirmed and an expression of interest to host the programme was sent out • The Christie in Manchester were the successful applicant and were appointed clinical leads for the programme implementation • The programme received funding in December 2015 and the programme team were recruited (to predefined roles) in early 2016. The direction for the service to take was decided by the Macmillan Cancer Support Consequences of Treatment programme manager and the Christie • Overarching Programme Governance was implemented (including Programme Board underpinned by clinical reference group, patient reference group and project delivery group)
• It was decided that The Christie would be the physical office for all direct facing roles for the 3year partnership, that programme management and line management responsibilities would also sit at the Christie. Ownership for delivery was Christie but governance ownership was Macmillan (chairing the board)
10
What happened / what change occurred • Programme funded by Macmillan for 3 years • Programme was a partnership between Macmillan Cancer Support and The Christie, with all the funding coming from Macmillan – but the physical space for programme roles and activities were provided by The Christie (decision made for this to occur – initial thoughts were that programme manager could be based at UKO) • Programme team that delivered the service was made up for three roles: • Programme Coordinator (1 FTE, 5 days a week, Band 6) to oversee programme delivery • Programme Officer (1 FTE, 5 days a week, Band 3) to assist programme coordinator with programme delivery • Also overseeing the programme at the Christie were: • Programme lead: programme lead and line manager to programme coordinator and programme officer • Senior Director at Christie: Grant holder for the Phase 2 programme vMDT system testing at The Christie • At Macmillan: • Macmillan SRO/Project Board Chair (changed during programme lifespan) - Responsible for strategic direction of project. Provide recommendations and decisions affecting project progress and delivery to Director/Services and Influencing. • Programme manager (assisting with overall development of programme as well as programme support) 11
Potential impact on programme • During the programme there were several staff changes for the programme team based at the Christie as well as Macmillan staff in the UK office. This potentially impacted on internal and external links (including recruitment) and networking opportunities
Lessons Learnt (what went well and considerations for future iterations) • Have an agreed RACI document/in service agreement/or similar between Partner and Macmillan, to be clear on Macmillan’s (including understanding of who has responsibility for managing the relationship) and all partners role in/link to programme/programme from the outset – e.g. funder, advisory, governance or actively involved in service delivery • Where possible undertake initial scoping for need of the service and level of demand before deciding on programme support needed/signing off on budget – this could be informed by engagement and insight with the intended service users or by running an initial pilot programme . • Be clear on job descriptions and skills needed to successfully undertake the role before commencing recruitment – for new programmes with development of new team employing someone with correct level of expertise (or provided with support to gain expertise) is crucial to success. Being clear on the fundamental skills and experience needs, e.g. evidenced service development • Ensure day to day project support is correctly identified, and consistent throughout the lifetime of the project, with clear lines of accountability when somebody leaves • Does the programme board have a point a certain number of months after the launch of a project that monitoring information (at minimum) will be reviewed to agree next steps between funder and partner – and before releasing next phase of funding • Ensure a clear risks log is written and ownership of this is agreed upon prior to project. The programme manager should be responsible for monitoring the risks (including external and internal potential changes in landscape and recognising staff turnover risks) • While the purpose of piloting is often to revise, redefine and develop roles, functions and activity there is a need to consider the level of funding appropriate for a pilot project in line with the expected outputs This is particularly important as the paediatric pilot study had similar problems with referral numbers even though late effects clinics were in every centre. Adult late effects recognition is well behind.ie Cannot pilot in paediatric setting and expect it to work in adult setting-not transferable
12
2. Software development System NH3 Network’s accessibility and functionality
Overview • Decision made by programme team (including the Christie and Macmillan) on the software required to successfully implement the vMDT programme • Tenders were issued publicly in January 2016. Only one tender application was received, full process was followed including a panel and interviewing of the applicant • Asckey was commissioned in June 2016 to develop a software solution and IT services for the vMDT. It was to be accessible to the programme team as well as referrers to be able to refer into and for experts to respond to cases • Data security was ensured through using the secure NHS3 network. It was not designed for patients to access directly • The vMDT was launched in November 2016 with design updates required in July 2017 to accommodate changes made to the programme to increase scope to include all cancer types and primary care. The management of Asckey was handled by the programme coordinator from the Christie
14
What happened / what change occurred
Potential impact on programme
• Macmillan commissioned and managed a supplier to develop an online platform to be built around the secure N3 network (online secure NHS network) to accommodate the processes required to successfully run the programme (take referrals, allow experts to respond, allow programme coordinator to collate responses and provide case report) • One tender was received from Asckey, which was accepted by Macmillan and for developing, maintaining and hosting the system • Limited changes were able to be made to the system once developed (October 2016) as many of the changes needed were not in the scope of the contract, and as such required additional payments which restricted opportunities to change and grow system as needed as programme grew • The system was only accessible to registered referrers, vMDT Experts and the Programme Team based at The Christies who has compatible NHS emails. Macmillan were not able to access the system • Feedback from referral surveys indicated that the system took too much time to register. There was no option for clinicians to upload patient data and notes to the system meaning that they needed to manually input all information each time they made a referral • The programme coordinator was required to manually allocate cases as there was no functionality to match experts to cases
• The system /online platform built did not meet the requirements for what the vMDT needed to run effectively or efficiently. This subsequently impacted the uptake (i.e. lack of referral) and delivery (i.e. time consuming) of the vMDT service. • No automation was possible – meaning all information received needed to be manually inputted, extracted, and anonymised. This lead to Programme Coordinator having to undertake additional work that could have been automated, (e.g. anonymising referral information). A better designed platform / system and the opportunity to review the programme once launched with software development company could have provided a time efficient system. • Lack of reporting functionality led to the Programme Team extracting data manually and analysing using Microsoft programs. • Not being a streamlined system or process also meant registration may have been a barrier due to lack of time to complete referrals. Potential barriers included: • How the system/name was originally set up meant it was not compatible with primary care systems, only the secondary care sector, meaning limitations in how GP’s could refer. This also meant that all referrals needed to be manually entered taking more time than if information could be drawn over from GP records • Only some NHS logins were possible (no nhs.net people able to access, no people from private or patients able to access) which further limited referrals and log ins • Users could only access system via work environment if they had access to a VPN facility. These limitations to accessibility (both who could register and how site was accessed) may have contributed to lack of engagement from busy healthcare professionals as not able to access the system/name from their secure work systems. This limited ability to respond outside of working hours may have limited vMDT Expert’s capacity to respond • Some vMDT Experts were found to have non-compatible email addresses and therefore discounted from participating. This could also have been the case for some referrers although this was not able to be monitored • Patients were not able to self-refer. Enabling patients to refer could have increased referrals. • Patients were not able to be empowered by being involved in feedback, which could have made the vMDT Expert more accountable to being involved
15
Lessons Learnt (what went well and considerations for future iterations) • It is recognised that it is difficult to create an IT system that can be future proofed completely. Where possible look at all eventualities and potential users of the system and try to design system that can be adapted to the current needs and anticipated future needs as much as possible. Background research and considerations of past models that may have similarities is recommended. • Fundamental to have a good understanding of what the programme team needs and wants a system to do, and a good understanding of the scope of the programme/service specification to ensure the system design is fit for purpose. • Ease of accessibility for all users of the platform/system is essential. Health care professionals work in busy environments and accessibility is a big incentive to assist with both referring and responding to requests, as well as consideration to information security environments (e.g. VPNs) • Consider the needs of PLWC. This system did not allow for referrals from PLWC but this is something that may have been in scope.. This would need to be monitored and considered how to best approach this model • Automated functionality for Programme Coordinators and central users, it is important to look at where processes can be automated to assist with workload
• Consideration of tenders and contracts of digital products and services should always be made in partnership with other departments (at Macmillan, at acute trusts etc.) and decisions made through collective expertise: these considerations should include costs of ongoing maintenance, what ability there is to change and make edits to the platform and what is included in contract • There is a need to be agile in the development of systems. The ease to tweak and refine iteratively is essential in a pilot situation to be able to test scenarios as they are encountered and come out with best recommendations • Overall all contracts with external parties should be entered into with full understanding that what is being built may need alterations and amendments once live. This should be addressed in the tender process and an agreement that both parties are happy (including costings for alterations and reiterations) should be carefully considered 16
3. Engagement Referrers and experts
Overview • A critical success factor for the vMDT was the recruitment and engagement of the vMDT Expert panel of Healthcare professionals in various fields and specialisms (see Appendix B) from across the UK • These vMDT Experts would be the panel to which patient cases would be sent onto for their professional feedback and opinion
• Another area critical to the success of the programme was engagement with referrers (across primary, secondary and community care settings) to refer patients into the vMDT. For this to occur the service was reliant on effective promotion of what the service offers, and the role healthcare professionals could have in the panel • To deliver the communication and advertising, a marketing and communications plan covering the 3-years was written by the programme team from both Macmillan and the Christie before programme launch in 2016 and both the Programme Coordinator and Macmillan were responsible for its implementation throughout the programme’s life
18
What happened / what change occurred
Potential impact on programme
Recruitment • The Programme team had no regular contact (regular email updates, L&D • From the start of the service (2016) Programme Coordinator and Programme opportunities etc.) with the vMDT experts to keep them engaged and active Officer were tasked to engage and recruit both experts and referrers from throughout programme. This may have led to low response rate from vMDT Healthcare professionals across UK. To do this they developed (in Experts to cases that they were allocated (see referral section for more partnership) a recruitment letter and boiler plate advertisement which was detail) sent to a range of professional bodies across the UK. The aim was to recruit • Query whether how engagement and promotion strategy was implemented as many different specialities as possible across a range of health contributed to lack of referrals professionals • Experts were recruited primarily by utilising Macmillan links and networks in addition to individual recommendations from the vMDT Clinical Reference Group and key stakeholders. Professional organisations/bodies, late effects clinics, cancer networks and alliances were also contacted to invite them to become involved with the project to help support this patient population • Close to 90 experts were recruited. A wide range of health professionals with different specialist areas were engaged to participate in the programme. The experts were mostly recruited early in the programme with no additional recruits in at least the last 18 months of the programme • Experts were provided with a welcome letter and user guide which gave an overview of the programme and expectations Marketing • Targeted promotional strategies were undertaken to promote the service to potential referrers. A marketing strategy was developed including promotion through conferences and events, online and print media, social media and promotion through communication channels of stakeholders. Much of the promotion was limited to surrounding area to The Christie Hospital and was not UK wide
19
Lessons Learnt (what went well and considerations for future iterations) • Understand clinical expertise required to launch programme to ensure recruitment of the right mix of experts into a MDT panel (or similar) • Ensure there is an ongoing engagement plan that incorporates both experts and referrers. Develop a clear communications plan with targeted and identified stakeholders as an initial step in the process and revisit this at regular intervals evaluating where there has been success as well as where improvements are needed. Ensure that all parties are clear of their requirements re: communication plan, implementation and ongoing promotion • Have an initial understanding of audiences (both referrers and experts in this case) and what they need to know (and what channels, e.g. posters in clinics, face to face meetings) to engage with new initiatives. All engagement strategies need to be targeted and need user involvement before being launched to ensure relevance and impact • Identification of clinical champions across sites and specialties is important to ensure there is clinical endorsement of new programmes to assist with buy in • National programmes need national engagement plans. Where a funded project team is in one area consideration should be made to engagement opportunities at a national level and where there is opportunity to get others to assist with this. Macmillan funded projects could link in to national opportunities through conversations with their Macmillan link person (geographical or UKO lead) Learning opportunities:
• Understand more about engagement of experts to understand how so many were recruited in a short period of time
20
4. Referral
Overview • The vMDT NHS3 platform was launched in November 2016. Referrals were made from health professionals who had registered as referrers (originally only from acute setting, until April 2017 when the programme scope extended to include referrals from Primary Care due to low numbers of referrals received). Referrals were manually allocated by the Project Co-ordinator to representatives from the expert panel. • Each case was allocated several different specialities dependent on the issues and symptoms the patient was experiencing. These ranged from specialist consultants within specific cancer groups, CNS and nurse specialists, psychologist, occupational therapist, dietitians, rehabilitation and late effects support specialist. The aim is to address as many of the issues or symptoms the patient was experiencing to provide a bespoke and holistic approach. • Usually five experts were allocated to a referred case. The decision on allocation was based made on expert’s area of expertise, cancer type and topic of referral. The first referral was received in March 2017, with the last being received in June 2018 (the decision to end the service was made in August 2018). In total 15 referrals were made by referrers (a total of 14 of these were deemed to be appropriate and were allocated to experts, the other was around treatment and prognosis and was not accepted as a case). 63 referrers registered for the system (70% (44) from secondary care and 30% (19) from primary care). Details on the referrals and referrers can be found in the Evidence/Additional Information pack.
22
What happened / what change occurred
Potential impact on programme
• Referrals initially only accepted from health professionals in Secondary care • Low referral rates: potentially demonstrates lack of national awareness in (November 2016) and were then expanded to health professionals in Primary primary and secondary care settings that programme was available (see care (April 2017). Total number of referrals was 15 (8 from Primary care, 7 marketing) from Secondary care). • Potential reasons for low response rates from experts to allocated cases: • Referrers needed a NHS login to register to then be able to refer. Some NHS • low engagement with experts (no regular contact or engagement plan for emails were identified post online programme development as not being able experts unless being allocated a case) to register for system (see software development) • technical barriers (only being able to access on secure N3 system meant • Feedback from vMDT Referrer survey respondents (n=4) that the system was limited access as it is anticipated that most health professionals would value not user friendly – an example included being unable to copy/paste from more flexibility and being able to access cases outside of the healthcare patient notes into the system. 33% (20) of the registered users logged onto setting) the system resulting in 15 referrals • whether experts were being correctly allocated to cases • There were low referral numbers throughout the programme (15 total referrals in 2 years, with 9 in 2017 and 6 in 2018) – (see Appendix F) • Cases were manually allocated by Programme Coordinator to experts looking at clinical expertise and specialty interests declared during the registration process. Where possible as many different health specialities were allocated to a case. Programme was reliant that the correct expert be allocated to cases • Each case went to an average of 5.8 experts. There was a 66% response rate from experts to cases which was inadequate as didn’t allow a full MDT experience • Intention that the process would allow for a 2-way conversation between referrer and expert during a 2-week feedback period. This didn’t occur, and the reality was low response rates from experts and experts not responding in most cases without receiving reminders that the case was open for review.
23
Lessons Learnt (what went well and considerations for future iterations) • A fit for purpose software system is required for success launch. Considerations should be made to where is there potential for automation in the process and to ensure that easily accessible and easy to use • A clear approach and continuous engagement plan should be devised in initial planning stages to keep both experts and referrers engaged. These plans should be considered from the perspective of each target audience and should be revisited at regular intervals. Look at how to keep experts engaged and consider who your audience is and their strengths and limitations
• Recognition that experts are a volunteer position. Consideration should be made to see is there need for incentives for responses (CPD points or similar) or for experts to sign a contract (or similar) to ensure they are aware of and act to their responsibilities. These considerations should be made in the initial planning stages 24
5. Resolution, expertise and advice
Overview • Once allocated experts were given two-weeks to respond to a case. There was an intended opportunity for a two-way conversation within the programme between referrer and experts to ask questions and for clarification until an appropriate resolution (this function was not utilised). A case report with comments from the vMDT Experts was manually collated by the programme coordinator, approved by Expert chair and emailed to the referrer. The programme coordinator then closed the case on Asckey. Most of the cases (n=14) were resolved.
26
What happened / what change occurred • Of 83 registered experts, advice was requested from approximately 55 (66%). Of the 55 experts allocated, 24 (44%) were utilised for more than one referral • Most referrals were related to pelvic cancer and haematology treatments, thereby influencing expert allocation • Expert allocation does not depict expert advice was provided, response rates were variable. • After 2-week feedback window a case report put together (manually) by programme coordinator and manually emailed to referrer • Of the 14 active referrals; 71% (10) achieved target of 21 days or less. Due to delayed expert responses/delayed approval from the Expert Chair, 29% (4) failed to meet the target. • There was not an opportunity for referrer to come back to vMDT Experts with additional questions • Cases received a variable response from allocated experts, which can affect the overall response. Some experts didn’t ever log into the system • 10 case studies were reviewed: • Range of time from treatment 1 year to 38 years • 4 had more than 1-line treatment • 3 very specialist advice – would seek specialist advice/refer on • 4 offered ideas re further testing and support advice – would possibly be offered by an effective MDT • 3 reassure and support advice – Probably managed by appropriately competent CNS
27
Potential impact on programme • Variability and quality response from experts and gaps in areas of expertise impacted the advice provided, as shown in the mixed feedback from referrers using the referral system: • “Sadly, I feel that the one time I needed to ask for information the advice provided was poor and was just reiterating what I already knew, it certainly wasn't expert advice” - Expert who made a referral • “The advice has been much appreciated... Patient seemed happy with the discussion process and range of opinions provided.” – Consultant Gastroenterologist • “It was difficult to fill in the form – took a long time.” – Macmillan GP • “Finding time to do the referral is the only limitation. Patient was very pleased that we did refer her though.” – Macmillan Nurse Consultant • Variable response rate and engagement from experts to case studies may impact on advice being able to be given Identified best case scenario (intended route of referral): 1.Referral made 2.Allocated to experts (at least 5 made up of an MDT of HP’s) 3.Experts log onto system and respond to case 4.Opportunity for 2-way conversation between experts and referrers, clarifications and further questions occur within 2-week feedback period 5.Case closed – case report sent to referrer 6.Opportunity for both referrer and expert to feedback on the experience
Lessons Learnt (what went well and considerations for future iterations) • Consider best and easiest access for everyone who will use the platform/tool/system and map their requirements accordingly when putting together programme requirements. This should include access, referral process, responses and resolution. Streamlining is key • Ensure clear identification of what needs to happen for experts (or stakeholders) to be engaged in the programme. What engagement principles will work for the specific audience and is there subsets of needs within this according to speciality (e.g. AHPs vs oncologists)
• Clear engagement and recruitment plan needed to ensure adequate skill mix for any expert panel • Ensure regular opportunity for feedback is built into the programme. Regularly monitoring what is and isn’t working and where quick improvements could be made is essential. 28
6. Sustainability and changes in landscape Considerations for long term sustainability and expansion of programme, evaluation methods and horizon scanning
What happened / what change occurred
Potential impact on programme
• The overall programme was a very manual process with need for the programme coordinator to physically do many aspects that could have been automated • There were several technology and system related issues that impacted the project (limitations of system designed, limitations of NHS3 system etc.) • Some experts were found to have email accounts ending in ac.uk or nhs.net that do not have access to the N3 network. This was only realised quite late on in the project that nhs.net accounts do not necessarily guarantee access to the N3 network and this has resulted in another barrier for expert contributions • Although it was originally planned for software company Asckey to incorporate data reporting features into the vMDT system, it became apparent during the project that the system did not provide appropriate data reporting functionality required for sufficient analysis of the service. It was also deemed too costly (and not cost-effective in relation to the rate of referrals) to implement required modifications. • The evaluation process was defined at the beginning of process but were not revisited throughout programme especially after changes made with opening referrals to primary care etc • The number of late effects clinics being introduced across the country have dramatically increased since inception of programme
• The vMDT system was originally set up with secondary care in mind and was not compatible in many ways with primary care systems. The referral process was manual and requires quite a lot of information, which was felt by the programme coordinator to be off-putting for HPs who would prefer forms that self-populate the required fields from patient records • With hindsight, it might have been more effective to have designed the system to fit in with primary care systems initially and target this area more than secondary care. Unfortunately, once the system was in place, no major changes of this kind could be made due to costing – it would not have been cost effective given the number of referrals being lower than expected • Granting access to selected individuals who have nhs.net or ac.uk accounts may be a consideration for future models, however, this may potentially be a further barrier for expert recruitment, as gaining access to the N3 network will take extra time and effort on the expert’s part, potentially putting them off the idea of registering. • The lack of data reporting functionality within the system led to the project team having to extract data manually and logging this in various methods using Microsoft programs. • It was hard to evaluate from initial evaluation standards as project had changed since they were developed. The low number of referrals meant that any additional data collected from users would be a large time investment with not a lot of additional benefit to the evaluation • Introduction of a number of late effects clinics across the country may have had impact on need of service as professionals had another route to refer to
30
Lessons Learnt (what went well and considerations for future iterations) • Be sure to be critical about need for programme. What is the current landscape and what is need for programme or project? Is there a way to anticipate national need for any programme on a smaller scale before funding multi-year programme? • Look at best approach to programmes and scaling – is national roll-out by a trust that is based in a local area best way to implement or are other approaches more suitable and sustainable? • Ensure programme is streamlined as much as possible – engagement with people with expertise to do this, taking time to step through processes, sense check etc. • Ensure evaluation methods clearly defined and information able to be collected in a well-defined way. Ensure these are being revisited and reviewed regularly. It is important to plan where information is collected, for what purpose, how it will be analysed (and by who and what cost is involved) and when the information is needed throughout the programme lifespan. There will be information that is useful for monitoring and therefore should happen throughout to inform the delivery of a service. • All monitoring and evaluation should be agreed with Macmillan and partner, and ownership to review and analyse and make judgement (i.e. to evaluate) informed by Macmillan and partner. • Ensure horizon scanning for potential changes in landscape is undertaken – look at possible ways of disruption proofing service including what is happening at local and national levels and what mandates the NHS and other health bodies may have
31
Appendices
Key Moments November 2016
March 2017
System launched
First referral received
April 2017
July 2017
August 2017
September 2017
May 2018
June 2018
Board Agreed to extend project scope to accept referrals for all cancer types and Primary Care
Completed system modifications for extended scope
Commenced promotion of project extension
Received first primary care referral
Board approved proposal for transition to ‘Ask the Expert’ model
15th referral received
Referral Dashboard Referrals by Month 5
4
3
2
SC PC
1
0
Referral ratio 8:7 (Primary : Secondary)
Referral Dashboard
Referral Dashboard
Referral conversion rate
%
14 of 69 ‘Registered Referrers’ have made a referral
20%
1 of 82 ‘Registered Experts’ have made a referral
1%
Ratio Primary Care v Secondary Care 8:7
Referral Dashboard 10
EXPERT RESPONSE RATES
9 8 7
44% 56%
6
29%
5
33% 11%
57%
33%
57%
20%
4
29%
0%
11%
60%
3
80%
17%
25%
60%
40% 14%
2
44%
0% 33%
43% 40%
798
807
811
29%
40%
20% 29%
20% 0%
0
785
50%
50%
1 0
67%
821
826
830
844
855
912
25%
14%
20%
929
931
938
Case number No response
Response not bespoke / doesn't offer advice
Good response
Case Studies 10 9 8 7
Case 821
44%
• Submitted Oct 2017
56% 6
29%
5
33% 11%
57%
33%
• Secondary care
57%
20%
4
29%
0%
11%
60%
3
80%
17%
25%
60%
• Disease group – Gynae
40% 14%
2
44%
0% 33%
40%
785
798
0 807
811
50%
29%
50%
1 0
67%
43% 20% 0% 821
826
40%
830
20%
• 5 experts allocated
29%
844
855
912
25%
14%
20%
• 5 responses received
929
931
938
• Positive feedback from referrer
Case number No response
• Region – South East
Response not bespoke / doesn't offer advice
Good response
Case Studies Case 830 • Submitted Oct 2017
10 9
• Primary care
8 7
• Region – South East
44%
• Disease group – Urology
56% 6
29%
5
33% 11%
57%
33%
• 5 experts allocated
57%
20%
4
29%
0%
11%
60%
3
80%
17%
25%
60%
40% 14%
2
44%
0% 33%
40%
785
798
0 807
811
50%
29%
50%
1 0
67%
43% 20% 0% 821
826
40%
830
20% 29%
844
855
912
25%
14%
20%
929
931
938
Case number No response
Response not bespoke / doesn't offer advice
Good response
• 3 responses received • Feedback received from referrer and patient – generally negative: “Sadly it had not effect on overall management of my patient.” •
vMDT panel had few experts with specialist knowledge of fatigue.
•
Experts with knowledge around fatigue were not available to comment on the case.
Case Studies 10
Case 912
9
• Submitted Jan 2018
8 7
• Primary care
44% 56%
6
29%
• Region – North East & Yorkshire
5
33% 11%
57%
33%
57%
• Disease group – Colorectal
20%
4
29%
0%
11%
60%
3
80%
17%
25%
60%
40% 14%
2
44%
0% 33%
40%
785
798
0 807
811
50%
20% 0% 821
826
40%
830
• 2 responses received
29%
50%
1 0
67%
43%
20% 29%
844
855
912
25%
14%
20%
929
931
938
Case number No response
Response not bespoke / doesn't offer advice
• 6 experts allocated
Good response
• Positive feedback received from referrer: “The advice was sound and a lot of test were suggested that she had not had done before, as well as advice regarding diet.”
User Experience Surveys (May 2018) Registered Experts 86.3% have logged on. 58.8% have been allocated to a case. Do you feel that the case was relevant to your area of expertise? Yes – 57.5% No – 0.0% I haven’t been allocated to a case – 38.3% Other – 4.3% “First case yes, second case no”
Has being a vMDT Expert Contributor had an impact on your general workload? Yes, significantly – 0.0% Yes, slightly – 10.0% No change – 82.0% Don’t know – 8.0%
51 responses
User Experience Surveys (May 2018) Registered Experts “Sadly I feel that the one time I needed to ask for information the advice provided as poor and was just reiterating what I already knew, it certainly wasn't expert advice. As an expert the advice I provided was then questioned as it was not available in that patients area.“ (Expert who made a referral.)
“It would be useful to have feedback on outcomes on the patients we have advised on.” “Perhaps this should be specifically for clinicians who do not have a well-resourced multi-professional MDT? In that case it could be factored into job-plans and not just exist as an add-on.”
“In theory a helpful an important source of advice for complex survivorship issues. In practice there is no allocated time within job plan, and being so over-run in clinical role the vMDT has to happen within personal time when clinicians should be having some down-time to avoid burn-out.”
“It became clear that the people reviewing my case hadn't read the provided patient information and suggested interventions that had already taken place. The time frame was too long.”
“Increase accessibility and visibility – I think the most important need is to identify the main target audience in terms of likely number of referrers .”
User Experience Surveys (May 2018) Referral Barriers 33.3% have logged on.
9 responses
Have there been any barriers which have prevented you from making a referral to the vMDT? I don’t have time to fill out the referral form – 11.1% I haven’t yet had a patient who would benefit from this service – 22.2% I find the vMDT system difficult to use – 11.1% I have an oncology team I can seek support from – 33.3% Other – 44.4%
“Don’t know enough about it.” “No time to complete referrals.” “I haven’t had a chance to look into what the system has to offer.” “Have to ask consent from patients before using it.”
Do you have any suggestions on how the vMDT could be improved? No – 77.8% Yes – 22.2% “Regular reminders – email MDTs to raise profile.”
Referral Feedback – 3 weeks post advice (referrer) 1. Was the referral process quick?
2. Was the system easy to navigate?
3. How much of a difference has the vMDT advice made to your decisions regarding management of this patient?
9 survey responses received
4. To what degree have you learned something from the vMDT?
5. Would you use the vMDT service again?
6. How did you hear about the vMDT? Contacted directly by Macmillan
X2
Word of mouth
X2
Social media
X1
Patient informed me
X2
Referral Feedback – 6 months post advice (patient) 1. How much of a difference has the vMDT advice made to your decisions regarding management of this patient?
2. To what degree has the vMDT advice changed the speed that your patient has been receiving tests/care for their issues?
3. Do you feel the vMDT advice has helped your patient to avoid unhelpful appointments?
5 survey responses received
4. Please estimate the effect that vMDT advice has had on your patient’s level of travel to appts re. their consequences.
5. Please estimate the effect that the vMDT advice has had on whether your patient’s holistic needs are being met.
6. Would you recommend the vMDT to a colleague?
Referral Feedback – 6 months post advice (patient)
2 survey responses received
1. How satisfied are you with how the vMDT advice has helped your situation overall?
4. How much do you feel your overall needs have been met, or are being addressed, since your HCP received advice from the vMDT?
2. How satisfied were you with the information given to you about the vMDT?
5. Thinking of the main symptoms / problems that concerned you the most, how much do you feel have been addressed since your HCP received advice from the vMDT?
3. How much has travel to hospital appointments been affected by the vMDT advice?
6. Overall, how much do you think your quality of life has changed since your HCP received advice from the vMDT?
The vMDT Process
https://vmdt.macmillan.thirdparty.nhs.uk
Registration
Referral
Allocation
New users complete online registration form
Referral form completed by healthcare professional and submitted to vMDT coordinator
vMDT coordinator allocates case to specialist experts for advice
Summary vMDT coordinator collates expert responses into a case summary for referrer
Closed case
Closed cases can be viewed on the system for reference or educational purposes
Registration vMDT Welcome Page
https://vmdt.macmillan.thirdparty.nhs.uk
Registration
Case Summary Report
vMDT Coordinator
Referring HCP
vMDT Referral Process HCP completes online referral form
Review case during ‘open forum’ stage
Notified referral allocated
Submits referral form
Notified case summary complete / case closed
Referral rejected / additional information required Notified of new referral
Review details
Referral accepted
Allocate Expert Panel (1 working day)
Chase response
Notify Expert Panel of allocation
Draft Report for chair approval
Submit final Report / close case
Expert Panel
Up to 5 working days to agree final content
Notified of allocation
Provide response
2 week live discussion period ‘Open Forum’ stage
Draft report approved by Chair