LifeCycle_Harkin 2023

InternationalJournalof TechnologyAssessmentin HealthCare

www.cambridge.org/thc

Policy

Citethisarticle: HarkinKR,SorensenJ, ThomasS(2024).Lifecycleevaluationof medicaldevices:supportingorjeopardizing patientoutcomes?Acomparativeanalysisof evaluationmodels. InternationalJournalof TechnologyAssessmentinHealthCare, 40(1), e2,1–12

https://doi.org/10.1017/S026646232300274X

Received:28November2022

Revised:16October2023

Accepted:14November2023

Keywords: medicaldevices;conceptualmodels; evaluation;lifecycle;review;qualitative synthesis;evidence

Correspondingauthor: KathleenR.Harkin; Email: harkinka@tcd.ie

Lifecycleevaluationofmedicaldevices: supportingorjeopardizingpatientoutcomes? Acomparativeanalysisofevaluationmodels

KathleenR.Harkin1 ,JanSorensen2 andSteveThomas1

1CentreforHealthPolicyandManagement,TrinityCollegeDublin(TCD),Dublin,Irelandand 2RCSIHealthcare OutcomesResearchCentre,SchoolofPopulationHealth,RoyalCollegeofSurgeonsinIreland(RCSI),Dublin,Ireland

Abstract

Objectives: Lackofevidenceregardingsafetyandeffectivenessatmarketentryisdrivingthe needtoconsideradoptingalifecycleapproachtoevaluatingmedicaldevices,butitisunclear whatlifecycleevaluationmeans.Thisresearchsoughttoexplorethetacitmeaningsof “lifecycle” and “lifecycleevaluation” asembodiedwithinevaluationmodels/frameworksusedformedical devices.

Methods: Drawingonqualitativeevidencesynthesismethodsandusinganinductiveapproach, novelmethodsweredevelopedtoidentify,appraise,analyze,andsynthesizelifecycleevaluation modelsusedformedicaldevices.Datawasextracted(includingpurpose;audience;characterization;outputs;timing;andtypeofmodel)fromkeytextsforcoding,categorization,and comparison,exploringembodiedmeaningacrossfourbroadperspectives.

Results: Fifty-twomodelswereincludedinthesynthesis.Theydemonstratedsignificant heterogeneityofmeaning,form,scope,timing,andpurpose.The “lifecycle” mayrepresenta singlestage,aseriesofstages,acycleofinnovation,orasystem. “Lifecycleevaluation” focuseson theoverarchinggoalofthestakeholdergroup,andmayuseasingleorrepeatedevaluationto informdecision-makingregardingtheadoptionofhealthtechnologies(Healthcare),resource allocation(Policymaking),investmentinnewproductdevelopmentormarketing(Tradeand Industry),ormarketregulation(Regulation).Theadoptionofalifecycleapproachbyregulators hasresultedinthedeferralofevidencegenerationtothepost-marketphase.

Conclusions: Usinga “lifecycleevaluation” approachtoinformreimbursementdecisionmakingmustnotbeallowedtofurtherjeopardizeevidencegenerationandpatientsafetyby acceptinginadequateevidenceofsafetyandeffectivenessforreimbursementdecisions.

Introduction

Patientsandhealthcarepayersvaluemedicaldevicesthataresafeandeffective.However,thereis verylittledirectclinicalevidenceregardingthesafetyandefficacyofmostmedicaldeviceswhen theyfirstenterthemarketorevenatthepointofreimbursement(1–7).Thishaspromptedmany stakeholderstoconsiderapplyingalifecycleapproachtoevaluationwiththeaimofimproving theevidencebaseovertime.

However,the “medicaldevicelifecycle” isapoorlydefinedconceptandthereisnoconsensus regardingwhatitencompassesnorregardingwhatismeantbylifecycleevaluation(8–11). Instead,theirmeaningisassumed,yetimplicitlyembodiedwithintheevaluationframeworks beingused.

©TheAuthor(s),2024.PublishedbyCambridge UniversityPress.ThisisanOpenAccessarticle, distributedunderthetermsoftheCreative CommonsAttributionlicence(http:// creativecommons.org/licenses/by/4.0 ),which permitsunrestrictedre-use,distributionand reproduction,providedtheoriginalarticleis properlycited.

Withoutcleardefinitions,differentpeopleassumedifferentmeaningsandusetheseconcepts fordifferentpurposes.Indeed,lifecycleevaluationmaymeangeneratingbetterevidenceovertime, butequally,itmaymeanacceptinglimitedorpoor-qualityevidenceearlyinthelifecyclebasedon thepremisethatmoreevidencewillbecomeavailablelater(12).Unfortunately,suchdifferences mayimpactpatientsafetyiftheyarenotmadeexplicit.Forexample,assumingthatadevicehas beenproventobesafeandeffectivebeforebeingmarketedmeanspatientoutcomesmaynotbe analyzedtotheextentthattheywouldbeifitwereknownthatthisisfrequentlyuntrue,thereby, delayingthedetectionofharmfuldevices(5;13–15).Therefore,itisnecessarytounderstandwhat differentpeoplemeanandunderstandbytheseconceptsandhowtheyusethem.

Objectives

Aim

Toexplorethedifferentmeaningsof “medicaldevicelifecycle” and “lifecycleevaluation” as embodiedwithintheevaluationmodelsusedbydifferentactorsinvolvedwithmedicaldevices, andthepotentialimpactthatdifferencesmayhaveonpatientsafety.

Methods

Theunitofanalysisforthisliteraturereviewisthemodelrather thanthearticles.SimilartoMichie,Stralen,andWest,weusethe term “model” tomean ““ahypotheticaldescriptionofacomplex entityorprocess.”” (16),whichinthisstudydenotesanyapproach, framework,modelortheoryutilizedtoevaluatemedicaldevices acrosstheirlifespan(SupplementaryMaterial1).Forthesakeof brevity,throughoutthisarticletheterms “model” or “conceptual model” willbeusedtoencompasstheentirerangeoftheoreticalor conceptualapproaches,models,andframeworks.

Theincludedmodelsweredrawnfromdiversedisciplinesand, consequently,demonstratedextensiveheterogeneity,requiring novelmethodsforanalysisandsynthesis.Thesearebriefly describedhereusingtheENTREQguidelines(17),withmoredetail providedwhereindicatedinthe SupplementaryMaterials

SynthesisMethodology

Usinganinductivequalitativeapproach,modelsweresynthesized usinganalytictechniquesdrawnfromgroundedtheory,qualitative contentanalysis,thematicanalysis,andmeta-ethnography(18–24).

ApproachtoSearching

Duetotheunexpectedlylargenumberoflifecycleevaluation approachesfoundintheliterature,anddrawingonmethods describedpreviously,apragmaticapproachtosearchingwas adopted(25–27).Indeed,complementaryandsnowballtechniques havepreviouslybeenfoundtobemoreeffectiveandefficientat identifyingrelevantmaterialwhenthetopiciscomplexandmultidisciplinary(26;27).Seminaltextsandmodelswereidentified throughscopingsearchesconductedinbiomedicalandmultidisciplinarybibliographicdatabases,theInternet,andspecificWeb sitesusingkeywordsandphrasescoveringtheconcepts “medical device,”“lifecycle,” and “model.” Snowballsearching(i.e.,searching references,referencesofreferences,andcitations)ofseminaltexts wasemployedtoidentifyadditionalmodels. ElectronicSearch strategy:(SupplementaryMaterial2).

DataSources

Systematicscopingsearches:PubMed,EMBASE,andtheCochrane Library.Pragmaticsearches:theinternet,GoogleScholar,and JSTOR.

InclusionCriteria

Anylifecyclemodelthatmaybeappliedtoamedicaldevice (however,thatmaybeconceptualized,e.g.,asaproduct,industry, innovation,therapy,technology,intervention,softwareorhardware,orasaspecifictypeofdevice)(SupplementaryMaterial1) coveringseveralaspectsorstagesofadevice’slifetimewaseligible forinclusion(SupplementaryMaterial3).Modelsdevisedsolelyfor evaluatingmedicineswereexcluded,becausetheirlifecyclestages differconsiderablyfrommedicaldevices,posingdifferentproblems andbeingsubjecttodifferentrequirements.

StudyScreeningMethods

Articlesweresystematicallyscreenedinphasesagainsttheinclusionandexclusioncriteria(SupplementaryMaterial4).Initial

screeningwasconductedconcurrentlywithsearching.Potentially relevanttextsappearingtotakealifecycleapproachwereidentified andsavedinadocumentarchive.

ModelSelection

Documentsinthearchiveweresystematicallyscreenedagainstthe inclusioncriteriafordataextractionintoanExceldatabase,where theywerethenscreenedagainstaselectionalgorithmtoselectone representativetextpermodel,withtheremainderservingasreferencematerials.Searching,screening,selection,anddataextraction iteratedwithdataanalysis.Selectionceasedwhendatasaturation hadbeenreached(i.e.,nosignificantlydifferentmodelsornew themeswereemerging).Potentiallyeligiblemodelsidentifiedsubsequentlyweresavedfortriangulation(SupplementaryMaterials 3 and 4).

ModelandStudyCharacteristics

Thecharacteristicsoftheincludedmodelsandtheirkeyreference textsaresummarizedin Table2 (SupplementaryMaterials8 and 9).

AppraisalRationale

Qualityappraisal(QA),thoughitsuseinqualitativeresearchis contested(28),servestodeepenresearchers’ understandingofthe databeingexamined,makingtheQAprocessvaluableforincreasinginsight(29). Appraisalitems:NoQAcriteriahavebeenuniversallyacceptedforappraisingconceptualmodels.Weadoptedthose usedbyD’Amouretal.(30),assessingwhetheramodelwas underpinnedbytheory,empiricaldata,and/oranexplicitliterature reviewstrategy,assigningascoreofoneforeachcriterionmet. Appraisalprocess:Modelswereappraisedratherthanarticles, therefore,wedrewonadditionaltextsfortheappraisalprocess sincecriteriamaybemetindifferentsources. Appraisalresults are summarizedin Table2 (SupplementaryMaterial7).Nomodelwas excludedbasedonitsscore.

DataExtraction

Extracteddataincludedstudycharacteristicsofreferencetexts;the model’spurpose,perspective,audience,characterization,scope, stagescovered,factorsincluded,focusofinterest,representation (e.g.,diagrams,graphs,mathematicalequations,ortextualdescriptions),levelofapplication,timepointsforanalysis,processdefinition;andthequalitycriteria.Datawereextractedprimarilyfrom thereferencetexts,butsupplementedfromothers. Software:Word andExcel2016forMac(Version16.70).

NumberofReviewers

ThisstudywasconductedaspartofaPhD,therefore,onereviewer (KH)conductedallaspectsofsearching,screening,selection,data extraction,qualityappraisal,analysis,andsynthesis,withsupervision,oversight,discussion,reflection,andreviewbytwosupervisors(STandJS).

Coding

Theunitofanalysiswasthemodel,consequently,codingfocusedon thedataextractedfromthedocuments.Eachmodelwascategorized intooneoffourbroadperspectives(SupplementaryMaterial5).

Asanalysisprogressedextracteddatawascodedintoincreasingly broadandmoreabstractcategories,enablingcomparisonwithin andacrosscategories.Summariesofeachmodelwereprepared,and visualtoolscreated(Analysiscards)toenableavisualcomparison ofthemodels.

DerivationofThemes

Usingtheconstantcomparativemethod,togetherwithreflective writing,whichsurfacedhigher-orderpatternsinthedata,themes relevanttopatientsafetywerederived. Quotations areusedto supportthefindings.

PositionalityandReflexivity

Theresearcherscomefromamedical,HealthTechnologyAssessment(HTA),and/orpolicybackgr ound,mostly,fromapositivist/ postpositivistparadigm,buttakeapragmaticworldview.Weare interestedinpatientoutcomesandhealthcareefficiency.Consequently,personalprofessionalperspectivesandpre-conceptions werebracketedduringtheinterpretiveprocessandaninductive approachwastakentogroundtheanalysisinthedata( 31 ,p.27, 193 –195).

Trustworthiness

Toensureresearcherconsistency,asystemofdouble-entry,comparison,andvalidationofanalyticdatawasdeveloped.ThedocumentarchiveandExceldatabaseprovidearesearchaudittrail,with SupplementaryMaterials ensuringtransparency.

SynthesisOutput

Severalsyntheseswerecreated,includingacomparativeanalysisof themodels,thedevelopmentofamodeltypology,andathematic analysis.Thisarticlereportsonthecomparativeanalysis.

Reporting

Thefindingsarepresentedinthreesections,withabriefoverview first,followedbyatabularandnarrativedescriptionofthemeaningsattributedto “medicaldevicelifecycle” and “lifecycle evaluation.” Thediscussionexploresimplications,whilsttheconclusionsetsoutthekeyissuesandpotentialsolutions.

Findings

Fifty-twomodelswereincludedinthesynthesis,with51keytexts, drawnfrommultipledifferentperspectives,andorganizedintofour broadcategoriesforcomparison.Ninemodelswerederivedfrom Healthcare(HC),16fromPolicymaking(Policy),eightfromRegulation(Reg),and19fromTradeandIndustry(T&I). Table1 lists themodelsaccordingtotheirperspectives,providingabbreviated namesandthereferencenumbersforkeytexts,withtheirreferencesprovidedin SupplementaryMaterial6.

Themodelswerecharacterizedacrosssevendimensions relatedtothetypesofdevice,model,lifecycle,evaluation,and data;dateoffirstappearance;andQAscores,whicharesummarizedin Table2,whilst Table3 summarizesandcomparestheirkey features,includingpurpose,primaryintendedaudience,lifecycle characterization,outputs,timing,andscope(Supplementary Material10).

WhatisMeantby “MedicalDeviceLifecycle”?

Inthissection,wedescribethedifferent forms, scopes,and typesof process usedtorepresentthelifecycleofamedicaldevice.The mostcommonlifecycle form usedisalinearseriesofstages (thoughiterationmayoccur)movingfromtheidea,throughits development,growth,maturity,decline,toend-of-life(orasubset ofthese).Asecondformisacycleofinnovation,witheachoriginal inventionbeingincrementallyrefined,whichmeansthat,for thesemodels,thelifecycleencompassesseveralgenerationsofa device.Athirdformisapatterndescribinghowasinglelifecycle phasechangesovertime(e.g.,salesoradoption),therefore,these modelsfocusonhowtheparameterisaffectedratherthanthe effectstheproducthasonpatients/end-users.Inasense,theseare notactuallydevicelifecycles,buttheyare,nevertheless,oftenwhat ismeantbyadevice ’slifecycle.Anothersetofmodelsdescribethe lifecycleasmovementthroughvariouslevelsinasystem,andare, therefore,multi-levelevaluatio ns.Forexample,whereanovelidea isdevelopedintoatechnology,whichbecomesacomponentina device,thatisitselfincorporatedintoasystem(e.g.,software developedforaswitch,usedinamobilearm,whichbecomesa componentinaroboticassistant).

Tocomparemodels’ scope, weutilizedtheHealthCareTechnologyLifecycle(HCTLC)model(thegreenpartin Figure1)from theWorldHealthOrganization(WHO)(SR-20).Itillustratesthe phasesofthemedicaldevicelifecycleintermsoftheactivities/ processesrequiredacrossitslifespan.Thisincludes17specific stagesofactivitygroupedintothreephases – Provision,Acquisition,andUtilization.Regulationmaybeconsideredasaseparate process,butinthismodelitisconsideredtobeanintegralpart, whereregulatorycomplianceisnecessaryforthedevicetotransitiontothenextstage.Thefocusofinterestduringtheprovision stageisprimarilyonthenewproductdevelopmentprocess,whilst theacquisitionphasefocusesprimarilyonadoptionorsales (i.e.,diffusion).Duringutilization,themainfocusofthemodels isonevaluatingoutcomes.However,whilstcertainmodelscoverall ofthesephasesandstages,generally,mostcoveronlysome.Infact, Table3 showsthatthescopeofthelifecyclemodelsvariedconsiderablyacrossperspectives.Forexample,theT&Imodelscoveronly theprovisionstages,andapproximatelyhalfofthePolicymodels donotincludetheutilizationstage,whichmeansthattheseevaluationmodelsdonotexamine(toanygreatextent)whathappens afteradeviceentersthehealthcaresystem.

Tocomparethe typesofprocess embodiedinthemodels,we utilizedVandeVen’sprocessdefinitions –“(i)alogicusedtoexplain acausalrelationshipinavariancetheory,(ii)acategoryofconcepts thatrefertoactionsofindividualsororganizations,and(iii)a sequenceofeventsthatdescribehowthingschangeovertime” (32, p.169).Wefoundthatfrequentlythemodelsencompassedboththe secondandthirdofthesedefinitions.Nevertheless,approximately halfofthemodelspredominantlyusedaprocessdefinitionthat referredtoactors’ actions,aquarterreferredtoasequenceofevents, whilstfiveusedboth.Themajorityofthesewerestagemodels,thatis theydescribedthelifecycleasdistinctstagesratherthanasacontinuousdistributionofaparticularvariable.Thiscontrastswiththe sevenapplyingthefirstdefinition,exploringcausalrelationships,all ofwhichcharacterizedthelifecyclequantitatively,andnoneofwhich focusedonevaluatingpatient/end-useroutcomes.

WhatisMeantby “LifecycleEvaluation”?

Asshownin Tables2 and 3,lifecycleevaluationdiffersintermsof what getsevaluated, when, how,and why. Mostcommonly, what is

Table1. Listofmodelsincludedinthesynthesis

Modelname

NewProductDevelopment

Lifeofaninnovation

Abbreviatedname, Author(s)ofReferencetext,PublicationYear (SupplementalReference-number)SymbolforBroadPerspectiveassignedto

Baldock-NPD, Baldock,1960 (SR-1)™

DOI, Rogers,1962 (SR-2)Π

BassModel Bass, Bass,1969 (SR-3)™

ProductLifeCycle

7stagesinthecareerofamedicalinnovation

Lifehistoryandroutinizationofaninnovation

NewProductProcess

BusinessLifecycle(High-technologyventures)

IndustryLifeCycle

NewProductDevelopmentProcess

Adiffusiontheorymodelofadoptionandsubstitutionforsuccessivegenerationsof high-technologyproducts

Stage-gatesystem(fornewproductdevelopment)

TechnologyAdoptionLifeCycle(High-TechMarketingModel)

GeneralizedBassModel

TechnologyReadinessLevels

MedicalDeviceDesignandDevelopmentprocess

VATechnologyTransferProcess

Aframeworkofiterativeeconomicevaluation

RE-AIMFramework

HealthCareTechnologyLifeCycle

Majorphasesinthelifespanofamedicaldevice

Asystems-baseduser-centeredapproachtohealthcaredesign

ConceptualModelforConsideringtheDeterminantsofDiffusion,Dissemination,and ImplementationofInnovationsinHealthServiceDeliveryandOrganizations

TotalProductLifecycle(TPLC)

TechnologyadoptionlifecycleConceptualattractor&HFramework

EquipmentLifeCycle

Technologyriskandreadinessassessmentframework

IDEALFramework

IndustrialEmergenceFramework

Medicaldevicedesignanddevelopmentstage-gateprocess

TheInnovationLifeCycle

LifecycleofTechnology

EAESguidelineonmethodologyofinnovationmanagementinendoscopicsurgery

PLC, Levitt,1965 (SR-4)™

7Sm-IC, McKinlay,1981 (SR-5)Π

IRP, Yin,1981 (SR-6)Π

BAH-NPD, Booz,Allen&Hamilton,1982 (SR-7)™

BLC, Galbraith,1982 (SR-8)™

ILC, Gort&Klepper,1982 (SR-9)™

CK-NPD, Cooper&Kleinschmidt,1986 (SR-10)™

Norton-Bass, Norton&Bass,1987 (SR-11)™

SG-CK-NPD, Cooper,1990 (SR-12)™

TALC, Moore,2001 (SR-13)™

G-Bass-M, Bass,Krishnan,&Jain,1994 (SR-14)™

TRL, Mankins,1995 (SR-15)Π

MDDP, FDA,1997 (SR-16)®

VA-NPD, Sheredos&Cupo,1997 (SR-17)Ψ

4S-IEE, Sculpher,Buxton,&Drummond,1997 (SR-18)Π

RE-AIM, Glasgow,Vogt,&Boles,1999 (SR-19)Π

HCTLC, Cheng,2003 (SR-20)®

MDLS, Cheng,2003 (SR-20)®

SUHCD, Clarksonetal.,2004 (SR-21)Ψ

DDDII, Greenhalghetal.,2004 (SR-22)Π

TPLC, FeigalJr.fortheInstituteofMedicine,2010 (SR-23)®

TALC-CAHF, Meade&Rabelo,2004 (SR-24)™

ELC, Worm(THET),2015 (SR-25)Ψ

IRM-TRL, Mankins,2009 (SR-26)Π

IDEAL, McCullochetal.,2009 (SR-27)Ψ

IEF, Phaaletal.,2009 (SR-28)™

SG-MDDP, Pietzschetal.,2009 (SR-29)™

IC+, Croslin,2010 (SR-30)™

TLC, Myttonetal.,2010 (SR-31)Π

EIM-2DA, Neugebauer&Beckeretal.,2010 (SR-32)Ψ

NewProductDevelopmentFramework Bhuiyan-NPD, Bhuiyan,2011 (SR-33)™

ConceptualFrameworkoftheUniversitySpin-offVenturingProcess

Medicaldevicelife-cycle

HealthProductVigilanceFramework

TheInnovationCycle(CIRAS)

TheInnovationCycle(WW)

TheintegrationofriskmanagementprocesswiththelifecycleofMDsoftware

Treatment “lifecycle” framework

ProductInnovationLifeCycle

USVP, Rasmussen,2011 (SR-34)™

MDLC, Velazquez-Berumen,2011 (SR-35)Π

HCanada-MDRegLC, HealthCanada,2013 (SR-36)®

IC, CIRAS,2013 (SR-37)™

WW-IC, Wright&Weinstein,2013 (SR-38)Ψ

IRM-SaMDDP, Pecoraro&Luzi,2017 (SR-39)™

RxLCF, Provoostetal.,2014 (SR-40)Ψ

PILC, Baeyens,2016 (SR-41)Π

Table1. (Continued)

Modelname

IDEAL-DFramework

Abbreviatedname, Author(s)ofReferencetext,PublicationYear (SupplementalReference-number)SymbolforBroadPerspectiveassignedto

IDEAL-D, Pennelletal.,2016 (SR-42)Ψ

Nonadoption,abandonment,scale-up,spread,andsustainability(NASSS)Framework NASSS, Greenhalghetal.,2017 (SR-43)Π

HealthTechnologyLifeCycle

Optimalmethodologyfortheintroductionofneworthopedicimplants

NASASpaceflightProjectLifeCycle

HTLC, Gutiérrez-Ibarluzea,Chiumente&Dauben,2017 (SR-44)Π

OIM-DA, Hannanetal.,2017 (SR-45)Ψ

PrLC, NASA,2014 (SR-46)Π

NewHealthTechnologies – Lifecycleforintegrationintohealthcaresystems nHTLC4I, Parisetal.,2017 (SR-47)Π

Lifecycleofmedicaldevices – Lifecycleapproachtoregulationandtheimportanceof reportingincidentstotheTGA

EUnetHTA’slifecycleapproach

TheDeviceDevelopmentProcess

Lifecycleofamedicaldevice

TGA-MDRegLC, Reeves&Garcia,2014 (SR-48)®

EUnetHTA-MDLC, Meyer,Brühl&Omstad,2018 (SR-49)Π

FDA-MDRegLC, FDA,2018 (SR-50)®

Swissmedic-MDRegLC, Swissmedic,2019 (SR-51)®

Similarabbreviationsindicateasimilarnameorfocus:DA,decisionalgorithm;DOI,diffusionofinnovations;EAES,EuropeanAssociationforEndoscopicSurgery;EIM,endoscopicinnovation management;EUnetHTA,EuropeannetworkforHealthTechnologyAssessment;FDA,FoodandDrugAdministration;HCanada,HealthCanada;IC,innovationcycle;IDEAL,idea,development, exploration,assessment,andlong-termstudy;4S-IEE,4-stageiterativeeconomicevaluation;IRM,integratedriskmanagement;IRP,innovationroutinizationprocess;MD,medicaldevice;MDDP, medicaldevicedevelopmentprocess;MDRegLC,medicaldeviceregulatorylifecycle;NASA,NationalAeronauticsandSpaceAdministration;nHTLC4I,newhealthtechnologylifecyclefor innovation;NPD,newproductdevelopment;OIM,orthopedicinnovationmanagement;PLC,productlifecycle;RE-AIM,reach,efficacy,adoption,implementation,maintenance;SaMDDP, softwareasamedicaldevicedevelopmentprocess;SG,stage-gate;TALC,technologyadoptionlifecycle;TGA,TherapeuticGoodsAdministration;TLC,Technologylifecycle;USVP,UniversitySpinoffVenturingProcess;VA,VeteranAdministration.

Symbolskey: ™,TradeandIndustry; Π,Policymaking; Ψ,Healthcare; ®,Regulation.

Table2. Summaryofmodelandstudy(referencetext)characteristics

Combinesbothasequenceofeventsthatdescribehowthingschangeovertimeandacategory ofconceptsthatrefertoactionsofindividualsororganizations

Table2. (Continued)

Table2. (Continued)

Table3. Summaryoffindingstable

(Continued)

Figure1. ModelarchetypesassociatedwiththeHealthcareTechnologyLifeCycle(adaptedfromWHO,SR-20).

evaluatedarethefactorsthatareusedtocharacterizethelifecycle, indicatorsoflifecycleprogression,factorsinfluencingthelifecycle trajectoryorthatareaffectedbyit,and/ortheoutcomesfrom deployingatechnology.However,relativelyfewofthemodels, andnoneofthosefromtheT&Iperspective,focusprimarilyon evaluatingpatient/end-useroutcomes.

Anevaluationmaybeaonce-offeventencompassingtheentire lifecycleinasingleevaluation(e.g.,usingempiricaldatatodescribe thesales/adoptionpatternovertimeorbyevaluatingdifferent lifecyclestagesatasinglepointintime,eitherretrospectivelyusing existingdataorprospectivelyusingmodelingtechniques)oritmay bealifecycleevaluationbyvirtueofthefactthattheevaluationis repeatedatseveraltimepointsacrossthelifeofatechnology.Itis thislattertypethatisgenerallyconsideredtobea “lifecycle approach” bytheHTAcommunity(33).However,22ofthemodels includedinthissynthesisdidnottakethisapproach.

How thelifecycleisevaluateddependsontheparticularpurpose ofagivenmodel.Itmaybeperformedusingquantitative,qualitative,ormixedtypesofdata,toconductadescriptive,explanatory, predictive,orprescriptiveevaluation. Table3 showsthatthemost frequentpurposeofevaluationistodefineparticularactionsor activitiesthatneedtobeundertakenatthedifferentlifecyclestages, only15ofwhichdescribeapproachestoevaluatingend-useror patientoutcomes.

Themainreasonforadoptingalifecycleevaluationapproachis toaddressaneedforinformation.Themodelsareusedfor gathering,generating,orconveyinginformation/evidence requiredformakingthebestdecisionstoachievetheoverarching goal.Thepurposeslistedin Table3 aresimplyintermediate purposes,whilsttheoverarchinggoal(andthecorresponding decision)variesacrossperspectives.TheprimarygoalforHCis toprovidesafe,effective,andefficienthealthcare,therefore,its decisionsrelatetotheadoptionofcl inicallyeffectivetechnologies. WhilstforT&Ithegoalistoachievecommercialsuccessand generateprofit.Todothisitmustdecideontheappropriate resourcestoinvestandthetimingofinvestment,usuallyinnew productdevelopmentand/ormarketing.Policymakersoftenhave dualpolicygoals – topromotetradeandindustryandtoensure theprovisionofhigh-qualityandcost-effectivehealthcare.Thus, theirdecisionsdependonthepolicygoal,butgenerallyrelateto marketregulationorresourceall ocation.Sometimes,however, thesegoalsareconflicting,becauseeasingaccesstohealthcare marketsfortradeandindustryusuallymeansreducingthequality andquantityofevidencerequiredformarketaccess,which reducestheevidenceavailableforevaluatinghealthtechnologies fortheircost-effectiveness. “ Tensioncanarisebetweenthese objectives.Expediencymustbebalancedagainstadequaterigor, affordabilitywithaccess.Aligningregulatoryobjectiveswith broadereconomicandindustrialpolicy(e.g.,topromoteinnovation,employment,growth,ex portandtrade)mayresultin tensionwithgoalsofmanagingcosts.Inaddition,eachobjective willbeprioritiseddifferentlybystakeholdergroups,addinga politicaldimensiontotheprocess. ” (SR-47,p.118).

Theexplicitgoalforthosefromtheregulatoryperspectiveis not,asisfrequentlyassumed,toensurethatmedicaldevices enteringthemarketaresafeandeffective(SR-35),butratherthat theregulatoryrequirementsoftheirparticularjurisdictionare met. “Pre-marketcontrolisperformedonthedevicetoensurethat the product tobeplacedon-marketcomplieswithregulatory requirements. ” (SR-20,p.20,boldinoriginal)Therefore,the safetyandeffectivenessofmedicaldevicesatmarketentryis dependentonthespecificlegalrequirementsinagiven

jurisdiction.Regulatorymodelsdiffer,butforthemostpart regulatorsendorsetheguidelinesoftheGlobalHarmonization TaskForceandfollowtheapproachesillustratedin Figure2 suggestedbytheWHO(SR-20).

Medicaldeviceregulatorsareresponsiblefordecidingwhether ahealthtechnologymaybelegallymarketedintheirjurisdiction. Insomejurisdictions,this sometimes involvesamarketauthorizationprocess,wheretheproductscanonlybeplacedonthe marketifauthorizedtodosobytheregulators(e.g.,intheUSand Canada)usuallyonthebasisofev idenceofsafetyandeffectivenessgleanedfromtheliteratureorclinicalinvestigations. “… the premarketnotification,or510(k),processisaclassificationprocess,whereasPMA[Pre-marketApproval]isadeterminationof safetyandeffectivenessthatleadstoapproval. Forthe510 (k)process,devicesare cleared formarketing,notapproved,and devicesmaynotbemarketedas ‘approvedbyFDA ’” (SR-23,p.12, emphasisinoriginal).Inotherjurisdictions(e.g.,Australia, EuropeanUnion(EU),Switzerland)marketauthorizationisnot required,insteadmanufacturersofhigh-riskdevicesmustapply foranassessmentoftheirconformitywiththelegalrequirements byanexternalbody,aconformityassessmentbody(CAS) (SR-51).IftheCASdeemsthemt obecompliantitauthorizes themtoaffixamarking(e.g.,theCEmarkingintheEU)ontheir device,alegalrequirementformarketingthedevicethatindicates itislegallycompliant.Thisisnotthesameasmarketauthorization, “ Unlikemedicinalproducts,medicaldevicesdonotundergo anofficialauthorisationprocedu re.Forthesedevices,Switzerland followswhatisspecifiedfortheEuropeanUnion(EU)systemof complianceassessmentandcertification,basedonbilateralagreements.Compliancewithinternationallyvalidnormsisevaluated byprivateentities ” (SR-51).

Allregulatorymodelsincludearequirementforanevaluationof clinicaldataforallbutthelowest-riskdevices,butnotnecessarily onthesamedevice,normusttheclinicaldataincludearandomized controlledtrial(RCT). “Whatconsumerprotectionsarerequiredto bringnewproductstomarket? FDAisrequiredtoprovidethe leastburdensomepathtomarket Thismeansthatthetrials shouldbeparsimoniousinsize,focusedinobjectives,andnot cumbersomeinexecution.Ifthequestionscanbeansweredina post-marketingperiodthentheyshouldnotberequiredbefore marketing” (9,slides5,14).Consequently,thisregulatorylifecycle approachmeansthatevidencegenerationisfrequentlydeferred untilthepost-marketphase(34).

Discussion

Thelackofevidenceavailableatmarketentryisdrivingthepushto adoptalifecycleapproachtoevaluationforhealthcarereimbursementdecisions(SR-49).However,thedifferencesnotedin Tables2 and 3 illustratethat,despiteusingthesamewords,peopleoften meanverydifferentthingsandhavedifferentreasonsforusinga lifecycleevaluationapproach.Nevertheless,theconcepts,as embodiedinthemodelarchetypes,havechangedlittleovertime. Whathaschanged,asevidencedbytherelativelyrecentemergence ofthehealthcaremodels,istheincreasingpressureforearlieraccess tonovelhealthtechnologies,resultinginincreasedpressureon healthcareprovidersandregulators.

However,thelackofevidenceavailableatmarketentry/reimbursementisrelatedtotheregulators’ adoptionofalifecycle approach. “Theconceptoflifecycleregulation alsoreferstoan approachtoregulationthatfactorstheprospectofevidence

Figure2. Regulationacrossthemedicaldevicelifecycle(adaptedfromWHO,SR-20).

generationpostapprovalintopreapprovaldecisionmaking” (34, p.824).Therefore,adoptingalifecycleapproachhasmeantthatthe regulatorshaveacceptedlimitedandpoor-qualityevidence,allowingdevicestoenterthemarketbasedontheexpectationthatmore evidencewillbecomeavailableoncethedeviceisinuse(34). However,thisisfrequentlynotthecase(35–37).

Oncedevicesareplacedonthemarketregulatorsrelyonpostmarketstudiestoaddressuncertaintyregardingeffectivenessand vigilance/surveillanceactivitiestodetectadverseoutcomes.Yet, Rathietal.haveshownthatmanypost-marketstudiesareslow tobecompletedornotconductedatall(38).Whilstpost-market surveillanceisdesignedtodetectonlythemostsevereadverse outcomes,whicharejustafractionofadverseoutcomesthatmatter topatients(3;39).

Furthermore,cliniciansandpatientsarenotreadilymade awareofevidenceregardingadevice ’spoorsafetyrecord,as demonstratedbyPeters,Pelle rin,andJanney,whosestudy showedthatrecallsofClassIIIdevicesintheUStookmorethan 8monthstocomplete( 40).Severalauthorshavealsodemonstratedthatalthoughadeviceha sbeenrecalleditmaystillbeused asapredicate(i.e.,asubstantiallyequivalent,legallymarketed) devicetosupportthemarketauthorizationofmanysubsequent devicesthroughequivalenceclaims(38 ;41 –44 ).Furthermore, becausethisevidenceeitherneverbecomesavailable,oronly becomesavailableyearslater,m anypeoplemaybeaffectedbefore itbecomesclearthatitisclinicallyinferiortoalternativeinterventions(13–15 ;45).

Therefore,ifHTAistofollowtheexampleoftheregulators’ lifecycleapproach,itwouldmeanacceptinglimitedandinadequate evidenceofsafetyandefficacyforreimbursementdecisionswith thepromisethattherequiredevidencewouldlaterbecomeavailable.However,thisassumesthatthedeviceislikelytobebeneficial, unlikelytocauseharm,andevidencewillbemadeavailable(ina timelymanner).Unfortunately,ifmanufacturershavealready securedpublicfundingfortheirdevices,thenthereisverylittle incentiveforthemtogenerateadditionalevidence,whichiscostly andmightshowthattheirdeviceislesssafeoreffectivethan alternatives,resultinginitsmarketfailure.However,withoutevidencetheassumptionthatadeviceissafeandeffectivecannotbe challengedorconfirmed.

Industrymayarguethathigh-qualityevidenceisdifficultto developduetoinherentdifficultiesinconductingRCTsformedical devices(46),andthatthecostsaretoogreataburdenforthesmallto-mediumenterprises(SMEs)thatapparentlypredominateinthe industry(47).However,whyshouldpatientsbeartheburdenof havingunprovendevicesusedonthemsothatSMEscanbespared thecostofproperlyevaluatingthem?

Instead,governmentscouldcreatehealthcareinfrastructures tosupportrigorousevidencegenerationinasefficientamanner aspossible.Thereareschemesfor “ coveragewithevidence development” ,however,fewdevicesareinvestigatedinthisway. Therefore,amoresystematicappr oachisneededthatenrollsall patientsandallinterventionsintoanongoingandadaptiveclinicaltrialcomparingclinicaloutcomesforalltreatments.This requiresrobustdatacollectionandevaluationsystemswithin healthcare,withsecureindividualpatientanddeviceidentifiers, datalinkage,andstrongmechanismsofdataprotection.Sucha systemwillrequiremulti-stakeholderengagementandhuge investmenttodevelop,andislikelytobecomplextooperationalize.Nevertheless,itspotentialforimprovingclinicaloutcomes andpatientsafetyisimmense.Additionally,itwouldprovidea meansforindustrytodevelopar obustevidencebasetosupport

applicationsforreimbursementandimprovepayers’ abilityto fundthemostcost-effectivei nnovations.Therefore,future researchshouldinvestigatehowbesttodothis.

Limitations

Thisisnotanaggregativesystematicreview,somodelsexistthathave notbeenincluded.Nevertheless,byincludingalargesamplefroma broadvarietyofsourcesandperspectivesthisreviewprovidesa reasonablepictureofthegeneralformandcontentoflifecyclemodels beingused.Duetothebroadinclusioncriteriamanyoftheincluded modelsarenotintendedspecificallyformedicaldevices,however, thisheterogeneityfacilitatedarichexplorationofeachdiscipline’ s perspectivesandmotivations.Additionally,themethodsweused, particularlytheQAcriteria,arenotvalidated,therefore,future methodologicalresearchshouldestablishbestpracticesforconductingreviewsofmulti-disciplinaryconceptualmodels.

Conclusions

Themedicaldevicelifecycleandlifecycleevaluationhavedifferent meanings,anddifferentpurposesfordifferentstakeholders.The lifecycleapproachadoptedbyregulatorshasresultedinthedeferral ofevidencegenerationtothepost-marketperiod,resultinginalack ofevidenceforreimbursementandclinicaldecisions.Therefore, researchisneededtoevaluatepatientoutcomesarisingfromearly accesstomedicaldevicesbasedonthepromiseoffutureevidence comparedwithevidence-basedinterventions.Furthermore,the HTAcommunitycouldbecomeinvolvedinsettinguprigorous systemswithinhealthcaretosystematicallycollectandanalyzedata onpatientoutcomesfromthehighest-riskdevices(SR-44).Meanwhile,HTAbodiesshouldnotallowlifecycleevaluationtobeused asanexcuseforacceptinginadequateevidenceandshouldinsiston appropriateevidencebeingmadeavailableforreimbursement decisions,asthismaybethelastdefensesomepatientshaveagainst sub-optimaldevices(1).

Supplementarymaterial. Thesupplementarymaterialforthisarticlecanbe foundat https://doi.org/10.1017/S026646232300274X

Dataavailabilitystatement. AccesstotheExceldatabasewillbemade availableonreasonablerequestfollowingcompletionofthePhD.

Fundingstatement. ThisworkwassupportedbytheIrishHealthResearch BoardthroughitsSPHeREProgramme(SPHeRE-2013-1).

Competinginterest. Theauthorsdeclarenone.

References

1. HulstaertF, PouppezC, Primus-deJongC, HarkinK, NeytM. Evidence gapsfordrugsandmedicaldevicesatmarketentryinEuropeandpotential solutions[Internet].Brussels:BelgianHealthCareKnowledgeCentre (KCE);2021[cited2021Dec17],p.227.ReportNo.:KCEReports347. D/2021/10.273/45.Availablefrom: https://kce.fgov.be/sites/default/files/ atoms/files/KCE_347_Evidence_gaps_Europe_Report_V2.pdf

2. VanNormanGA.Drugsanddevices:ComparisonofEuropeanandUS approvalprocesses. JACCBasicTranslSci. 2016;1(5):399–412.

3. SamoreMH, EvansRS, LassenA,etal.Surveillanceofmedicaldevicerelatedhazardsandadverseeventsinhospitalizedpatients. JAMA. 2004; 291(3):325-334.

4. SamuelAM, RathiVK, GrauerJN, RossJS.Howdoorthopaedicdevices changeaftertheirinitialFDApremarketapproval? ClinOrthop. 2016;474 (4):1053-1068.

5. WhiteSK, WaltersAN.Assessingriskbyanalogy:AcasestudyofUS medicaldeviceriskmanagementpolicy. HealthRiskSoc. 2018;20(7–8): 358-378.

6. ZhengSY, DhruvaSS, RedbergRF.Characteristicsofclinicalstudiesused forUSFoodandDrugAdministrationapprovalofhigh-riskmedicaldevice supplements. JAMA. 2017;318(7):619-625.

7. CurfmanGD, RedbergRF.Medicaldevices – Balancingregulationand innovation. NEnglJMed. 2011;365(11):975-977.

8. WormA. Managingthelifecycleofmedicalequipment[Internet].London, England:TropicalHealthandEducationTrust(THET);2015Sep[cited2018 Jan9],p.12.Availablefrom: https://www.thet.org/wp-content/uploads/2017/ 08/THET_Managing_the_medical_equipment_lifecycle_LOW-RES.pdf

9. FeigalD. Totalproductlifecycle[Internet].Onlinepresentedat;2003Jan 16[cited2021Jun1].Availablefrom: https://webarchive.library.unt.edu/ eot2008/20090120125024/http://www.fda.gov/cdrh/strategic/presenta tions/tplc.ppt

10. McKinlayJB.Frompromisingreporttostandardprocedure:Sevenstages inthecareerofamedicalinnovation. MilbankMemFundQHealthSoc 1981;59(3):374–411.

11. PecoraroF, LuziD. Theintegrationoftheriskmanagementprocesswith thelifecycleofmedicaldevicesoftware. MethodsInfMed. 2014;53(2):92-98.

12. CampbellB, KnoxP. Promiseandplausibility:Healthtechnologyadoption decisionswithlimitedevidence. IntJTechnolAssessHealthCare. 2016;32 (3):122-125.

13. AnandR, GravesSE, deSteigerRN,etal.Whatisthebenefitofintroducing newhipandkneeprostheses? JBoneJointSurgAm. 2011;93(Suppl3):51-54.

14. IbrahimAM, DimickJB.Monitoringmedicaldevices:Missedwarning signswithinexistingdata. JAMA. 2017;318(4):327–328.

15. Kynaston-PearsonF, AshmoreAM, MalakTT,etal.Primaryhipreplacementprosthesesandtheirevidencebase:Systematicreviewofliterature. BMJOnline. 2013;347(2):f6956.

16. MichieS, vanStralenMM, WestR. Thebehaviourchangewheel:Anew methodforcharacterisinganddesigningbehaviourchangeinterventions. ImplementSciIS. 2011;6:42.

17. TongA, FlemmingK, McInnesE, OliverS, CraigJ. Enhancingtransparencyinreportingthesynthesisofqualitativeresearch:ENTREQ. BMCMed ResMethodol. 2012;12:181.

18. StraussA, CorbinJ. Basicsofqualitativeresearch:Techniquesandproceduresfordevelopinggroundedtheory,2nded.ThousandOaks,CA:Sage; 1998.336p.

19. BraunV, ClarkeV. Usingthematicanalysisinpsychology. QualRes Psychol. 2006;3(2):77–101.

20. HsiehHF, ShannonSE.Threeapproachestoqualitativecontentanalysis. QualHealthRes. 2005;15(9):1277–1288.

21. ErlingssonC, BrysiewiczP. Ahands-onguidetodoingcontentanalysis. AfrJEmergMed. 2017;7(3):93-99.

22. ThomasJ, HardenA. Methodsforthethematicsynthesisofqualitative researchinsystematicreviews. BMCMedResMethodol. 2008;8(1):45.

23. FranceE, JepsonR. Whatisqualitativeevidencesynthesisandwhatismetaethnography?[Internet].2015Dec;online.Availablefrom: https://emerge project.org/wp-content/uploads/2016/10/MetaethnographyForNovice.pdf

24. FlemmingK, NoyesJ. Qualitativeevidencesynthesis:Whereareweat? IntJ QualMethods. 2021;20:1609406921993276.

25. BoothA, CarrollC. Systematicsearchingfortheorytoinformsystematic reviews:Isitfeasible?Isitdesirable? HealthInfLibrJ. 2015;32(3):220-235.

26. CooperC, LovellR, HuskK, BoothA, GarsideR. Supplementarysearch methodsweremoreeffectiveandofferedbettervaluethanbibliographic databasesearching:Acasestudyfrompublichealthandenvironmental enhancement. ResSynthMethods. 2018;9(2):195–223.

27. GreenhalghT, PeacockR. Effectivenessandefficiencyofsearchmethodsin systematicreviewsofcomplexevidence:Auditofprimarysources. BMJ. 2005;331(7524):1064-1065.

28. GarsideR. Shouldweappraisethequalityofqualitativeresearchreportsfor systematicreviews,andifso,how? InnovEurJSocSciRes. 2014;27(1): 67–79.

29. BoothA.ESQUIREDayTwo – Selecting,extracting,assessingandsynthesisingdata.ScHARRShortCoursepresentedat:EvidenceSynthesisof QualitativeResearchinEurope(ESQUIRE)2018;2018Sep12;HalifaxHall Hotel&ConferenceCentre,EndcliffeValeRoad,Sheffield,Universityof Sheffield.

30. D’AmourD, Ferrada-VidelaM, SanMartinRodriguezL, BeaulieuMD Theconceptualbasisforinterprofessionalcollaboration:Coreconceptsand theoreticalframeworks. JInterprofCare. 2005;19(Suppl1):116-131.

31. BazeleyP Qualitativedataanalysis:Practicalstrategies.ThousandOaks, CA:Sage;2013.p.476.

32. vandeVenAH.Suggestionsforstudyingstrategyprocess:Aresearchnote. StrategManagJ. 1992;13(S1):169-188.

33. Home-HTAi2022UtrechtAnnualMeeting [Internet].Home.[cited 2023Feb3].Availablefrom: https://www.htai2022.org/htai.eventsair.com .

34. HerderM. Pharmaceuticaldrugsofuncertainvalue,lifecycleregulationat theUSFoodandDrugAdministration,andinstitutionalincumbency. MilbankQ. 2019;97(3):820-857.

35. RathiVK, KrumholzHM, MasoudiFA, RossJS.Postmarketclinical evidenceforhigh-risktherapeuticmedicaldevicesreceivingFoodandDrug Administrationpremarketapprovalin2010and2011. JAMANetwOpen. 2020;3(8):e2014496.

36. DarrowJJ, AvornJ, KesselheimAS.FDAregulationandapprovalof medicaldevices:1976–2020. JAMA. 2021;326(5):420–432.

37. CiprianiA, IoannidisJPA, RothwellPM,etal.Generatingcomparative evidenceonnewdrugsanddevicesafterapproval. Lancet. 2020;395(10228): 998–1010.

38. RathiVK, KrumholzHM, MasoudiFA, RossJS.Characteristicsofclinical studiesconductedoverthetotalproductlifecycleofhigh-risktherapeutic medicaldevicesreceivingFDApremarketapprovalin2010and2011. JAMA. 2015;314(6):604–612.

39. NationalCompetentAuthorityReportWorkingGroup.Medicaldevices: Post-marketsurveillance:NationalCompetentAuthorityReportexchange criteriaandreportform[Internet].InternationalMedicalDeviceRegulators Forum(IMDRF);2017Sep[cited2020Sep28]p.18.ReportNo.:N14 FINAL2017(2ndEdition).Availablefrom: http://www.imdrf.org/docs/ imdrf/final/technical/imdrf-tech-170921-pms-ncar-n14-r2.pdf

40. PetersW, PellerinC, JanneyC. Research:Evaluationoforthopedichip devicerecallsbytheFDAfrom2007to2017. BiomedInstrumTechnol. 2020;54(6):418-426.

41. ArdaughBM, GravesSE, RedbergRF The510(k)ancestryofametal-onmetalhipimplant. NEnglJMed. 2013;368(2):97–100.

42. HeneghanCJ, GoldacreB, OnakpoyaI,etal.Trialsoftransvaginalmesh devicesforpelvicorganprolapse:AsystematicdatabasereviewoftheUS FDAapprovalprocess. BMJOpen. 2017;7(12):e017125.

43. ZargarN, CarrA. Theregulatoryancestralnetworkofsurgicalmeshes. PLoSOne. 2018;13(6):e0197883.

44. PremkumarA, ZhuA, YingX,etal.Theinterconnectedancestralnetwork ofhiparthroplastydeviceapproval. JAmAcadOrthopSurg. 2021;29(4): e1362-e1369.

45. WildC, ErdösJ, ZechmeisterI. Contrastingclinicalevidenceformarket authorisationofcardio-vasculardevicesinEuropeandtheUSA:Asystematicanalysisof10devicesbasedonAustrianpre-reimbursementassessments. BMCCardiovascDisord. 2014;14:154.

46. ErginaPL, CookJA, BlazebyJM,etal.Challengesinevaluatingsurgical innovation. Lancet. 2009;374(9695):1097-1104.

47. MedTechEurope.TheEuropeanmedicaltechnologyindustryinfigures 2022[Internet].2022[cited2023May24].Availablefrom: https:// www.medtecheurope.org/wp-content/uploads/2022/09/the-european-med ical-technology-industry-in-figures-2022.pdf