Performance & Outcome Measurement in Clinical Capacity Building

Performance & Outcome Measurement in Clinical Capacity Building Evalua;on Strategy for Program to Integrate HIV into Primary Care Jamie Orose, MPA 10/17/13

• Advances effec;ve preven;on, care, support, and health equity for people living with or at risk for HIV and hepa;;s C—par;cularly within LGBT and other underserved communi;es—by providing educa;on, capacity building, health services research, and advocacy to organiza;ons, communi;es, and professionals

2

• Understand the background and purpose of the AIDS Educa;on and Training Centers Na;onal Center for HIV Care in Minority Communi;es (AETC NCHCMC) • Describe the targe;ng and evalua;on strategy used to measure program outcomes for the AETC NCHCMC

3

• To increase access to comprehensive, high quality HIV primary care for racial and ethnic minority communi;es severely impacted by HIV/AIDS by building the capacity of non-­‐Ryan White funded community health centers.

4

• Develop a model for integra;ng HIV into primary care • Capacity Building Assistance – Customized Clinical Training – Organiza;onal Capacity Development

5

• Merges – Stages of HIV Clinical Care with – Models of HIV service delivery

• Provided a progressive staging framework for CBA delivery and a stepwise approach to HIV integra;on

6

External HIV Care • Foundation • Provide HIV testing (routine or targeted) but no medical management for patients with positive test results

Introductory HIV Care

Collaborative HIV Care • Stage 1 • Onsite HIV care integrated with primary care through extensive consultation from clinical mentor for initiation of ARV regimen and for management of more complex patients

Supported HIV Care • Stage 2 • Onsite HIV care integrated with primary care, supported by consultation with clinical mentor as needed for management of treatment failure and/or co-morbid conditions complicated by HIV disease and/or treatment

Comprehensive HIV Care Management • Stage 3 • Onsite integrated HIV care with primary care, including management of complex comorbid conditions, PMTCT, and complex OI's with access to consultation from clinical mentors as needed

HIV Primary Care

• Ensure the effec;ve recruitment of CHCs • Measure Improvements in CHCs capacity to integrate HIV services • Evaluate the effec;veness of the model used for HIV care integra;on • Review data to monitor program performance and inform modifica;ons • Disseminate findings to key audiences to inform the effort moving forward 8

AETC NCHCMC Monitoring and Evaluation Framework Application

Process Monitoring

Process Step: Review of applications

Tool(s):

• Application

Assessment

CBA Activities

Anticipated Outcomes

Process Step:

Process Step:

Process Step:

Staging of Health Centers

Performance Improvement and Technical Assistance Coaching

Tool(s):

Tool(s):

Tool(s):

Needs Assessment: • Clinical • Performance Improvement

• HIV in Primary Care Learning Community Assessment • Self-Assessment & Proficiency List

Clinical HIV training and mentoring

Completion of Training and Proficiencies

• End of Cycle Evaluation

• HIV Training Package • HIV Training Plan • Mentor/Coach Notes • Site Visit Reports

Outcome Evaluation

Tool(s): Tool(s):

• Baseline Tiered Performance Indicators

Tool(s):

• Qualitative Site Assessments for Stage Setting

Tool(s):

• Monitoring Framework • ER/PIF • Webinar Evaluations

• End of Cycle Tiered Performance Indicators • End of Cycle Self ReAssessment • Proposed Enhancement & Performance Modifications

• Targe;ng for Need and Impact • Organiza;onal Assessments • Performance Measurement

10

• R&E team u;lized publicly available data sources to iden;fy CHCs in areas: – where impact of increased HIV services will be most impacYul – of high need for HIV services

11

• Relevant summary measures were iden;fied and included HIV data as well as data on broader social determinants of health.

• Summary Measures were ranked to provide a list of states of greatest impact Summary Measures

Data Source

Percent Minority HIV-­‐Posi;ve Popula;on

CDC – HIV Surveillance Data

Unmet Need

Statewide Coordinated Statements of Need

Age (20-­‐49)

US Census Bureau

Sexual Orienta;on

2006 American Community Survey Es;mates

Federal Poverty Level (under 100%)

US Census Bureau

Limited English Proficiency

US Census Bureau

Percent Minority Popula;on

US Census Bureau

Educa;on Level (high school or less)

US Census Bureau

12

• States/Territories were impact of increased HIV services would be greatest: 1. California 2. Illinois 3. Washington, DC 4. Texas 5. New York 6. New Jersey 7. Georgia 8. Arizona 9. Alabama 10. Maryland 13

• Geo-­‐mapping – HIV Prevalence and Incidence – Minority Popula;ons Living with HIV – Unmet Need – County level HIV prevalence

• CHC data on minority pa;ent popula;ons

14

15

! (

! (

! (

! (

! (

! (

! (

! ( ! (

! ( ! (

! (

OK

OK ! (

! (

! (

! (

! (

! (

! (

AR

! ( ! (

! ( ! ( ! ( ! (

! ( ( ! ! (

! (

! (

!! ( ( ( ! ( ! ( !

! ( ! (

! (

! (

! (

! (

! (

! ( (!

! ( ! (

! ( ! ( ! ( ! ( ( ! ( !! !! ( ( (

! ( ! ( ! (

TX

! (

! (

! ( ! ( ! ( ( ! ( ! ! (! (! ! ( ( ! ( ! ( ( ! ! ( ( ! ( ! ! ( ( ! ! (! ( ! ( ( ! ! ((! ( ! (

! (

! ( ! ( ! (

! ( ! ( ! (

! ( ! ( ! (

! (

! ( ! ( ! (! ( ! ! (! ! ( ( ( ( ! ! ! ( ( ! ( ( ! ( ! ( ! ( ( ! ! ! ( ( (( ( ! ( !! ! ! (( ( ! ! (! ( ! ! ( ( ! ( ! (

! ( ! (

! (

! (! (

! ( ! (

! ( ! (

! (

! (

! ( ! (

! (

! ( ! (

! (

! ( ! ![ ( ! ( ( ( (! !! _ ( ! (! ( ! ! ( (

! (

! (

! (

! (

! ( ! (

! ( ! (

! (

! (

! (

! ( ! (

! ( ! (

! ( ! _ [ ! ((

! ( ! (

! (

! (

( ! (!

! (

! (

! ( ! ( ! (

FL

488 - 932 933 - 2474

! (

0

20

40

Counties with higher HIV rates and existing Community Health Centers are targeted for assistance to improve or expand HIV care.

Circled geographic areas provide a target list of centers and primary care providers for mentoring.

! (

! (

! (

! ( ! (

! (

AL

! (

! (

! ( ! (

! (

! (

! (

! (

! (

MS!(

! (

! (

! (

! (

! ( ! (

! (

LA

! (

! ( (!

! ( ! (! (

! (! (

! ( !! ( ( ( ! ! ( ( ! (! (! (! ! ( ! (

! (

! (

!! !( ( ! ( (

! ( ! ! ( (

! ( ! ( ! ( ! ( ! ( ! ( ! ( ! ! ! ( (! ( ( ! ! ( ( ! ( ! ( ! ! ! ( ( ( ! ( ! ( ! ( ! ( ! ( ! ! ! ( ! ( ( ( ! ( ! ( ! ( ! ( ! ! ( ( ! ! ! ( ! ( ( ! ! ( ! ( ( ! ( ( ! ( ( ! ( ( ! ! ! ( ( ! ( ! ! ( ( ! !! ( ! ( (! ( (! ! ! ( ! ( ( ! ( ( ! ! (! ( ( ! ! ( ( ! ! ! ( ! ( ( ( ! ! ( ! ( ! ( ( ! ( ! ( ! ( ! ! ( ( ! (! ( ! ( ( ! (! ! ( ! ! ( ( ! ( ! ( ! ! ! ( ( ! ( ( ! ! ! ( ( ( ! ! ( ! ( ! ( ! ( ! ( ( ! ! ( ! ( ! (! ( ! ( ( ! ! ( ( ! ! ( ( ! ( ! ( ! ( ! ( ( ! ( ! ( ! (! ! ! ( ( ! ! ( ( ( ( ! ! ( ! ! (! ( _ [ ! ! (! ( ( ( ! ( ! ! ( ! ( ! ( ! ( Community Health Centers ! ( ! ( ! ! ! ( ( 2008 County HIV rates ( ! ! (! (! ( ! ! ( (! (! ! ! ( ( ! ! ( ( ( ! (( ! ! (( ( ! ( ! ! ( 12 - 124 ! (! ! ( ( 125 - 269 ! (!

270 - 487 ( ! ( ! (

! (

! (

! (

! (

! ( ! (

! (

! ( ! (

! ! ( ( ! ! ( (

! ( ! (

! (

! (! (

! (

! ( ! (

! ( ( ( ! (! !! (

! ( ! ( ! (

! (

! (! (

! (

! (

! ( ! ! ! (( (

( ! ! (

! (

! (

! (

! ( ! (

! (

! ( ! ( ! (

! (

! (

! ( ! (

! ! ( (

! (

! ! ( ( ! ( ! ( ! (

! (

! (

! (

An Example of Geo-Mapping in the South ! (

! (

OK

! (

! (

! (

! (

! (

! (

80 Miles

! ( ! ( ! (

! ( ! ( ( !! (

• Once CHCs were iden;fied, R&E team verified minority pa;ent popula;on using Uniform Data System (UDS) data

18

• The HIV in Primary Care Learning Community Assessment Tool – Determined the stage of HIV care integra;on that the CHCs fell into – The capacity of the organiza;on to engage in CBA

• Self-­‐Assessments – Individual par;cipant competency in the learning proficiencies of the HIV Training Package

19

Performance Measurement

20

• Limita;ons – Indicators were not inline with the CHCs capacity to provide HIV care and treatment services – Non-­‐Ryan White funded CHCs ojen did not have the capacity to collect and report data

• Solu;ons – Revised performance indicators that were ;ered to beker align with CHCs capacity to provide services and incorporated latest research (IOM, Treatment Cascade) • Tier 1 – Tes;ng, Diagnosis and Linkage to Care • Tier 2 – Reten;on, Monitoring and Adherence, Viral Suppression

– Implemented Evalua;on Capacity Building to assist CHCs in collec;ng, repor;ng and using HIV performance indicators 21

Stages of Engagement in HIV Care

MMWR December 2, 2011 / 60(47);1618-1623

• Tier 1: 1. Number of total pa;ents seen 2. Number of pa;ents offered an HIV test 3. Number of pa;ents who refused an HIV test 4. Number of HIV tests conducted 5. Number of posi;ve HIV tests 6. Number of newly diagnosed who received their test results 7. Number of newly diagnosed linked to a medical visit within three months • Tier 2: 8. Number of unique HIV posi;ve pa;ents 9. Number of HIV posi;ve pa;ents with 2 or more HIV medical visits at least three months apart 10. Number of HIV posi;ve pa;ents with 2 or more CD4 T-­‐cell counts 11. Number of HIV posi;ve pa;ents on ART 12. Number of HIV posi;ve pa;ents that receive 2 or more viral load tests 13. Number of HIV posi;ve pa;ents that have undetectable viral load

23

• Challenging to see measurable changes in pa;ent outcomes due to: – One-­‐year project period – Compe;ng priori;es – Realis;c amount of ;me to engage in CBA ac;vi;es – Exis;ng HIV service infrastructure in CHC communi;es

24

• Core Measures – Added/Expanded HIV clinical services – Enhanced service delivery/QI structure to support HIV integra;on

• Addi;onal Measures – Changes in rate of competency in HIV training package learning proficiencies – Overall project sa;sfac;on

25

• Pa;ent level indicators were not an accurate gauge of program successes.

• The scope of the project was too broad to measure in the beginning stages • It is difficult to isolate outcomes with program interven;ons due to the mul;ple ini;a;ves, quality improvement goals, and funding priori;es available to CHCs. • It can’t be assumed that CHCs have strong capacity to collect and report HIV data.

26

Jamie Orose, MPA Research and Evalua;on Manager HealthHIV 202-­‐507-­‐4743 jamie@healthhiv.org

27