PEN-Plus Monitoring and Evaluation Toolkit

PEN-PLUS MONITORING AND EVALUATION TOOLKIT

Resources and Guidance for PEN-Plus Implementers, Clinicians, and Healthcare Planners on Local and National Levels

Cover Photo Captions (Clockwise from top left) Emmanuel Joseph Fofanah, a clinical officer at the PEN-Plus clinic in Koidu, Sierra Leone, has undertaken comprehensive training in diagnosing and treating acute noncommunicable diseases. (©Michael Duff/World Health Organization) Namazzi Mercy, a medical intern, takes the blood pressure of 11-year-old Lubwama Jackson, who receives treatment for his sickle cell disease at the PEN-Plus clinic in Nakaseke, Uganda. (©Badru Katumba/World Health Organization) Okoboi George Williams, the microscopist for the PEN-Plus clinic at Atutur General Hospital in Uganda, tests blood samples to screen for sickle cell disease. (©Badru Katumba/World Health Organization) At the PEN-Plus clinic at Atutur General Hospital, Akuyo Regina, a nursing officer, gives encouragement to a patient’s mother. (©Badru Katumba/World Health Organization) Disclaimer This toolkit is a selection of high-value tools developed by the NCDI Poverty Network and included in the PEN-Plus Resource Library, an online repository of tools aimed at helping to support initial implementation of PEN-Plus. All tools included in the toolkit have received input and validation from the PEN-Plus Expert Groups (Diabetes, Sickle Cell Disease, and Cardiac) and are aimed at supporting frontline PEN-Plus providers.

The Resource Library will continue to incorporate edits. These tools are consistent with vetted PEN-Plus training tools; where applicable, however, providers should defer to national treatment guidelines.

Feedback Appreciated We welcome comments on this toolkit and will incorporate them in the next version.

Please send any comments or questions to Collin Whelley at cwhelley@bwh.harvard.edu.

Version 1. June 2025

Contents Introduction 2 Monitoring and Evaluation Roadmap for PEN-Plus

4

PEN-Plus Standards and Protocols

6

Programmatic Standards Clinical Condition Protocols PEN-Plus Data Dictionary 8 Implementing Locally 10 Facility Assessments Training, Supply Chain Management, and Staffing Building Nationally 12 Core Indicators Clinical Priority Indicators National Data System Planning PEN-Plus Clinical Patient Forms 16 Summary Page Intake Form Every Visit Form Reporting Resources and Data Aggregation Support

18

Paper Tally Sheet Digital Tally Sheet Appendix A: PEN-Plus Resource Links

19

Appendix B: Reporting Schedule for 2025–2027

20

PEN-PLUS MONITORING AND EVALUATION TOOLKIT

1

Introduction

Figure 1: PEN-Plus Worldwide (as of June 2025)

C

ountries worldwide are beginning to launch PEN-Plus clinics to provide care for people living with severe noncommunicable diseases in low- and lower-middle-income settings. To ensure these clinics follow best practices, the NCDI Poverty Network has compiled standardized tools, resources, and

learnings to support new PEN-Plus implementing sites not only with their clinical practices but also with their population surveillance and the monitoring and evaluation of their progress and impact.

The Network created this PEN-Plus Monitoring and Evaluation Toolkit to outline essential monitoring and evaluation components of PEN-Plus programs, to provide links to key resources and tools, and to offer guidance on using those resources and tools. The toolkit’s intended audiences are:

2

•

Prospective facility implementers;

•

Implementing facilities and partners;

•

Government monitoring and evaluation teams;

•

Technical partners;

PEN-PLUS MONITORING AND EVALUATION TOOLKIT

•

Data system vendors; and

•

Others looking to support the worldwide expansion of care for people living with severe, chronic NCDs and to build out NCD data systems.

The Network provides implementing partners and health facilities with PEN-Plus tools and standards and helps them implement and improve their monitoring and evaluation systems for surveillance and reporting of severe NCDs. Clinical experts, researchers, implementing partners, and Network members collaborated in creating the PEN-Plus programmatic standards, condition protocols, clinical forms, data elements, and reporting requirements. A subset of these partners formed the PEN-Plus Monitoring and Evaluation Working Group, which also drafted the data element definitions.

More specifically, the Network’s PEN-Plus Monitoring and Evaluation (M&E) Team supports implementers, programs, policymakers, and clinicians with: •

Implementing and adapting patient clinical forms for patient tracking;

•

Reporting outputs and outcomes locally and nationally;

•

Ensuring that clinic staff have a foundational knowledge of data elements and indicators;

•

Improving data quality and use;

•

Integrating digital health systems; and

•

Monitoring ongoing implementation elements, including training, supply chain management, and clinical components.

RESOURCES SURVEILLANCE, MONITORING, EVALUATION, AND REPORTING Collin Whelley, the NCDI Poverty Network’s associate director of data systems and monitoring and evaluation, and Devashri Salvi, monitoring and evaluation lead, serve as the points of contact for the Monitoring and Evaluation (M&E) Team. Email them at cwhelley@bwh.harvard.edu or ddsalvi@bwh.harvard.edu for additional guidance on surveillance, monitoring, evaluation, and reporting for PEN-Plus programs.

PEN-PLUS MONITORING AND EVALUATION TOOLKIT

3

Monitoring and Evaluation Roadmap for PEN-Plus

I

n the absence of PEN-Plus, the collection of severe NCD data varies significantly across systems and localities. The transition from a fragmented approach to the PEN-Plus model—which provides a unified, precise, and comprehensive system that supports meaningful and comparable data—requires change. To

guide this transition, the NCDI Poverty Network has created the Monitoring and Evaluation Roadmap for PEN-Plus, which provides a framework for developing and implementing a standardized data collection and reporting system for facilities implementing PEN-Plus. The roadmap stems from the programmatic standards and clinical condition protocols that are critical to the comprehensive PEN-Plus care package.

The Network has identified and defined key terms and compiled them in the PEN-Plus Data Dictionary, a comprehensive catalog of data elements for each condition. The Data Dictionary has since become a foundational element of the monitoring and evaluation framework, as it helps streamline data collection and reporting and enhances the resulting monitoring and evaluation practices.

From the Data Dictionary, two monitoring and evaluation branches emerged: implementing locally and building nationally. The bottom branch in Figure 2 shows critical steps implemented locally and emphasizes the practical aspects of data collection and use at the local level. The Network’s PEN-Plus Monitoring and

ROADMAP DESIGN CONSIDERATIONS In designing the Monitoring and Evaluation Roadmap for PEN-Plus, Network experts worked with several assumptions: •

Most clinics start with paper-based health information systems in advance of instituting digital health data collection systems.

•

Countries seeking to implement PEN-Plus typically start with one to three facilities before expanding to a national PEN-Plus scale-up.

•

Initial implementation of PEN-Plus tends to aim for a draft set of high-quality indicators that can then feed into national systems, such as DHIS2.

•

Implementation and reporting processes are iterative, with multiple opportunities and avenues for feedback.

•

Data element definitions and indicators represent the baseline PEN-Plus implementation. They are intended to be standardized measures used across countries and sites. As clinical care and data systems improve, however, countries are encouraged to add, adapt, and contextualize beyond the scope of these baseline documents.

4

PEN-PLUS MONITORING AND EVALUATION TOOLKIT

Figure 2: Monitoring and Evaluation Roadmap for PEN-Plus Evaluation Working Group has created forms to aid clinical staff members in tracking the core elements they need while diagnosing and treating PEN-Plus conditions; see page 18.

Once the clinic has begun treating patients and collecting data, it will set up reporting structures. This PEN-Plus Monitoring and Evaluation Toolkit contains reporting tools to support clinics engaged in this process (see the “PENPlus Patient Clinical Forms” section on page 16 and the “Reporting Resources and Data Aggregation Support” section on page 18). After clinic staff members have captured quality data, they can use those data to evaluate clinic and patient outcomes and outputs to support continuous improvement.

In Figure 2, the top branch focuses on a broader, national perspective and illustrates a progression from foundational metrics—the core indicators—to a comprehensive data system for national-level reporting.

Although patient clinical outcomes are prioritized, higher-level processes, systems, and clinical indicators— such as tracking infrastructure, staff, training, supplies, and basic clinical activities—are also essential for charting successful PEN-Plus implementation.

The clinical priority indicators—which track the number of clinical outputs and outcomes needed to generate a baseline understanding of the operations and quality of care—follow the core indicators (see pages 12–14). The standardization of both core and clinical indicators enables national scale-ups of NCD data.

PEN-PLUS MONITORING AND EVALUATION TOOLKIT

5

PEN-Plus Standards and Protocols PROGRAMMATIC STANDARDS The NCDI Poverty Network’s PEN-Plus Clinical Tools and Programmatic Standards handbook defines PEN-Plus clinical care, including the model’s diagnostic, treatment, and patient support services. The Network created the handbook based on years of experience in implementing PEN-Plus in 14 countries and information by various sources, including clinical experts, ministries of health, healthcare providers, clinical expert groups, clinical outcomes data, and PEN-Plus training packages.

The handbook also includes a helpful checklist for tracking implementation.

A complementary document, PEN-

Plus: Essential Programmatic Standards, is included in this toolkit because it: •

Provides a comprehensive vision for PEN-Plus implementation;

•

Serves as a benchmark against which PEN-Plus clinics can be evaluated;

•

The PEN-Plus clinic at Bardiya Hospital in western Nepal provides care for people with severe, chronic noncommunicable diseases. Pooja Baduwal, the head nurse, is in the left foreground. (Photo: Neil Gupta/NCDI Poverty Network)

Can be used as a guide for planning implementation at new PEN-Plus sites and assessing and refining existing facilities; and

•

Outlines expectations for on-site availability and functionality of all necessary components for providing the defined services.

The Essential Programmatic Standards document includes a checklist that serves as a quick guide for determining whether a given PEN-Plus clinic meets programmatic standards and helping to identify gaps for action. Programmatic standards are accompanied by indicators to monitor the effective implementation of care for PEN-Plus conditions, social support, and supply chain management.

6

PEN-PLUS MONITORING AND EVALUATION TOOLKIT

CLINICAL CONDITION PROTOCOLS The PEN-Plus Clinical Tools and Programmatic Standards handbook outlines the step-by-step process for testing, diagnosing, treating, and identifying ongoing needs for patients with PEN-Plus sentinel conditions as well as other severe, chronic NCDs that are included in the PEN-Plus Training package. These trainings and protocols were created in consultation with PEN-Plus Expert Groups, composed of experts and stakeholders from across the Network’s partner countries.

The PEN-Plus model provides training and standardized protocols for: •

Type 1 diabetes;

•

Sickle cell disease;

•

Cardiovascular diseases and other heart conditions;

•

Chronic respiratory diseases;

•

Renal diseases; and

•

Liver diseases.

Facilities with existing treatment protocols can use the PEN-Plus Clinical Tools and Programmatic Standards handbook to evaluate existing clinical workflows and to identify where updates are needed. Facilities without such protocols can apply these protocols as they are or adapt them where needed.

The handbook is intended as a compilation of supplementary tools and teaching materials to the World Health Organization’s guidelines and national guidelines. These tools can also be used to fill in gaps where guideline granularity is lacking for severe NCDs at the rural hospital level.

RESOURCES TRAINING RESOURCES AND PROTOCOLS For more information on training resources and protocols, access the Network’s PEN-Plus Training Folder or email Collin Whelley at cwhelley@bwh.harvard.edu.

PEN-PLUS MONITORING AND EVALUATION TOOLKIT

7

PEN-Plus Data Dictionary

T

he PEN-Plus Data Dictionary catalogs and communicates the structure and content of

data that PEN-Plus clinics collect. The dictionary includes the names, definitions, and attributes of data elements and provides guidance on the accepted meaning and purpose of data elements within a project’s context. The dictionary also supports the process of determining which data elements, across a long list of options, should receive the highest priority for inclusion in the initial paperbased PEN-Plus data systems. These data elements are termed “Phase Zero.”

In addition, the Data Dictionary: •

Identifies data conventions to be

After receiving his rheumatic heart disease diagnosis just a month after the PEN-Plus clinic in Nhamatanda, Mozambique opened, João Mindo was sent to Maputo for the surgical repair of his mitral and tricuspid valves. Now back at home, João—pictured with Dr. Lindolfo dos Santos—receives his ongoing care at the same clinic. (Photo: ©Ivan Simone Congolo/World Health Organization)

used across PEN-Plus projects and provides precise definitions for each element; •

Provides a common, vetted vocabulary to describe and discuss data resources and implementation processes;

•

Supports consistency and data quality by ensuring that the meaning, relevance, and quality of data elements are consistent for all users;

•

Makes data easier to analyze across the program; and

•

Helps build systems and applications of data across the program’s electronic medical records.

The Data Dictionary is organized into multiple sheets: •

Introduction;

•

Sociodemographic (All Patients);

•

General Clinical (All Patients); and

•

Condition-specific sheets, including Diabetes, Respiratory, Cardiovascular, Sickle Cell Disease, Liver, Kidney, Epilepsy, and Other Conditions.

The “Introduction” tab provides an overview of the other sheets and explains the consistent column headers across sheets. Additionally, column “H” of this tab prioritizes data elements for PEN-Plus implementation by

8

PEN-PLUS MONITORING AND EVALUATION TOOLKIT

using the “Phase” column on each sheet. Data elements are prioritized in four phases: Phase 0: Mandatory; Phase I: Basic; Phase II: Intermediate; and Phase III: Advanced. While the goal is to adopt all data elements, this prioritization scheme outlines a progressive path for data element adoption while maintaining standardized measures across clinics and countries.

The “Sociodemographic (All Patients)” sheet includes data elements relating to demographic details, contact information, socioeconomic status, food insecurity issues, and healthcare access. Clinic staff are expected to collect these data elements on all patients.

The “General Clinical (All Patients)” sheet lists diagnosis, history, risk factors, routine vitals, past hospitalizations, and disability-related issues. Again, clinic staff are expected to collect these data elements on all patients.

Finally, a separate sheet focuses on condition-specific data elements covering Diabetes, Respiratory, Cardiovascular, Sickle Cell Disease, Liver, Kidney, Epilepsy, and Other Conditions. Clinic staff are expected to collect data elements for patients with those conditions.

THE FOUR PHASES OF DATA ELEMENTS Phase 0: Mandatory: All PEN-Plus sites must collect these mandatory data elements, as they form the basis of the collectively determined and defined indicators that Network partners found to be necessary for basic care and standardized reporting and data sharing. A document containing only Phase 0 data elements can be found here. Many data elements did not make the Phase 0 data dictionary, as the PEN-Plus team selected only the most critical and fundamental data elements to establish and evaluate care.

Phase I: Basic: Beyond the mandatory Phase 0 data elements, the PEN-Plus team has identified other important basic data elements, which all sites should collect as they are able.

Phase II: Intermediate: The intermediate data elements, which are also important, may require greater levels of clinical sophistication and time availability.

Phase III: Advanced: Although the collection of advanced data elements is not always feasible for early PEN-Plus clinics, those elements may be helpful for clinics to consider as they expand.

PEN-PLUS MONITORING AND EVALUATION TOOLKIT

9

Implementing Locally FACILITY ASSESSMENT The Network’s facility assessment is designed to review the overall structure and function of the midlevel hospitals and clinics implementing PEN-Plus.

Before initiating PEN-Plus services, usually at the beginning of a grant cycle, sites complete a baseline facility assessment to identify gaps in readiness for care in terms of staffing, infrastructure, data systems, and the availability of medicines, diagnostics, and equipment. Later, at the middle and end stages of the grant period, sites complete follow-up facility assessments to record changes and overall implementation progress.

The results of each assessment can help identify gaps in infrastructure and supplies. When used in combination with the programmatic standards checklist at the end of the PEN-Plus Clinical Tools and Programmatic Standards handbook, the assessments can also be tools to support national expansion.

TRAINING PEN-Plus training primarily focuses on doctors, nurses, clinical officers, and other clinical providers at PENPlus clinics. For these activities, the M&E Team supports efforts to track training coverage and assess training impacts, including the knowledge and practice of clinical areas (most notably, type 1 diabetes, sickle cell disease, and rheumatic and congenital heart disease). The quarterly reports include training indicators that track training coverage and help identify training gaps for sites. The Network’s Training Folder offers additional resources.

RESOURCES TRAINING, SUPPLY CHAIN MANAGEMENT, AND STAFFING Some monitoring and evaluation resources for training, supply chain, and staffing are included in previously discussed tools, such as quarterly reports, retrospective record reviews, and facility assessments. Many more such resources are in development. For more information, contact Collin Whelley at cwhelley@bwh.harvard.edu or Devashri Salvi at ddsalvi@bwh.harvard.edu.

10 PEN-PLUS MONITORING AND EVALUATION TOOLKIT

Twenty-two-year-old Rukia Aumo registers her details in the reception area of the PEN-Plus clinic in Atutur, Uganda. Aumo and her two sisters all live with sickle cell disease. (Photo: ©Badru Katumba/World Health Organization)

SUPPLY CHAIN MANAGEMENT Working with sites and countries to ensure access to affordable medications for all PEN-Plus patients is a Network priority. In collaboration with the Supply Chain Team, the M&E Team seeks to support sites in their supply chain management. Efforts to track the cost, type, and quantity of medications and equipment are intended to help both sites and the Network identify gaps, set priorities, and determine areas needing additional support.

STAFFING Staffing is both challenging and important, as clinics can neither implement PEN-Plus programs nor treat patients without staff. That does not mean, however, that every staffing model needs to be identical. Staffing levels and configurations are tracked primarily during facility assessments, which helps the Network identify gaps and opportunities. The Network’s M&E Team is available to work with clinic teams to determine the most impactful staff configurations and to help advocate for any additional needed staff.

PEN-PLUS MONITORING AND EVALUATION TOOLKIT

11

Building Nationally CORE INDICATORS Implementers report core indicators, which are aggregate measures that track essential functions of PEN-Plus clinics, to the Network every quarter. This system was designed to enable national scale-ups. Together, the core indicators represent a standardized and collectively accepted set of indicators that will eventually be included in national health information systems, such as DHIS2.

Core indicators fall into two categories: health systems indicators and clinical indicators. Health system indicators include counts related to training, mentorship, and stockouts.

Clinical indicators are numbers that represent the patient cohort: counts of patients by condition, those ever enrolled into a PEN-Plus

Figure 3: Sample Screenshot of the PEN-Plus Core Indicators Report

clinic, newly enrolled patients, and current patient outcomes, such as those actively receiving care, those lost to follow-up, those who have died, and those who have transferred to a different care facility.

In addition, clinics capture information on patients’ age, gender, and hospitalizations. These indicators are basic yet important markers of clinic functionality and tracking.

Clinical priority indicators, which are described in the next section, are condition-specific process and outcome measures based on the Phase 0 data elements that clinics report every 12 to 18 months.

Each quarter, countries implementing PEN-Plus receive an Excel-based report form, on which they record health system indicators and clinical indicators. The report has several sections: •

Instructions (one tab): Outlines the process for reporting

•

Facility/Clinic (as many tabs as there are implementing facilities in a country): Provides fields for aggregated quantitative data

12 PEN-PLUS MONITORING AND EVALUATION TOOLKIT

•

Supply Chain/Procurement (one tab): Provides fields for information on the cost and quantity of critical drugs, tests, and critical equipment, which are reported every six months

•

Narrative (one tab): Provides fields for qualitative data

•

Indicator Definitions (one tab): Reminds data staff of the definitions of the indicators on which they are reporting

Clinic staff members should access the report form to review templates of the quarter report and indicator lists. The form changes periodically, so staff members should contact their regional advisor or the M&E Team to ensure they have the most recent form. Figure 3 provides a sample screenshot of the PEN-Plus Core Indicators Report from the third quarter of 2024.

In addition to appearing on the quarterly report, core indicators can be found in the PEN-Plus Core + Clinical Priority Indicators document, a comprehensive list of both types of indicators.

CLINICAL PRIORITY INDICATORS Clinical priority indicators, which track the aggregated clinical progress of patients, are organized by: •

Diagnosis/category;

•

Type of indicator (output or outcome);

•

Indicator name;

•

Numerator;

•

Denominator (if applicable);

•

Disaggregation; and

•

Target (if applicable).

RESOURCES BASELINE INDICATORS The M&E Team is available to support teams and countries seeking to expand on the baseline indicators; contact Collin Whelley at cwhelley@bwh.harvard.edu or Devashri Salvi at ddsalvi@bwh. harvard.edu for more information.

PEN-PLUS MONITORING AND EVALUATION TOOLKIT

13

Building Nationally (continued) All PEN-Plus clinics should be able to calculate clinical priority indicators as long as they have been adding Phase 0 data elements from the Data Dictionary to their patient forms.

These indicators are reported less often—only once or twice a year—than the core indicators and are usually coupled with a retrospective record review.

The retrospective record review is a process of examining existing patient charts and records to

Eight-year-old Elisa Edson receives treatment for type 1 diabetes at the PEN-Plus clinic in Nhamatanda, Mozambique. (Photo: ©Ivan Simone Congolo/World Health Organization)

report on indicators that provide insight into patients’ clinical progress, symptom control, socioeconomic status, and barriers to care. The Network uses the RedCap surveys platform to collect these data.

The retrospective record reviews, which take place every 12 to 18 months, capture only information consistent with Phase 0 mandatory data elements and patient clinical forms. Sites with digital health record systems can export data files to supplement this review.

The PEN-Plus Core + Clinical Indicators document serves as the baseline for monitoring and evaluation built on Phase 0 data elements. Sites are encouraged to take the initiative and capture and analyze their data beyond these indicators.

NATIONAL DATA SYSTEM PLANNING The conversion from paper to digital health records can improve the efficiency, accuracy, completeness, and accessibility of patient information. Digital records reduce errors and missing data, streamline data sharing between healthcare providers, and enable faster and better-coordinated care. They also improve patient safety, reduce paperwork, and support better long-term data storage and analysis, leading to more informed

14 PEN-PLUS MONITORING AND EVALUATION TOOLKIT

medical decisions and improved outcomes. In addition, digital systems provide enhanced security for sensitive health data and help healthcare organizations more easily comply with regulations.

At the same time, the move to digital systems is incredibly complex, can be quite costly, and, if not designed carefully and appropriately, can lead to additional problems. To start, this process can be challenging because of the difficulty of selecting an appropriate system, the high initial costs of implementing electronic systems, the need for staff training and dedicated staff, and potential disruptions during the transition.

Additionally, data migration from paper to digital formats can be time-consuming and prone to errors. The need to ensure data privacy and security in digital systems also adds complexity, and some staff who are accustomed to paper-based processes may be resistant. Integrating new systems with existing workflows and maintaining compatibility with other healthcare platforms can further complicate the process.

Finally, the design of digital systems is critically important, especially for the user experience of healthcare workers (digitization should make their lives easier!). Yet digital systems must also be designed with all data users and intended audiences from the start.

RESOURCES DIGITIZATION OF RECORDS AND DHIS2 DESIGN Clinics embarking on the digitization process should reach out to the M&E Team for support in the data collection transfer to a digital medium, whether that entails training, troubleshooting, or meeting facilitation. Digitization project leads should start with a discussion with their regional advisor; the M&E Team can provide support beyond the regional advisor’s scope.

The M&E Team is also available to help with DHIS2 design as governments seek to start tracking NCDs and NCD care nationally.

Email Collin Whelley at cwhelley@bwh.harvard.edu for more information about both digitizing health records and customizing DHIS2 systems.

PEN-PLUS MONITORING AND EVALUATION TOOLKIT

15

PEN-Plus Patient Forms

T

he NCDI Poverty Network used its programmatic standards, condition protocols, and Data Dictionary to inform the creation of PEN-Plus patient clinical forms. These templates provide guidelines on how and when to collect data during condition-specific interactions, including diabetes, sickle cell disease,

cardiovascular conditions, respiratory conditions, chronic liver disease, chronic kidney disease, and epilepsy.

The templates can be used as they are or modified. If clinics choose to modify existing forms, however, they are still expected to incorporate the mandatory Phase 0 data elements.

The form designs assume that clinics can assign unique ID numbers to patients on their first visit or intake at the PEN-Plus clinic. Every template follows a consistent format: Summary Page; Intake Form; and Every Visit Form. Each section is a separate page on which the patient’s name, unique ID number, and enrollment date can be captured on the top line.

SUMMARY PAGE The Summary Page, which serves as the cover sheet of the patient file, provides a snapshot of the most important information that clinicians need to track during every visit and report on each quarter.

The PEN-Plus Team created the Summary Page to solve a problem often observed in many chronic care systems: Charts are often designed with a baseline section collected only at intake or during the first visit and a shorter section for follow-up visits. Yet important information collected at baseline may change over time, and those changes are critical for understanding a patient’s progress. For example, the initial diagnosis may have been refined over time, the patient’s ability to work or attend school may have changed, or the patient may have had a hospital stay.

By tracking current and future visits on the Summary Page, sites can easily identify when a patient has missed an appointment, thus triggering actions to bring patients back to care. This page also enables easy aggregation of patient outcomes. Summary Page information is organized in an easy-to-read table that includes: •

The date of the visit;

•

The current diagnosis, along with any recent changes;

•

Information on any hospitalizations since the last visit;

•

An update on whether the patient has had to miss school or work because of their condition;

•

The patient outcome, which includes a range of options: active in care, lost to follow-up, transferred out (with a notation of where the patient was transferred), died (with a listing of the cause of death), or clinician stopped treatment, and (with a notation of why treatment was stopped);

16 PEN-PLUS MONITORING AND EVALUATION TOOLKIT

•

Notes or reminders for the clinician about the patient’s next visit; and

•

The date of the patient’s next scheduled visit (which supports reminders before the visit and tracks patients who have missed an appointment afterward).

INTAKE FORM The Intake Form, which provides a complete snapshot of the patient’s health status at the outset of care, has two pages. The first is designed to capture essential information during the patient’s initial visit, including: •

Patient demographics;

•

Parent or guardian contact information, if appropriate;

•

Primary and secondary diagnoses;

•

Risk factors and potential complications;

•

Medical history of HIV and tuberculosis; and

•

The impact of the patient’s condition on daily life, such as school or work absenteeism.

The second page of the Intake Form focuses on data related to socioeconomic status, including questions on housing, food insecurity, and healthcare access. The M&E Team recommends that these data be collected at intake and updated during follow-up visits with any major changes.

EVERY VISIT FORM The Every Visit Form provides a comprehensive snapshot of patient progress and treatment efficacy during follow-up visits. The form also documents essential information from each subsequent patient visit. Required fields for all conditions, except for epilepsy, include: •

Vital signs;

•

Symptom control and monitoring;

•

Common laboratory or imaging results;

•

Complications and adverse events;

•

Treatment details, such as medication dosages;

•

Referrals and consultations; and

•

Socioeconomic factors.

PEN-PLUS MONITORING AND EVALUATION TOOLKIT

17

Reporting Resources and Data Aggregation Support PAPER TALLY SHEET The Paper Tally Sheet helps clinic staff aggregate patient data for the core indicators. Each column of the sheet corresponds to a core indicator on the quarterly report. While flipping through patient files, clinic staff can use this sheet to aggregate their patient and clinic outputs and outcomes.

Using this tool, clinic staff can record information from each patient’s file on the tally sheet, including whether the patient is active, has been lost to follow-up, has transferred out, or has died. Once they have reviewed all the patient files, staff members can calculate column totals for all indicators and, where applicable, condition breakdowns. They can then use that information to fill out the quarterly report.

For detailed instructions on using this tool, refer to the Paper Tally Sheet Training Resource. In addition, the PENPlus Team has several recommendations for counting patients at the end of each quarter: •

Assign at least two staff members—including a data clerk and a PEN-Plus clinician—to participate in the tallying;

•

Include dedicated reporting time in workflow scheduling, such as two or three hours each quarter, depending on the clinic size;

•

File all paper clinical forms by Patient ID; and

•

Consider using two colored pens.

DIGITAL TALLY SHEET The Paper Tally Sheet has an electronic counterpart, the Digital Tally Sheet, which automatically generates the totals for each indicator and condition breakdown. Detailed instructions on creating and using that form are available in the Digital Tally Sheet Guide. Although this guide details the creation of a web form in Microsoft Forms, the same process can be followed in Google Forms, Kobo Toolbox, or any other webform application.

RESOURCES TALLYING SYSTEMS The M&E Team is available to support teams in setting up quarterly tallying processes; email Collin Whelley at cwhelley@bwh.harvard.edu or Devashri Salvi at ddsalvi@bwh.harvard.edu for more information.

18 PEN-PLUS MONITORING AND EVALUATION TOOLKIT

Appendix A: Links to Reporting Resources and Data Aggregation Support PEN-Plus Resources GENERAL PEN-PLUS RESOURCES NCDI Poverty Network Website PEN-Plus Resource Library PEN-Plus Training Folder

CLINICAL GUIDANCE PEN-Plus: Essential Programmatic Standards PEN-Plus Clinical Tools and Programmatic Standards

MONITORING, EVALUATION, AND REPORTING RESOURCES PEN-Plus Monitoring and Evaluation Toolkit PEN-Plus Data Dictionary PEN-Plus Patient Clinical Forms PEN-Plus Monitoring and Evaluation Indicators PEN-Plus Quarterly Report Templates and Guidance • Paper Tally Sheet • Paper Tally Sheet Training Resources • Digital Tally Sheet • Digital Tally Sheet Guide

PEN-PLUS MONITORING AND EVALUATION TOOLKIT

19

Appendix B: Reporting Schedule for 2025–2027 REPORTING SCHEDULE SPECIFICS The NCDI Poverty Network has created a cadence of reports to ensure the standardization of clinic protocols: •

Quarterly reports are due 30 days after the end of each quarter.

•

Supply chain components are reported on the Q1 and Q3 reports each year.

•

Retrospective Record reviews are conducted every 12 to 18 months.

•

Facility assessments are conducted at the beginning and end of the grant cycle.

•

PEN-Plus Health Care Provider (HCP) Assessments will take place twice a grant cycle.

•

Financial reporting is due 15 days after the reporting period closes. For example, for the period of February 1 to July 31, the report would be due on August 15.

20 PEN-PLUS MONITORING AND EVALUATION TOOLKIT

MONITORING AND EVALUATION AND NARRATIVE REPORTING PERIODS Reporting Period

Monitoring and Evaluation and Narrative*

Due Date

Q4 2024 (October–December)

Quarterly Indicators and Brief Summary

Q1 2025 (January–March)

Quarterly Indicators, Brief Summary, and Supply Chain Review

Q2 2025 (April–June)

Quarterly Indicators and Brief Summary

Q3 2025 (July–September)

Quarterly Indicators, Brief Summary, and Supply Chain Review

Q4 2025 (October–December)

Quarterly Indicators and Brief Summary

Q1 2026 (January–March)

Quarterly Indicators, Brief Summary, and Supply Chain Review

Q2 2026 (April–June)

Quarterly Indicators, Brief Summary, and Retrospective Record Review

Q3 2026 (July–September)

Quarterly Indicators, Brief Summary, and Supply Chain Review

Q4 2026 (October–December)

Quarterly Indicators and Brief Summary

Q1 2027 (January–March)

Quarterly Indicators, Brief Summary, and Supply Chain Review

Q2 2027 (April–June)

Quarterly Indicators and Brief Summary

Q3 2027 (July–September)

Quarterly Indicators, Brief Summary, and Supply Chain Review

Q4 2027 (October–December)

Quarterly Indicators, Brief Summary, Retrospective Record Review, 31 January 2028 and Facility Assessment

31 January 2025

30 April 2025 30 July 2025 31 October 2025 31 January 2026 30 April 2026 30 July 2026 31 October 2026 31 January 2027 30 April 2027 30 July 2027 31 October 2027

* The technical reporting template combines both Monitoring and Evaluation and Narrative.

PEN-PLUS MONITORING AND EVALUATION TOOLKIT

21

HAVE QUESTIONS? Contact either Collin Whelley, the NCDI Poverty Network’s associate director of data systems and monitoring and evaluation, at cwhelley@bwh.harvard.edu, or Devashri Salvi, monitoring and evaluation lead, at ddsalvi@bwh.harvard.edu.

NCDI POVERTY NETWORK CO-SECRETARIATS Center for Integration Science

Universidade Eduardo Mondlane

in Global Health Equity

Maputo, Mozambique

Brigham and Women’s Hospital Boston, Massachusetts, USA

22 PEN-PLUS MONITORING AND EVALUATION TOOLKIT

www.ncdipoverty.org