4.4 Kuunika evaluation hypotheses relevant to decentralization: findings from Special Study 3

Constraining Factors

Supporting Factors

• 2019-2020 DIP does not include BNA • Lack of clarity as to why the DHIS2 indicators included in the Excel spreadsheet on BNA (PS2) for 2020-2021 + 2021-2022 DIP have been chosen • Many District programs + their indicators are not (yet) on DHIS2 • It is unclear whether there is any automated link between DHIS2 + PS2 + the BNA • 2021-2022 PS2 spreadsheet shows that only circa 50% of the DHIS2 indicators listed have a national target • Many indicators in the 2020-2021 and 2021-202 DIP state 'no activity logged against this indicator' • Most of the DHIS2 indicators on PS2 are not in the BNA Application, making for limited analysis + read across • The 2021-2022 DIP lists more indicators performing 'equal or below' the national target measures than above • It is not possible to gauge the quality of the data included in the DIP • Issues of disaggregated data or an equity focus appear not to be part of DIP planning (or implementation?) - are these essential levels of complexity too much to ask for at present? • It is unclear whether the BNA has informed inclusion of new DIP activities • The extent to which the DIP has been implemented year on year is unclear, yet tracking that is key • Compared to the 2019-2020 DIP, the subsequent 2 years' Plans indicate more engagement with data + more contextualizing of data to address Zomba priorities • The BNA uses DHIS2 data + informs identification of root causes • The fact that the DIP is now populated using charts based on DHIS2 + other data is positive again, no facility to gauge quality of the data used

4.4 Kuunika evaluation hypotheses relevant to decentralization: findings from Special Study 3

Here we track the entirety of the evaluation work from 2017 with regard to the hypotheses that underpin the Theory of Change and the originally five, now seven, evaluation questions. Five of the 21 hypotheses have particular relevance for this Special Study.

The degree of organisational and political decentralization can affect use of evidence in

decision making (hypothesis 5)

2017 (pre-baseline planning + baseline): weak and conflicting. KAP shows some culture in data use, uptake and challenge (probably reflecting large % of clinicians in the sample), also increasing at higher levels. The qualitative research shows that zonal and district decision makers (both generalists and health sector) are unsure of the overall parameters of decentralization e.g. will the zonal health teams continue to exist? Will the generalists without health expertise and also the many 'new' levels, e.g. village and Ward committees, have more sway in deciding annual foci for health?

2019 (midline): It has been difficult to gauge the extent to which the ongoing decentralisation (a process that has been ongoing for twenty years or more) is influencing the use of evidence in decision-making. The few District level respondents available to the midline did mention increased use of DHIS2 to develop District Implementation Plans; its expansion was considered potentially positive for creation of increasingly detailed District plans. The development of the Local Authority MIS system was described as a potentially valuable tool for decentralised ownership of data. As at the baseline, the midline found very little use of financial or Human Resources for Health data, reflecting the limited nature of fiscal decentralisation to date.

2021 endline and Special Study 3: The hypothesis is assumed to be true, but has never been tested. The Malawi health sector is de jure quite decentralized, de facto not so much. Still to find evidence of greater use of data in DIPs, etc. How to break this circle? Covid-19 monitoring experience has possibly reinforced the sense of a centralized, push system - and perhaps led to greater acceptance of this by some at District level? The pandemic does not appear to have led to more democratization/localization of data capture and use. Districts continue to be out of the loop for use of evidence for decision making - and to an extent in its actual use.

Policy making is often messy and opportunistic - ‘a disorderly set of interconnections

and back and forthness' (links primarily into hypotheses 7 and 17)

Hypothesis 7

2017 (pre-baseline + baseline): There is more than 1 route by which better data influence policy and practice – by feeding into higher level (especially, for HIV, international) policy making and by informing local planning and decision-making.

2019 (ML): the formal/international policy making level dominates in HIV in Malawi. Need to dig more into ‘informal’ and messy decision making in the districts in the special studies? Pretty messy for sure, given decentralisation (or more precisely, its absence, despite 'formal' enactment) and also external partners' influence.

2021 endline and Special Study 3: governance issues are relevant here. Data collection indicated that analysis and modelling of Covid-19 data by Kuunika showcased the power of data amongst central planners and excited a lot of debate.

Hypothesis 17

2017 (pre-baseline + baseline): by limiting access to the improved DHIS2 data, the role of other groups (outside the formal health system) that help accelerate change may be restricted e.g. the District Policy Planning Officers, who don't know if they'll have such access, but seem to be increasingly important pivot points in the decentralised structure.

2019 (midline): the key pivot point here in terms of developing a knowledge culture that supports policy/ practice effectiveness is for individuals and institutions to realise that effective data use and evidence-based policy making are likely to result in enhanced service delivery, better health outcomes and ultimately support clinical health workers, health planners and data entry personnel in their work. In other words, how to change 'institutional norms' and make engagement with data use a truly positive attitudinal and behavioural action.

2021 endline and Special Study 3: has such 'back and forthness' continued under the weight of pandemic response? Has the pandemic speeded up any policy/governance actions and/or shifts? Are there signs of greater/lesser order and coherence in policy development processes, not least given the urgency of dealing with C-19 in 2020?

Hierarchical management of information, organisational silos can limit access to data and its use. Divisions of responsibilities and ‘silos’ can also limit consideration of

evidence (hypothesis 15)

2017 (pre-baseline + baseline): widely/more accessible (within the health system) data on DHIS2 may break down these hierarchies

Depends though on who has user rights and relatively easy access to the system. The 2017 baseline found very little use of DHIS2 at facilities and extensive views that data collection is extractive. There were few obvious feedback loops. There were clear ‘job silos’ - Data Clerks enter data but do not analyse; in-charges (supposedly) analyse but do not enter.

2019 (midline): while the health system is undoubtedly hierarchical, not least in terms of positioning data collection at the lowest level of a steep pyramid, that structure does not result in lack of awareness of the important work done by Data Clerks, or any demonstrated superiority from those with clinical and/or management training and experience. The Kuunika-led cluster meetings and QI sessions which we found at the midline to be well accepted could be interpreted as working by breaking down hierarchies and job silos around data use. By limiting access to the improved DHIS2 data, the role of other groups (outside the formal health system) that help accelerate change may be restricted.

2021 endline and Special Study 3: it appears that Kuunika has not really expanded focus specific to (more? less?) hierarchical management of data. There are indications that Covid-19 response/data collection and use has become a silo and for some District and lower level key

informants has represented challenging demands.

Individuals are empowered through access to data (hypothesis 18)

2017 (pre-baseline + baseline): DHIS2 will be more effective if it is designed with the users’ needs in mind.

2019 (midline): limited evidence of such shifts (empowerment is interpreted in this context as meaning personal job satisfaction, acknowledgement that access to data improves the quality of one's work and opportunities for advancement). No evidence has been found of individuals (as compared to facilities) being awarded DHA Certificates, receiving commendations or other professional satisfaction because they have increased access to data.

There was expressed concern about achieving an appropriate balance between greater access to routine health data and what might be required of individual health workers. Empowerment might come with attached demands for increased data productivity, more analysis, etc., so whose empowerment would it actually be?

'Super data users' very clearly understand and celebrate the opportunities given by greater collection and analysis of routine health data and are empowered personally (this includes the one Data Clerk who is such a user, who has led data quality improvement activities at Health Facility level, with expressed support from all clinical colleagues). So too do policy makers at national level, whose position in the health hierarchy is already empowered.

As at baseline: DHIS2 will be more effective if designed with users' needs in mind. The midline found that Kuunika steps to develop DHIS2 using user-driven design approaches appears to be working to promote interest and anticipation as to what can be done + in connecting with a few self-propelled 'super (data) users'.

2021 endline and Special Study 3: what might have changed as a result of the pandemic - any greater horizontal empowerment? The pandemic has resulted in (temporary?) cessation of WhatsApp groups and review meetings. Access to DHIS2 does not seem to have greatly expanded. How might the slowly expanding group of 'super users' become more integrated into District digital data systems (without placing unreasonable demands on them)?