CMI REPORT
MID-TERM REVIEW OF THE ANGOLA PROGRAMME OF NPA
REPORT R 2006: 1
3: Impact. Measuring how mine action impacts on the communities and societies it concerns is a daunting task. Although NPA probably has done more through the TIAs than any other MA organisation to assess the impact of its mine action work, a number of improvements may still be made to measuring that impact. We provide three suggestions for such improvements. First, to improve the measurement of impact we see the further development of the TIA method along the lines suggested above as essential. Improving the TIA questionnaires, using more actively simple databases for storage retrieval and analysis of the data, training survey staff in social science skills, and including female staff in the survey teams are all likely to help NPA improve the measurement of the impact of its mine action. Second, indicators that are simple enough and will reflect a number of aspects of community well-being should be created and included in the measurement of impact. Some such indicators are already included in the draft new TIA manual. Examples of possible indicators are village production/distribution, village population, ‘months of hunger’, number of schools and health posts, land distribution, and women’s participation in village associations. Finally, once the updated TIA becomes operational with all its indicators, it would be sensible in a few cases to check whether the new indicators give the ‘right’ impression of impact. To do so, the check could benefit from one of the increasing number of qualitative and quantitative methods in existence to measure the impact of various types of development projects. One possibility would be to compare the outcomes of the TIA indicators to the outcome when using a participatory method, such as the ‘Most Significant Change’ method. 4: Output. As noted, NPA defines the output of mine clearance as the mathematical product of quality, cost efficiency, and impact. Putting the three indicators together in the above formula seems excellent for pedagogical purposes, which is how the MA management proposes to use it. It may be worthwhile, however, to consider a further development of the concept, although not necessarily in the day-to-day work of NPA Angola. Below, we consider what developments could be undertaken.
3.2.4Further development of the output formula? With the use of the formula, NPA Angola’s MA programme has produced a visualisation of the cost/impact relationship in mine clearance that goes right to the heart of the present international discourse. Since the output is a product of the three indicators, the formula demonstrates, importantly, that if only one of them is zero or very small the overall output delivered will be low or zero. Furthermore, it indicates that the same level of output may consist of, for example, a low degree of cost efficiency or security (quality) with a high impact, or a low impact with a high degree of efficiency and security. The latter case would indicate that low-impact clearance may be permissible if it can be done inexpensively and safely. The formula can hardly, however, be applied straightforwardly in a quantitative way. First of all, apart from cost efficiency its components cannot be quantified directly. Indirectly, though, it would be possible to quantify the components by using indicators for each of them. For impact, for example, one could use TIA’s grading of high, medium or low – giving each grading the values 2, 1 and 0, as suggested by NPA Angola. But even if indicators with quantitative scores were created for each of the three components in the formula, it would still be difficult to determine what weights to apply to each of the components. For instance, some tricky choices could arise as to whether or to what extent it would be permissible to leave out very heavily mined and high impact areas and go for areas less mined, but where it would be easy to achieve high efficiency and high security. Secondly and importantly, an output indicator has no relevance by itself. For a particular output score to be meaningful, it has to be compared to another output score – in another operation, or in the same operation at another time. The indicator would probably be most relevant for comparison from year to year, and less relevant for comparison between different bases and different organisations, since these bases and organisations would work under different conditions and possibly also use different methods to obtain the output.
15