Evaluation of the Barents Health Programme

Evaluation of the Barents Health Programme – Project selection and implementation

A report prepared for the Norwegian Ministry of Health by the Fridtjof Nansen Institute Geir Hønneland and Arild Moe

FNI Report 7/2002 ISBN 82-7613-431-9

September 2002

The Fridtjof Nansen Institute

P.O. Box 326, 1326 Lysaker Norway Phone: +47 67 11 19 00, fax: + 47 67 11 19 10 URL: www.fni.no Authors’e-mail: geir.honneland@fni.no arild.moe@fni.no ii

Contents 1

INTRODUCTION ....................................................................................................................................... 1 1.1 1.2 1.3

2

BACKGROUND ........................................................................................................................................... 1 SCOPE AND METHODOLOGY ....................................................................................................................... 2 OUTLINE OF THE REPORT ........................................................................................................................... 4

PROJECT SELECTION AND COMPOSITION OF PROJECT PORTFOLIO.................................. 5 2.1 AREAS OF ACTIVITY AND GENERAL GUIDELINES ........................................................................................ 5 2.2 GEOGRAPHICAL DISTRIBUTION .................................................................................................................. 7 2.3 PROJECT CRITERIA ..................................................................................................................................... 8 2.3.1 Area 1: Infectious disease control................................................................................................... 9 2.3.2 Areas 2 and 3: Reproductive health and child health care, life-style-related health problems .... 10 2.3.3 Area 4 – Improving services for indigenous people ...................................................................... 11 2.3.4 Area 5 – Quality improvement of medical services ....................................................................... 12 2.3.5 Area-specific subgoals – general impression................................................................................ 12 2.4 SIZE OF PROJECTS .................................................................................................................................... 12 2.5 PROJECT CHARACTERISTICS..................................................................................................................... 13 2.6 THE SELECTION PROCESS ......................................................................................................................... 14

3

PROJECT IMPLEMENTATION ........................................................................................................... 17 3.1 GENERAL IMPRESSIONS ........................................................................................................................... 17 3.2 TB CONTROL – THE PROGRAMME’S FLAGSHIP ......................................................................................... 19 3.3 THE ADMINISTRATION OF LESS TANGIBLE PROJECTS................................................................................ 20 3.4 DILEMMAS AND PROBLEMS IN PROJECT IMPLEMENTATION ...................................................................... 22 3.4.1 Size and scope of projects.................................................................................................................. 22 3.4.2 Choice of project partner.................................................................................................................... 22 3.4.3 Co-ordination between projects ......................................................................................................... 24 3.4.4 Budget subsidies ................................................................................................................................. 25 3.5 COST EFFECTIVENESS .............................................................................................................................. 25

4

CONCLUSIONS AND RECOMMENDATIONS................................................................................... 27 4.1 CONCLUSIONS ......................................................................................................................................... 27 4.1.1 Project selection and composition of project portfolio ................................................................. 27 4.1.2 Project implementation ................................................................................................................. 28 4.2 RECOMMENDATIONS ............................................................................................................................... 29

ANNEX 1: EVALUATION FORM (QUESTIONNAIRE).............................................................................. 31 ANNEX 2: LIST OF PERSONS INTERVIEWED DURING VISIT TO ARKHANGELSK AND MURMANSK:..................................................................................................................................................... 35 ANNEX 3: LIST OF PROJECTS IN THE BARENTS HEALTH PROGRAMME FINANCED BY NORWAY 1999-2002.......................................................................................................................................... 36

iii

iv

1

Introduction

1.1

Background

The Barents Euro-Arctic Region (BEAR) was established through the Kirkenes Declaration in 1993, covering co-operation between Norway, Sweden, Finland and Russia at both the regional and national level. At the regional level, BEAR initially included the three northernmost counties of Norway, Norrbotten in Sweden, Lapland in Finland, as well as Murmansk and Arkhangelsk Oblasts and the Republic of Karelia in Russia. BEAR’s geographical scope has subsequently been extended. In 1997, Nenets Autonomous Okrug, located on the territory of Arkhangelsk Oblast, became a member of BEAR in its own right.

The counties of Västerbotten and Oulu (Sweden and Finland, respectively) were included in January 1998. Finally, the Republic of Komi in Russia became a member of the co-operative arrangement as of January 2002. BEAR covers co-operation between the regions of the member states in a variety of functional fields, ranging from industrial co-operation to cultural and educational exchange. So far, the ambition to turn the European Arctic into a functional region with substantial trade and industrial links across the East–West border has yet to be fully achieved. Joint ventures in other areas, such as research and education, have been more successful.1 At the 5th Barents Euro-Arctic Council meeting in Luleå in January 1998, it was decided to raise awareness of health issues under BEAR’s auspices. The Council’s communiqué stated: ‘Taking into consideration the health situation in Northwest Russia several national Governments as well [as] the Regional Council have decided to give priority to health issues.

Special attention should be paid to joint actions that will lead to rapid improvements in the health situation.’2 In accordance with this resolution, the Health Co-operation Programme in the Barents Euro-Arctic Region 1999–2002 (hereafter the Barents Health Programme or simply ‘the programme’), was established. The programme does not create new multilateral structures, it is based on bilateral projects and projects carried out by international organisations. Project co-ordination was supposed to be carried out with the help of an international reference group by means of the exchange of information facilitated by the database Barents Information Service, administered by the Barents Secretariat in Kirkenes.

According to the basic document of the Barents Health Programme, Norway intended to grant 10–15 mill. NOK annually to the programme for the period 1999–2002, Sweden ‘possibly 5 mill. SEK’, and Finland approximately 3–4 mill. FIM. In addition would come contributions 1

For previous evaluations of BEAR projects, see R. Castberg & A. Moe, Evaluering av enkelte prosjekter i Barentsprogrammet, Lysaker: The Fridtjof Nansen Institute, 1998; B. Kjensli and E. Pedersen, I tjeneste for det mellomfolkelige–- evaluering av 19 avsluttede prosjekter i Barentsregionen, Bodø: Nordlandsforskning, 1999; and A.K. Jørgensen & G. Hønneland, Over grensen etter kunnskap? Evaluering av 13 prosjekter innenfor satsningsområdet kompetanse og utdanning finansiert over Barentsprogrammet, Lysaker: The Fridtjof Nansen Institute, 2002.

2 Cited from Health Co-operation Programme in the Barents Euro-Arctic Region 1999-2002, the 6th Barents Euro-Arctic Council, Bodø, 4–5 March 1999, p. 10.

Evaluation of the Barents Health Programme

1

from the Nordic Council of Ministers (2 mill. DKK in 1999) and the WHO.3 On the Norwegian side, the bilateral health projects have been administered by the Ministry of Health; a small secretariat has selected the projects to be financed, with a national programme committee consisting of representatives of the Ministry of Health, Ministry of Foreign Affairs, the Barents Secretariat, North Norwegian health authorities and other national health authorities in an advisory role.

1.2

Scope and methodology

The present report only covers activities under the Barents Health Programme that are financed by Norway. The term ‘Barents Health Programme’ will hence in the following be used to denote the portfolio of projects financed by the Norwegian side, not the entirety of bilateral and multilateral health projects financed by Norway, Sweden, Finland and various international organisations in the BEAR area.4 This evaluation does not pretend to measure in any definite way the end results of the Barents Health Programme, i.e. improvements in the health situation in Northwestern Russia. In many areas this would be premature, due to the relatively short time the programme has been functioning. Measuring health effects would further require a much more comprehensive evaluation, and finally, the evaluators do not have the medical competence. Rather, this evaluation, in accordance with our remit from the Ministry of Health, deals with the programme’s overall profile and its implementation, addressing questions like: • • • •

To what extent does the project profile reflect defined goals?

To what degree has implementation of projects been as intended?

Is there any pattern in the type of problems that have occurred during project implementation?

Can project selection and implementation so far provide lessons that could be of use in the management of future co-operation programmes of this nature, or a possible continuation of the Barents Health Programme?5

Work on this evaluation amounted to two man-months (one for each of the authors of this report). A detailed assessment of individual projects was not foreseen, nor was any investigation of archive material.6

3

Ibid.

The term Barents Health Programme is used to denote the Norwegian project portfolio for the sake of simplicity, avoiding more elaborate labels such as ‘the Norwegian share of the Barents Health Programme’. On the other hand, Norway has been the major contributor of funds to the Barents Health Programme per se.

5 Letter to the FNI from the Ministry of Health, 26 February, 2002.

6 Cf. offer to carry out an evaluation by the Fridtjof Nansen Institute of January, 2002 and acceptance of this offer by the Ministry of Health in a letter dated 26 February 2002.

4

Evaluation of the Barents Health Programme

2

Instead, the evaluation builds on three main sources of information. First, the project managers (mostly Norwegian, but in some cases Russian, Finnish or representatives of international organisations) completed questionnaires (see Annex 1) in which they set out their experiences with the projects. The forms were produced, distributed and collected by the Ministry of Health. The completed forms were thereafter submitted to the authors of this report. Second, personal interviews were conducted with a range of Russian project participants during an eight-day trip to Arkhangelsk and Murmansk in June 2002. The interviews were semi-structured and open, leaving room for the Russians to convey what they felt to be most important for an evaluation of the projects. All interviews were carried out in Russian, which has probably enhanced the validity of the data: avoiding an interpreter reduces the risk of misunderstandings and usually encourages openness and candour on the part of interviewees. A list of persons interviewed is given in Annex 2. Third, one of the authors of the report attended a one-day user conference with the Norwegian projects managers in Tromsø in August 2002, where the preliminary results of the evaluation were presented and discussed. Finally, there has been sporadic e-mail and telephone contact with some of the Norwegian project managers to settle questions that arose during the interviews with their Russian counterparts.

Although the evaluation does not include any in-depth assessment of individual projects, a certain amount of selection had to take place in order to choose interviewees on the Russian side from the approximately sixty projects that have so far been financed by Norway under the Barents Health Programme. A complete list of projects is presented in Annex 3. As will be further elaborated in Chapter 2, the project portfolio consists of a large number of small projects and relatively few large projects. We opted for an emphasis on the larger projects as these represent a larger share of the total project portfolio (from a financial point of view, at least) and are also believed to be more mature in their development (as most of them have received funding several years in a row). In total some 47 mill. NOK has been allocated to the projects. Fourteen projects have so far received a million NOK or more. For purposes of representativeness and practicality, it was decided to conduct interviews in the cities of Arkhangelsk and Murmansk. This left out a few large projects in Karelia and other parts of the Kola Peninsula than Murmansk City. One other large project was also left out since it mainly involved the financing of conferences, which clearly would be of less interest to the evaluation. This left us with nine projects, each with an input of one million NOK or more (see Table 1.1), from which interviewees were sought in Arkhangelsk and Murmansk. The list covers the major projects under the Barents Health Programme in terms of financing and also includes projects from various functional fields under the Programme. Further, several of the projects have been implemented in both Murmansk and Arkhangelsk Oblasts, which gave us the opportunity to compare experiences between the regions.

Evaluation of the Barents Health Programme

3

Table 1.1:

Projects from which Russian interviewees were selected

Number:

Name (abbreviated):

Norwegian partner:

Russian region:

Y9710

TB in Arkhangelsk (including prisons) Used medical equipment to Northwestern Russia

The Norwegian Heart and Lung Association (LHL) The University Hospital in Northern Norway (Tromsø)

Arkhangelsk

Healthy nutrition for women and children in the BR Vaccination in Arkhangelsk

World Health Organization

Y9727 Y9722 Y9720 Y9714 Y9711 Y9716 Y9713 B006

Parents and birth in the Barents region TB control in Arkhangelsk Breast feeding in the Barents region TB control in Murmansk prisons Russian Red Cross against TB

Norwegian Institute of Public Health Norwegian Institute of Public Health Norwegian Institute of Public Health Norwegian People’s Aid The Finnish Lung Health Association Norwegian Red Cross

Arkhangelsk, Murmansk, Karelia and Nenets Arkhangelsk and Murmansk Arkhangelsk

Total sum (in NOK): 8 390 000.4 200 000.3 050 000.2 600 000.-

Arkhangelsk and Murmansk Arkhangelsk

2 469 000.1 800 000.-

Murmansk

1 560 000.-

Murmansk

1 450 000.-

Arkhangelsk and Murmansk

1 000 000.-

It might be argued that the lack of in-depth discussions with project managers and other participants on the Western side (a consequence of the financial scope of the evaluation) represents a methodological weakness. In project investigations based on self-reporting in written form, there is always a danger that project managers will ‘under-report’ difficulties and exaggerate successes, especially when they know that presenting their projects as ‘success stories’ might enhance possibilities for further financing. On the other hand, the chosen methodological approach – with a main emphasis on in-depth interviews with Russian project participants in their own language – is suitable for investigating how the chosen projects function in a Russian context. Added to this, the authors are experienced in evaluations of East–West co-operation in the Barents region and can hence view the Barents Health Programme against a wider background, taking into account experiences from similar programmes in other functional fields.

1.3

Outline of the report

The substantive discussion of the report is divided into two main parts. Chapter 2 discusses the extent to which the project portfolio is in accordance with the defined objectives of the Barents Health Programme. Chapter 3 reviews the lessons gained so far from implementation of the nine major projects in Arkhangelsk and Murmansk Oblasts defined above, based primarily on interviews with Russian project participants. The chapter also brings some reflections on the cost effectiveness of the projects. Conclusions are summed up and recommendations for further work given in Chapter 4.

Evaluation of the Barents Health Programme

4

2

Project selection and composition of project portfolio

The Barents Health Programme has established a rather long list of objectives, activity areas, main guidelines, general project criteria as well as specific project criteria and subgoals.7 In the following, the project portfolio is described and the correspondence between the portfolio and the priorities and considerations in the programme is discussed

2.1

Areas of activity and general guidelines

Based on a general picture of the health-related situation in Northwestern Russia, five fields of activity were singled out in the programme: 1.

2.

3.

4.

5.

Combating new and re-emerging infectious diseases Supporting reproductive health care and child health care Counteracting life-style-related health problems Improving services for indigenous people Quality improvement of medical services.

These five fields have since served as project categories.

Three main guidelines or principles were also established: • • •

Special attention should be paid to joint actions that will lead to rapid improvements in the health situation Within all prioritised areas, special attention should be given to projects focusing on children The health programme must support existing and future bilateral and multilateral health projects under the umbrella of the Barents Euro-Arctic Council

These guidelines are formulated in a general way and must be understood as criteria steering the selection of projects in all the activity areas.

The programme did not spell out any particular distribution of resources between the activity areas. But it would be reasonable to expect substantial efforts in all five areas. However, as depicted in chart 1, this did not turn out to be the case. The first two areas predominate – Combating new and re-emerging infectious diseases, and Supporting reproductive health care 7

Health Co-operation Programme in the Barents Euro-Arctic Region 1999–2002, Sixth Barents Euro-Arctic Council meeting, Bodø, 4–5 March 1999.

Evaluation of the Barents Health Programme

5

and child health care – having received 39 and 36 per cent of total funds respectively. Area 5 – Quality improvement of medical services has received considerably less – 22 per cent. The striking feature of the chart is that the two remaining areas – area 4 – Improving services for indigenous people and area 3 – Counteracting life style related health problems, received very little funding, 2 and 1 per cent respectively.

Chart 1 Budget allocations 5: Quality improvement of medical services 22 % 4: Improving services for indigenous people 2%

1: Combatting new and re-emerging infectious diseases 39 %

3: Counteracting life style related health problems 1%

2: Supporting reproductive health care and child health care 36 %

This disparity may have several explanations. Characteristics of the first two activity areas as opposed to the three others may be important. Areas 3 and 4 may be harder to reconcile with the priority given to ‘joint actions that will lead to rapid improvements in the health situation’.

Especially ‘counteracting life-style-related health problems’ seems to imply a long-term effort. Area 4 – Improving services for indigenous people – also shares some of these characteristics and there is no denying that indigenous people represent a very small share of the total population, warranting perhaps a smaller share of total funding.

But another reason may be the emphasis on supporting existing health projects. One consequence of this would be that the programme would be less open for applications in areas with no or little co-operation in place than in geographical as well as thematic areas with existing partnership patterns to build on. That this is the case is illustrated by the selection of a project that maps the problem (Y9718), as the main project in Area 3. Indeed, there have been few applications in areas 3 and 4.

Again, since the programme is explicitly instructed to build on and complement existing bilateral and multilateral projects, one must assume that if an activity area is well covered by

Evaluation of the Barents Health Programme

6

another programme, it would probably attract less new funding from the Barents Health Programme. This evaluation does not include data on parallel activities in projects outside the Barents Health Programme in any detail, but the various ongoing international and bilateral co-operation projects that are listed in the Programme do not seem to include activities under areas 3 and 5 to any extent.

Thus, what we see is that various priorities (formulated as selected activity areas or general guidelines) may counteract each other, and since the priorities are not attributed specific weight, a large room for interpretation is left to the programme administration and it becomes difficult to judge whether the selection of projects corresponds to the programme priorities.

(The third main goal – emphasis on children – is less problematic in this respect.)

Nevertheless, given the extent of the under-representation of projects in areas 3 and 4 we conclude that it represents a weakness in the programme. This does not mean that the total effect of the programme would have been better if more resources had been channelled into areas 3 and 4. As noted earlier, this evaluation does not pretend to measure health effects, and there may be many good explanations why so little has been done in areas 3 and 4. The upshot is rather that given the formulation of priorities, undue expectations to input in areas 3 and 4 may have been created. This is basically a weakness in the formulation of the programme.

2.2

Geographical distribution

An even regional distribution of projects within the Russian part of the Barents Region is not explicitly stated as a goal in the Programme. However, the programme is presented as an initiative covering the whole area. A certain equality in geographical distribution should therefore be expected, and we will term this an implicit ambition of the programme.

The geographical distribution is presented in chart 2. The biggest recipient of funds is Arkhangelsk – 41 per cent. Murmansk oblast is second with 31 per cent. Karelia has received 4 per cent and Nenets almost nothing. On the other hand, there is a relatively large portion of funding, 24 per cent, that goes to projects that cut across these regions (‘Barents’ in the chart), and Nenets and Karelia have a larger ‘stake’ in these projects than among the projects targeting specific regions.

The geographical distribution must of course be seen in light of the demographics in the various regions.8 When adjusted for population size, Murmansk and Arkhangelsk come out very evenly. Karelia and Nenets have received little, notwithstanding their share in the crossregional projects.

8

Population as of 1 January, 2002. Murmansk oblast: 977 600, Arkhangelsk oblast (excluding Nenets) : 1 384 000, Nenets autonomous okrug: 44 900, Republic of Karelia: 756 400. Source: Goskomstat.

Evaluation of the Barents Health Programme

7

The issue then is the under-representation of Karelia and Nenets. It would seem reasonable to attribute much of this to less developed links. Since the programme is supposed to support already established activities and contacts, this will naturally benefit Arkhangelsk and Murmansk, which have enjoyed much more extended contacts with Norway than Karelia and Nenets. Also, there is an informal ‘sharing of responsibility’ among the Nordic countries in BEAR work, with Finland as the main partner for Karelia.

Chart 2

Geographical distribution of allocations

Barents 24 %

Arkhangelsk 41 % Nenets 0% Karelia 4%

Murmansk 31 %

2.3

Project criteria

But there are additional criteria. There is a list of criteria for the ‘general basis for cooperation’, which needs to be understood as a specification of priorities and requirements to project development and implementation. It includes: • • • •

Broad and regular exchange of information Competence building Co-operation between international, national, regional and local authorities Support to the work of non-governmental organisations

The first two criteria are of a very general nature, and it is easy to see that they are reflected in most of the projects. Co-operation is also involved in most projects. However, if the third criterion is interpreted somewhat more demandingly as co-ordination, the picture is not so Evaluation of the Barents Health Programme

8

clear. As mentioned in Chapter 1, the programme intended to secure co-ordination at the international (Nordic) level by the establishment of an international reference group and a database for projects.9 The reference group, which also has Russian representation, meets fairly frequently. It had its fifth meeting in May 2002. It discusses overall priorities, but, according to the Norwegian secretariat, there is no standard or comprehensive format for the presentation of projects. Thus, the reference group is not an instrument for project coordination to any significant extent. The database for registration of Nordic health projects in the Barents Region is operated by the Barents secretariat in Kirkenes and financed by Norway.10 The Programme stated that it would file information on Russian-Finnish as well as Russian-Swedish projects, in addition to Russian-Norwegian projects. According to the programme secretariat, the need for the database has been brought up several times at the Nordic level, as well as in the reference group, and there seems to be general agreement on its potential benefits. Nevertheless, only a few of the Swedish and Finnish projects have been entered; Finnish and Swedish project operators tend unfortunately not to submit information.

(All Norwegian projects are registered in the database). Thus it is difficult to find an updated overview of all projects, which naturally increases the risk of overlap between projects and the suboptimal use of resources.

The fourth criterion – support to the work of NGOs - is of a more specific character. Russian NGOs have been involved in several projects, and the composition of the project portfolio may be said to correspond reasonably to the criterion. The Russian Red Cross has been the major player here– both in Arkhangelsk and Murmansk. Whether Russian NGOs could have been more extensively involved will be discussed under project implementation.

In addition to the general criteria, special criteria or subgoals for the five fields of activity were formulated. Since these criteria were not attributed weight either, for evaluation purposes they can only be applied in a relatively crude manner, i.e. as a checklist for the content of the project portfolio in each activity area: Have some criteria/subgoals not been met (at all)? How central are the criteria in the definition of the projects?

With this in mind we will briefly assess the project portfolio, on the basis of the project descriptions, in the various activity areas:

2.3.1 Area 1: Infectious disease control There are altogether 16 projects in this activity area, with a combined budget of some 18 mill.

NOK. 9

As noted earlier, this evaluation does not cover the totality of Nordic efforts under the Barents Health Programme, only the Norwegian part. The following remarks on the co-ordination between the Norwegian and other Nordic efforts are included because they have direct relevance for the effectiveness of Norwegian projects.

10 http://www.barents.no/health/engelsk/index.html Evaluation of the Barents Health Programme

9

Subgoal 1 – Regular meetings with the infectious disease control authorities of the relevant countries and with the participation of the relevant regions seems to be a central aspect in several projects such as, for instance, the main immunization project Y9720 – Organisation of epidemic control and immunisation in Arkhangelsk region, and the main tuberculosis projects Y9710 – TB Control in Arkhangelsk, and Y9720 – Tuberculosis control in Murmansk prisons.

It is harder to identify subgoal 2 – Regular exchange of updated statistics concerning cases of infectious disease in the relevant countries, both nationally and at regional level in the projects. No project specifically aimed at exchange of statistics has been launched, but data from projects dealing with epidemic diseases have been collected and published by the Norwegian Institute of Public Health in co-operation with Russian and Baltic health authorities in a journal and web site financed by The Nordic Council of Ministers.11 Subgoal 3 – Development of alert systems for infectious diseases has been central in the TB area, but to a lesser extent with regard to HIV/AIDS. General conclusion – the subgoals can be identified but do not constitute central aspects of the projects selected in area 1.

2.3.2 Areas 2 and 3: Reproductive health and child health care, life-stylerelated health problems Area 2 contains some 19 projects with a combined budget of 16.5 mill NOK. In area 3 there are just two projects, with a total budget of 662 000 NOK. Subgoal 1 – Strengthen primary health care as a basis for services that reach out to the local community: Several projects are directed directly towards this goal (e.g. Y9714 – Safe Motherhood, and Y9717 – Dental health co-operation between Apatity and Finnmark County), others address the issue through organisation of, and participation in conferences.

Subgoal 2 - Health promotion directed towards target groups is reflected in e.g. projects targeted at disabled children (B107 – Children’s health in the Barents Region Conference) and asthma patients (B101 – Asthma problems under Control). Projects aimed at infants and mothers also belong in this group.

Subgoal 3 – Support prevention of unwanted pregnancies and sexually transmitted diseases.

Prevention of unwanted pregnancies is necessarily a by-product of projects directed at 11

http://www.epinorth.org/english/epi_data.html. Data are also contributed by other Nordic countries.

Evaluation of the Barents Health Programme

10

prevention of HIV/AIDS in Area 1, but no project has unwanted pregnancies as its main concern.

Subgoal 4 – Support the children vaccination programmes against infectious diseases like tuberculosis, poliomyelitis, diphtheria, pertussis/whooping cough, measles is not an explicit element in any of the projects.

Subgoal 5 – Support care and habilitation of mentally retarded and disabled children is the main content of projects Y9715 – Development programme for Monchegorsk home for children with disabilities, and Y0379 – Activity and Training Centre in Kirovsk.

Subgoal 6 – Increase knowledge and support prevention of premature death, e.g. accidents, suicide is not the main focus of any project in this area, but may be an aspect of Y9723 – ‘… a full and decent life’, which deals with competence building in psychiatry, and B118 – Cross cultural alcohol and drug prevention – family intervention initiatives. It is definitely central to project B112 – Suicide intervention training programme in Arkhangelsk, Area 5.

Overall, most of, but not all subgoals are reflected in the projects in area 2 and 3.

2.3.3 Area 4 – Improving services for indigenous people Area 4 includes 2 projects with a combined budget of 750 000 NOK. Subgoal 1 – Increase the knowledge and the understanding of the specific health problems among the indigenous people in the region, is a central part of project Y0383 – Alcohol and drug abuse programme for indigenous people.

Subgoal 2 – Strengthen primary health services that address this group’s special needs, is at the core of project Y9719 – Medical development in Lovozero and is also part of Y0383.

Subgoal 3 – Encourage health projects that involve the indigenous people themselves in improving their health situation has to some extent been part of project Y0383.

In this activity area the subgoals are quite close to the core of the projects.

Evaluation of the Barents Health Programme

11

2.3.4 Area 5 – Quality improvement of medical services Area 5 includes 19 projects, totalling 10.5 mill NOK. Subgoal 1 – Health systems development with a focus on primary health care and by means of training personell, improving financing and management, and quality assurance is central to a number of projects (Y0372 – Quality development of diagnostic methods in histopathology service in NW Russia , Y0375 – Primary Health Care Project in Arkhangelsk, Y0376 – Further development of heart surgery and circulatory lab, B109 - Further development of heart surgery and circulatory lab., B111 – Quality improvement of psychiatric services in Arkhangelsk Regional Hospital).

Subgoal 2 – More collaboration between the health institutions in the region, is reflected in projects from all activity areas, but notably Y0374 – Co-operation within the nursery sector in Arkhangelsk and Tromsø’s regional hospitals and Y9721 – Four different projects under the University of Tromsø.

Subgoal 3 – Develop further co-operation in the field of telemedicine is part of project Y0372 and also of projects in other activity areas.

All in all, the subgoals are reflected in many projects in this area of activity. It should be noted though, that the most costly projects in this area (Y9727 – Used medical equipment to NW Russia), is not directly part of any of the subgoals.

2.3.5 Area-specific subgoals – general impression Almost all of the subgoals can be identified in projects. However, the subgoals are very often not at the core of the projects and it is not clear what role they have played in the selection of projects. (For further comments, see Chapter 4.)

2.4

Size of projects

As depicted in chart 3, the project portfolio is dominated by small projects. Only one project is larger than eight mill. NOK, one is 4.2 mill., one is 3.05 mill., eleven lie between one and three million, and the remaining 44 have less than a million, most of them less than 500,000 NOK.

Evaluation of the Barents Health Programme

12

Chart 3 Projects - by size (NOK) 9 000 000 8 000 000 7 000 000 6 000 000 5 000 000 4 000 000 3 000 000 2 000 000 1 000 000 0

However, some projects are closely intertwined. In particular, there are some smaller projects that can be regarded as add-ons to larger projects in the TB sector.

2.5

Project characteristics

The project questionnaires, which are the main source of data for this part of the evaluation, do not present an accurate breakdown of project costs by use. However, on the basis of the budgets presented and information on the content of the projects, we have arrived at a makeshift picture of resource allocation, see chart 4.

It is quite evident that the lion’s share of the money has been used on the training of Russian personnel and exchanges of Russian and Norwegian health care workers, both a central element of the ‘general basis for the co-operation’. The second largest category is purchase and transportation of medicine and equipment. This category is not mentioned directly in the general and specific project criteria. The third largest category – participation at professional conferences – is explicitly part of the general basis for the co-operation. Again, the general impression is one of considerable flexibility in the execution of the programme.

Evaluation of the Barents Health Programme

13

Chart 4 Allocation of resources according to purpose Other Mapping of problems, preparatory work

Construction

Medical equipment, medicine

Training, exchange

Professional conferences Information work

2.6

The selection process

Project selection is administered by a secretariat, comprising two civil servants in the Ministry of Health. A programme committee, with members from the Norwegian health authorities and other implicated government agencies take part in the selection process. Officially, the committee is only an advisory organ to the secretariat, which makes the final decisions. The selection process is described thus: 1 2 3

4 5

Upon expiry of an application deadline, applications for project support are collected and sent out to the program committee.

A crude sorting of applications is undertaken by the committee members After a meeting in the committee, promising new projects are sent out to appropriate bodies (typically the Norwegian Board of Health [Helsetilsynet] or the Norwegian Institute for Public Health [Folkehelseinstituttet] for comments.

A new meeting in the program committee discusses the proposals in more detail, in light of the received comments.

The secretariat decides on the final selection.

Altogether, some 220 applications have been received, including applications for extensions – ’repeaters’. Approximately 60 projects have been carried out. Eighty-nine project applications have been declined. According to the secretariat many of the rejected projects would have

Evaluation of the Barents Health Programme

14

been approved had more funds been available. Funding, not quality or relevance, is seen as the major bottleneck. Almost all approved projects have had their budgets reduced. Among the rejected project proposals research projects stand out as a group the secretariat feels lies beyond the brief of the programme.

In addition to the formal application process for new projects, the programme has ‘taken over’ activities previously funded from other sources, e.g. the Ministry of Foreign Affairs. There has also been a general wish to fund WHO projects through the Barents Health Programme, the support of UN organisations being a goal in itself for Norway. The secretariat has not only passively received applications, it has actively encouraged project applications from institutions deemed competent to deal with different aspects of the programme.

All in all, the impression is that the selection process is carried out by a small group of civil servants that is quite autonomous and demonstrates the use of a considerable amount of discretion in the execution of the programme.

Evaluation of the Barents Health Programme

15

Evaluation of the Barents Health Programme

16

3

Project implementation

This chapter brings together observations on the implementation of projects under the Barents Health Programme. As noted in Chapter 1, the intention of this evaluation is not to conduct a detailed assessment of the individual projects. Rather, it focuses on the general experiences gained so far, as reflected in the questionnaires completed by the (mostly Norwegian) project managers, and the interviews with participants in nine major projects on the Russian side.

In the first section, some general impressions of project implementation are presented, based primarily on the questionnaires and supplemented by interview data. The next sections delves somewhat deeper into the material gained from the interviews with the Russian project managers. Among these, four projects are related to TB control in Arkhangelsk and Murmansk (including the by far largest single project under the Barents Health Programme).

One project involves the purchase of used medical equipment; one is a vaccination project and the three remaining target awareness raising and behavioural change on the Russian side in the fields areas of breast-feeding, nutrition and child-birth issues. The emphasis on TB control in the Barents Health Programme warrants a separate discussion of these projects. The projects involving vaccination and purchase of used medical equipment are fairly straightforward and concrete, and their implementation is covered in the first section below. A separate section is devoted to the more complex and less tangible projects involving awareness raising and encouragement of behavioural change among Russians. The section rounds off with some remarks on dilemmas met in project implementation and a brief discussion of the cost effectiveness of the projects.

3.1

General impressions

The general picture evolving from both the questionnaires and our interviews in Russia, is one of a largely successful implementation of the projects under the Barents Health Programme.

In the vast majority of the projects, the established goals are either achieved or things are proceeding according to plan. The main obstacles met are associated with Russian bureaucratic procedures, primarily in the area of customs, that lie outside the ability of either Norwegian or Russian project participants to influence. In cases where such obstacles were met, the project workers seem to have learnt from the experience and adjusted their approach on subsequent occasions so as to reduce the loss of time and money. The best example is the large project on purchase of used medical equipment in Norway for distribution in Northwestern Russia (project Y9727; the second largest projects under the Barents Health Programme in financial terms). After some negative episodes in similar ventures in the 1990s, the current project is depicted by the Russians involved as a model project in all aspects, from the manner in which the customs barrier was dealt with to the way the equipment is modified

Evaluation of the Barents Health Programme

17

to function in a Russian context. As put by a Russian official: ‘this is administration at a very high level’.

In addition to the largely successful implementation of the projects in the Barents Health Programme, much learning has taken place on the Russian side, at least in comparison with other types of East–West co-operation in the Barents region. In the case of the vaccination projects (Y9720 is the main project) it seems that they contributed to a change of approach to Rubella vaccination. The Russian side had earlier given priority to Rubella vaccination of children of pre-school age where the occurrence of Rubella is highest, whereas the Norwegian partner – the Norwegian Institute of Public Health - argued that priority should be given to teenage girls, who had not contracted Rubella as children and thus had become immune. The Norwegian view is that Rubella among children is not dangerous and that the important thing is to prevent it during pregnancy, since it is a serious threat to the fetus. The Norwegian side felt that the Russian side were receptive to their arguments. A similar change has been seen with regard to vaccination against hepatitis. In this case the Norwegian side gave priority to newborn, arguing that if infants acquire the illness, e.g. during birth or breast-feeding, it is difficult to rid themselves of it, and that it entails serious long-term risks, including cancer.

The Russian side gave priority to teenagers where the occurrence was highest, from e.g. shared needles among drug addicts. But according to the Norwegians, hepatitis acquired at that age does not constitute a grave risk and may even pass almost without symptoms. Also in this case the Russians were receptive to change.

Further, many projects have stimulated contacts between Russian agencies and organisations that would otherwise not have co-operated themselves in this type of endeavour. Both the learning and the new contacts between rigorously hierarchical Russian bureaucracies, as well as inclusion of NGOs, are most evident in the TB projects and will be discussed in more detail below.

Judged on the basis of the project questionnaires, only one project under the Barents Health Programme appears to be a failure thus far. It is project YO380 on the preventive work against drug abuse and the spread of HIV and hepatitis in Northwest Russian schools. The Norwegian project management has yet to receive the necessary permits from Russian federal authorities to carry out the programme. On the one hand, it seems naïve to think that such permits would not be required or that they could be acquired in a relatively short time at the regional level, a view shared by our Russian interviewees to whom we mentioned this project.12 On the other hand, the Norwegian project management was met by a request from a federal civil servant to organise and finance a visit to Norway to fascilitate co-operation. The project manager found this proposal unacceptable. Efforts are now made to implement the project through the establishment of a local institution instead. (Such an institution does not require a federal permit to work in schools, and the local authorities are supportive). Further, 12

We did not talk with the Russian participants of this project itself as it is a fairly small project and hence not among those we singled out for further study.

Evaluation of the Barents Health Programme

18

it might be unfair to call it ‘the failure’ of the Barents Health Programme; given the uncertainties connected with self-reporting by project managers mentioned in Chapter 1. As will follow from section 3.3, our interviews disclosed the presence of quite substantial problems in at least two of the nine projects selected for interviews; none of them were mentioned in the reports by the Western project managers.

3.2

TB control – the programme’s flagship

The four TB-oriented projects selected for further investigation are, from a financial point of view (see Chapter 2), the flagship projects of the Barents Health Programme. The depiction also fits as it happens as far as implementation and results are concerned. First, the Russian project participants express in general a high level of satisfaction with the administration of these large projects, particularly those in Arkhangelsk (reservations expressed by participants in Murmansk are given at the end of this section). The head of the health administration of Arkhangelsk Oblast characterised the TB project13 as ‘the most successful [of the joint projects with Norway]’ in our interview with him, and went on: ‘the TB project is the ideal, the perfect project – it has superb planning, implementation, monitoring and transparency.’

Second, the TB projects have spurred a quite extraordinary extent of learning on the Russian side, described by several prominent Russian project participants as ‘a revolution’. The present report, prepared by two political scientists, will not delve into the medical aspects of this ‘revolution’. Suffice it to say that, traditionally, the Russians have tended to be concerned more with diagnostics and less with prevention and treatment in their dealings with TB. Unlike many other fields of East–West interaction, where Westerners attempt to force ‘competence’ on the Russians in areas where they are fully competent already, the Russians here let themselves be convinced by the methods prescribed by the Norwegians (in Arkhangelsk) and Finns (in Murmansk). The general theme of our interviews in both regions was that the Finns and Norwegians tried to convince the Russians to change their practice without forcing their own approaches on them. After some time, the Russians were convinced. Hence, the TB projects are clearly sustainable. As expressed by the head of the health administration of Arkhangelsk Oblast: ‘If the TB project had been discontinued today, there would still be a substantial gain from it for our region. You can give a starved person a crust of bread or give him a fishing rod and teach him to fish. This project avoids a “client attitude”.’

Third, the TB projects emerge as ideals as far as co-ordination among various agencies and inclusion of NGOs on the Russian side are concerned. One of the four TB projects (project B006), managed by the Norwegian Red Cross and implemented by the regional committees of the Russian Red Cross in Northwestern Russia, can be regarded as a supplement to the 13

Actually, there are at least three of them, but many Russians seem to view various projects within one field as one large project.

Evaluation of the Barents Health Programme

19

larger and more comprehensive TB projects. The Red Cross project involves direct work with TB patients discharged from hospital, particularly the homeless and alcoholics and is mainly directed at securing healthy nutrition and proper medication. Red Cross personnel see to it that the sick take their medication before they give them food and supplies of hygienic articles to avoid any interruption in the treatment. Co-operation between the Red Cross and public authorities is reportedly very good in both Arkhangelsk and Murmansk. It is quite unusual to hear a leading Russian civil servant say anything like the following about the work of an NGO: ‘I am simply enthused with what the Red Cross has achieved. It is an enormous help for us. [...] We have very close contact with the Red Cross.’

The projects under the Barents Health Programme have ‘forced’ civilian health authorities (subordinate to the regional administration, i.e. the executive branch of regional government) and prison authorities (a federal agency located in the region, subordinate to the Ministry of Justice and hence not under the authority of the governors) to co-operate. In both Arkhangelsk and Murmansk, this partnership is of relatively recent date (starting around the turn of the millennium) and is the direct result of the joint projects under the Barents Health Programme.

It might be argued that partnerships such as these are a good thing for development of a democratic society, which is a prime concern of Norwegian policies towards Russia. In this case, co-operation is also a simple necessity to combat the spread of TB. The same thing can be said about the involvement of the Red Cross in the TB projects. Not only is it a good thing, from a Norwegian point of view, for NGOs to be included in public work, the Red Cross also fills a void in the treatment of people with TB in Northwestern Russia. Without the efforts of NGOs, the present system would not be capable of ensuring that discharged patients continued to take their medication.

The only complaints about the TB projects in our interviews came from prison authorities in Murmansk. While generally pleased with the project, they commented that the Finnish project management14 showed ‘no flexibility’ in project implementation, that it was a standard project developed to be implemented in various settings and that the project management was not open to common sense in situations when it might be necessary. The Russians also argue that it would have been better with smaller, one-year projects, with clearly defined and measurable goals, instead of three-year programmes.

3.3

The administration of less tangible projects

Of the nine projects in our sample, one stood out as more problematic than the others. It was project Y9722 on healthy nutrition for women and children in the Barents region, managed from WHO’s regional office in Copenhagen. For one thing, there seems to be a divergence in how the goals of the project are perceived by the WHO project management and the Russian 14

The project in the Murmansk prisons has been operated by the Finnish Lung Health Organisation – FILHA.

Evaluation of the Barents Health Programme

20

project participants. The WHO defines the main objective of the project as ‘to ensure access to safe, nutritious variety of food by developing a food and nutrition policy and to provide a nutrition education strategy for women and their children’.15 Our interviewees on the Russian side said mostly that the project was a research project. The former Arkhangelsk project manager16 depicted it as a pure research project, while the Murmansk manager emphasised its potential practical implications (meaning that public authorities could use the research results to design fresh nutrition policies), not as an integral part of the project itself. Second, the project ran into serious organisational problems in Arkhangelsk, where the former project manager allegedly failed to inform her superiors sufficiently about the project and was hence removed from it. Perhaps as a result of these problems, there seemed to be considerable discontent with the project in the regional administration in Arkhangelsk Oblast. Various regional administration officials commented on the project in the following ways: ‘let’s be honest, the results are not as good as they should have been’; ‘the project is way too massive, gigantic, and probably also came too early; one has to take the existing situation as a point of departure’.

Another instance of possible internal conflict on the Russian side is found in project Y9716 on breast-feeding in the Barents region. A peculiar situation arose when we showed up for an interview with one of the two persons indicated in the questionnaire as major participants on the Russian side. The person represented an information agency which, according to the questionnaire, contributed to raising public awareness about breast-feeding. For quite a while, she politely but firmly tried to convince us that we had arrived at the wrong address – she knew nothing about the Barents Health Programme or projects on breast-feeding! Then she remembered: ‘yes, there was some fuss a couple of years ago’. She went on to explain that they had wanted to be part of the project and in fact participated at a preparatory meeting.

However, they were never invited to actually join the project itself because, she explained, the main Russian institution in the project, a maternity clinic, did not want interference from ‘non-experts’. She argued convincingly that the information agency could have contributed professional information services thereby reaching more people. The main Russian project member was not available for interviews, but a Norwegian project participant claimed that the information agency was not included as a paid associate due to budgetary constraints, not as a result of any pressure from the maternity clinic.17 This said, the breast-feeding projects, which also include the Safe motherhood project (Y9714) have clearly brought results; several of our interviewees used the word ‘revolution’ to describe the changes that have taken place in recent years regarding breast-feeding in Northwestern Russia. The WHO project on healthy nutrition for women and children has allegedly also had results, at least in Murmansk. According to its project manager there, 15

Questionnaire for project Y9722, p. 2; on file at the Norwegian Ministry of Health.

She was recently replaced and knows the project better than the new project manager, her boss.

17 The Norwegian project manager also claims that employees at the information agency worked on the project on a voluntary basis.

16

Evaluation of the Barents Health Programme

21

regional authorities have actively used the results of the project in amending their nutrition policy. The organisational problems of the two projects point at dilemmas in project implementation that will be further discussed in the next section.

3.4

Dilemmas and problems in project implementation

3.4.1 Size and scope of projects While we have no data to indicate that smaller projects are more successfully implemented than large ones –the only obvious ‘failure’ under the programme being a rather small one, while the gigantic TB project in Arkhangelsk is a huge success – it is clearly a dilemma whether one should opt for limited, clearly defined projects aimed at solving concrete problems (e.g. lack of vaccines, or spread of a communicable disease) or larger ‘programmelike’ structures with multiple and sometimes less tangible objectives. The impression we gained from interviews in Arkhangelsk and Murmansk is that the Russians prefer the former variant. We registered considerable exasperation with the complex structure of the WHO project, apart from the confusion about the project’s actual goals. Even in the successful TB projects, project participants say that shorter and more concrete projects are preferable to the present programme structures. An important reason is that shorter, and more focussed projects are easier to monitor. Experienced Russian project participants know they have to demonstrate results to secure funding for future projects.

3.4.2 Choice of project partner A more immediate dilemma concerns how to select project participants on the Russian side. It is our distinct impression from other areas of co-operation between Russia and the West that once contact has been established between a Russian and a Western institution, the Russian party does not expect to have to ‘share’ its Western partner with other Russians. In addition come the strict vertical divisions in the Russian bureaucracy and the Russian tradition for ‘expert rule’ (i.e. engineers run factories and medical doctors hospitals; all-rounders, i.e. ‘nonexperts’, are not expected to interfere).18 A classical example of what often happens when the Western partner tries to include a second Russian structure was the story recounted in the preceding section about the information agency, that was ‘forced out’ by the main Russian partner (that is if we take the information agency’s version as ‘true’).19 In addition here comes the fact that the third structure was a typical ‘non-expert’ organisation which can easily have been viewed as superfluous by the Russian ‘expert’ institution. Rather than lecturing about what should have been done by the Norwegian project manager in this case, we want to point to the dilemma found in situations 18

In fact, we met this attitude during our interviews for this evaluation. During one interview an Arkhangelsk physician complained that ‘it is so difficult for me to talk about these things with non-mediki’.

19 As mentioned above, the main Russian participant was not available for interview for this evaluation.

Evaluation of the Barents Health Programme

22

like this. It may ‘insult’ the Russian project manager to insist on the inclusion of an additional ‘non-expert’ body. On the other hand, it might on occasion be necessary to ‘force through’ such measures in order to further ‘democratisation’ aims underlying Norwegian efforts in Northwestern Russia, notably the building of a ‘civil society’. There is reason to believe that the Northwest Russian civil society is now strong enough to be included to a larger extent in joint projects with Norwegian institutions than has been the case so far. Such ‘forced’ cooperation, between public authorities and NGOs or between governmental agencies belonging to different vertical structures, is often initiated through international partnerships. They are usually highly unpopular at first, but in most cases increase the capabilities of public authorities to solve problems in society.20 Indeed, the breast-feeding projects are an example of this too, since they included groups of voluntary mothers teaching others about breastfeeding. This was controversial to start with among officials in the health system. However, there now seems to be widespread support for the constructive role such groups can play.

Another example is from the TB sector: one can only imagine how the federal semi-military prison authorities at first must have looked upon the ‘forced’ partnership with ‘civilian’ health authorities at the regional level.21 Now, both sides acknowledge the necessity of such collaboration.

It might be discussed whether the encouragement of a ‘civil society’ is a relevant goal within the Barents Health Programme (or indeed whether it permeates all areas of co-operation). On the other hand, there is reason to believe that the inclusion of Russian NGOs could enhance project implementation in several cases, cf. the fruitful role of the Red Cross in the TB projects. However, this is a complicated issue, requiring considerable patience.

A related dilemma concerns the choice of governmental structures to include on the Russian side. One of our Murmansk interviewees (admittedly a representative of an NGO) complained that ‘As soon as the Norwegians have a joint project – be it in the area of health care or other fields – they know only one address: Prospekt Lenina 75 [the address of the regional administration]’. It should be borne in mind that Russia is a federal state with a complex mixture of regional and federal agencies located in the territory of a region. In the project on TB control in Murmansk prisons, for instance, representatives of the prison authorities (as mentioned, a federal agency subordinate to the Ministry of Justice) wondered about the inclusion of the regional health committee in the project: neither has any authority over the other, argued our interviewees. On the other hand, and in line with the argument set out in the preceding paragraph, such co-operation – ‘untraditional’ in a Russian context – has the 20

For examples from the fields of fisheries and nuclear safety, see G. Hønneland (2000), ‘Enforcement cooperation between Norway and Russia in the Barents Sea fisheries’, Ocean Development and International Law 31: 249-267; and G. Hønneland & A. Moe (2000), Evaluation of the Norwegian Plan of Action for Nuclear Safety. Priorities, Organisation, Implementation, Evaluation Report 7/2000, Oslo: The Ministry of Foreign Affairs.

21 The ‘semi-military’ self-perception of the prison system is reflected in term used by prison officials about the health authorities as ‘the civilian system’. Most groups of uniformed personnel in Russia have a very strong sense of professional fellowship, often involving a somewhat condescending view of the ‘civilian’ sector of society.

Evaluation of the Barents Health Programme

23

possibility of enhancing governance. Our conclusion is that Norwegian project managers should be conscious about which Russian institutions they choose to work with and of the formal and informal relationships between them.

Awareness of the bureaucratic borderlines in Russia is important. In Russia, health care and social services are usually quite separate. Many issues do of course cut across boundaries, and co-operation between agencies is necessary, and is widely practised in Norway. The integration of social services and medication aid in the TB sector is an illustration of a successful solution to a difficult problem. Also other projects need to take the social dimension of health issues into consideration.

3.4.3 Co-ordination between projects As mentioned in Chapter 2, co-ordination of Norwegian funded activities with projects funded

by Finland and Sweden has been lacking. Several Norwegian project participants have also said they have a distinct feeling, or actual evidence, that their Russian partners are engaged in parallel activities, funded by other Nordic sources or other countries or organisations. The Russian project participants make no effort to inform about ‘parallel’ projects. It seems reasonable to believe that lack of information and co-ordination makes some projects less effective than they otherwise could have been. It should be stressed though, that this problem is far from unique in the Barents Health Programme context, it is rather the rule than the exception in areas where western countries or international organisations have carried out joint programmes in Russia. As noted above, the failure to inform is not only found on the Russian side. But more should be done to demand information from the Russian side, as well as increasing co-ordination at the Nordic level.

However, there also is a co-ordination problem among the Norwegian-funded projects. Even though project applicants are required to familiarise themselves with related projects, and projects are accepted on the condition that co-ordination is sought, there seems in some cases to be little co-ordination going on. Apparently there is very little contact between the various projects dealing with care of infants and mothers. Communication and co-ordination would seem all the more natural in such projects that deal with changing established practices of the Russian health care system, indicating that the projects meet many of the same problems. In other project clusters co-ordination seems to function well, notably between the Norwegian Heart and Lung Association (LHL) and the Red Cross in the tuberculosis projects.

Project co-ordination would probably benefit from the organisation of ‘experience-sharing seminars’ for project participants, along the lines of the conference organised by the programme secretariat in August 2002. But they could be smaller, bringing together participants from one topical area at a time.

Evaluation of the Barents Health Programme

24

3.4.4 Budget subsidies Among the projects, only some have a definite start and end points, meaning that they are intended to solve a specific problem for good. In such ‘ideal’ cases it is relatively easy to avoid dependence on Norwegian financing. In other cases Norwegian projects help start activities that are to be taken over by regular Russian sources. It is obviously important that the prospects of regular funding are discussed before such projects are started.

But in some cases what is needed most is not start-up of new activities, but rather support for existing programmes. This has also been part of the realities of the Barents Health Programme. It is safe to say for example, that Norwegian money has replaced regular Russian funding in some of the vaccination programmes. One challenge is to avoid dependence on Norwegian funding in the long term, another is related to confusion concerning who has responsibility for what. This issue was raised with several Russian project participants. They all seemed to be aware of the potential for problems and said that ‘replacement financing ’ had been needed in 1999 in the aftermath of the Russian economic collapse in 1998, but that the government financing situation was much better now. They all seemed to prefer Norway to fund projects that had no place in the official budget, perhaps also fearing that ’budget subsidies’ would only deprive them of regular funding. In any case it is our conclusion that one should seek to establish a picture of the financial situation surrounding new project initiatives, to avoid a situation with ‘replacement financing’.

3.5

Cost effectiveness

A systematic cost/ benefit evaluation was not part of our remit, since the measuring of the ultimate benefit, i.e. improvement of the health situation, is outside the scope of the evaluation. But some observations can be made on the basis of the cost structure alone.

Although the organisation of projects is often not very bureaucratic, there is no reason to believe that substantial sums have been diverted for other purposes. The projects are mostly not very large and involve a limited number of people. Commercial interests are not involved to any great extent. Many participants have a strong devotion to the cause they are working for. We argue that a simple, unbureaucratic organisation of projects under such circumstances is likely to be more cost effective than projects with more emphasis on formalities, which may temper the idealism from which projects currently benefit.

However, it is often difficult to ascertain the full cost of a project on the basis of project documentation. Most of them involve Norwegian health care workers. The labour put into the projects, as well as administrative overheads, is usually not counted. In these cases, the project budget only covers direct costs to the Norwegian side: equipment, travel etc. The full cost is concealed and covered by the participating institutions. If there is slack in the system, this may in reality mean very little. But since the general impression is of considerable Evaluation of the Barents Health Programme

25

tightness in the Norwegian health care system, one must assume that real costs are incurred if personnel devote work-hours to health care work in Russia. In general we would say that for projects involving Norwegian health care workers the costs are understated.

Norwegian NGOs calculate project costs differently. They usually include full costs: i.e. labour and overhead. Thus, within the Barents Health Programme such projects generally appear to be more costly than projects in the former category. This also applies to WHO projects. However, the private organisations take great pain to cover their administrative costs from other sources, as does WHO, but to a lesser extent. The Norwegian Red Cross usually includes an overhead of 10 per cent on foreign projects, but in the Barents Health Programme it covers this share themselves. The Norwegian Heart and Lung Association (LHL) and Norwegian Peoples’ Aid ‘subsidize’ their projects in Russia, too. But there is no standardised approach to this issue.

From the point of view of the programme secretariat it is only the net costs – i.e. the costs covered by the programme – that matter. However, seen in a broader context it would seem correct also to take into account the indirect costs associated with many projects and which are at present not always transparent. That said it is also important that projects can continue to benefit from voluntary work by Norwegian health personnel as well as NGOs.

Evaluation of the Barents Health Programme

26

4

Conclusions and recommendations

4.1

Conclusions

4.1.1 Project selection and composition of project portfolio The project portfolio of the Barents Health Programme reflects the overarching objectives of the programme. However, there has been a rather uneven distribution of funds among the five prioritised areas. Area 1 – Combating new and re-emerging infectious diseases – and Area 2 – Supporting reproductive health care and child health care – have received 39 and 36 per cent of total funds respectively. Area 5 – Quality improvement of medical services –received a little less, 22 per cent, while the two remaining areas, Area 4 – Improving services for indigenous people – and Area 3 – Counteracting life-style-related health problems, received 2 and 1 per cent of the total funds, respectively. The imbalance is striking, but can partly be explained by conflicting priorities within the programme documents. This imbalance can also help explain the uneven geographical distribution of projects. Arkhangelsk Oblast received 41 per cent of the programme’s funds, Murmansk Oblast 31, the Republic of Karelia 4 per cent and Nenets Autonomous Okrug almost nothing from the projects targeting specific regions.

However, twenty-four per cent of the funding goes to projects that cut across the four regions of Northwestern Russia.

The project portfolio is dominated by small projects. One project received 8.39 mill. NOK, another 4.2 mill., and one 3.05 mill. Eleven projects received between one and three million.

The remaining 44 projects received less than a million, most of them less than 500,000 NOK. Some projects are closely intertwined; for instance, some of the smaller projects can be regarded as add-ons to the larger projects in the area of TB control.

A rough and ready breakdown of the project costs by use shows that the lion’s share of the money has been used for training of Russian personnel and exchanges of Norwegian and Russian health personell, which is a central part of the established ‘general basis for the cooperation’ under the Barents Health Programme. The second largest category is purchase and transportation of medicine and equipment. This purpose is not mentioned explicitly among the general or specific project criteria.

The programme applies a ‘bottom-up approach’ in the sense that it is based on applications for specific projects. At the same time, the list of priorities and project criteria is so long that it nevertheless resembles a ‘top-down approach’. Altogether, the programme contains too many objectives, priorities and concerns.22 Two important selection criteria – quality and cost 22

The programme board may already have taken note of this problem. In the announcement of applications for autumn 2001, reference to the general criteria (except the emphasis on children’s health) as well as the specific criteria is absent. Letter from the Ministry of Health, 3 September, 2001.

Evaluation of the Barents Health Programme

27

effectiveness – are not mentioned in the programme document. We take it for granted that such considerations have played a major role when projects are selected. But, logically, the larger the number of other priorities and considerations, the less room will be left for quality and cost-efficiency considerations. Nevertheless, we are left with the impression that the long list of priorities and concerns has not put serious constraint on the programme committee or the secretariat.

4.1.2 Project implementation Project implementation under the Barents Health Programme has largely been successful. In the majority of projects, the established goals have either been achieved or the projects develop according to plan. The main obstacles were associated with Russian bureaucratic procedures, primarily in the area of customs, and lie outside the control of both Norwegian or Russian project participantsworkers. A considerable degree of learning has taken place in how to deal with such obstacles.

Also, at the professional level, the projects have contributed to new knowledge on the Russian side. For instance, changes in Russian attitudes to TB treatment and breast-feeding are described as ‘revolutions’ by Russian project participants.

The TB projects emerge as examples of successful co-ordination of different agencies and of the inclusion of Russian NGOs in the project work. The Red Cross has fulfilled a vital role in the Northwest Russian system for treatment of TB patients. Red Cross personnel ensure that healthy nutrition reaches discharged patients, particularly alcoholics and the homeless, and see to it that they take their medication. Further, the TB projects have led to co-ordination between the regional health authorities and the prison authorities, which represent federal authorities (the Ministry of Justice).

There have been some problems in the Russian organisation of some of the projects, notably the WHO project on healthy nutrition for women and children in the Barents region. Also, the goals of this project are perceived differently by the WHO project management and the Russian project workers. The Russians describe it as primarily or solely a research project, whereas WHO emphasises the elaboration of a public policy for healthy eating habits.

Leading health officials in Northwestern Russia describe the project as ‘too massive [and] gigantic’, indicating that projects with a simpler administration and more clear-cut objectives are to be preferred.

There seems to be a need for closer integration of health and social perspectives in some projects. Norwegian project developers should be aware of the administrative divisions in Russia, which differ from those in Norway. Some Russian authorities acknowledge that they have things to learn from Norway in this regard.

Evaluation of the Barents Health Programme

28

Some co-ordination problems at the programme level, as well as between projects, have been observed. Swedish and Finnish projects are not properly registered in the database that has been set up. There could also have been more contact between Norwegian projects with related content but functioning in different parts of the Barents region.

4.2

Recommendations

A considerable amount of individual and institutional learning among Norwegian project participants has taken place during the implementation of the Barents Health Programme, which is likely to benefit an extension of the programme. Many important projects under the programme are still in progress and would suffer from a disruption in Norwegian funding.

Nevertheless, there are several points that need consideration before deciding on a continuation of the programme. Some of them are mentioned below.

On the definition of priorities: •

‘Top-down’ vs ‘bottom-up’ selection of projects: There has been a ‘bottom-up’ selection of projects under the Barents Health Programme so far, but the detailed lists of priorities for the programme and the various prioritised areas give the impression of a ‘top-down’ approach. In a continuation of the programme, there should be fewer and wider priorities.

If the programme administration feels there is a need to establish a project addressing a special or partial issues, it should be announced separately rather than presenting a long list of subgoals to include in the projects.

On the structure of projects: •

Complex vs clear-cut project organisation and objectives: As a general observation, we would stress the importance of clarity and transparency with regard to project objectives and organisation. Due to language problems and ‘cultural differences’, there is a constant risk of misunderstandings in dealings with Russia. It is generally difficult to evaluate the effectiveness of the projects. In some cases the results will only emerge in the longer term.

However, more could probably be done to establish intermediate milestones.

•

Transfer of competence vs purchase of commodities: A line could be drawn between projects that involve professional co-operation with Norwegian health care specialists and exchange of competence, and projects which merely involve financing of activities in Russia. The need for budgetary support in Northwestern Russia has been acute for some time, but seems to have fallen somewhat now. However, with a view to co-operation support, projects in the former category would seem to be preferable. Within the second category –‘budgetary support’ – distinctions can also be made. In most cases these

Evaluation of the Barents Health Programme

29

projects involve purchases of medicine or equipment – highly relevant for dealing with health problems. But we also find a few instances where funds have been used to establish general infrastructure – e.g. housing improvements such as roofing and windows.

Although sound health care cannot be provided in a hospital with a leaking roof, we argue that support over the Barents Health Programme should primarily be reserved for purposes more directly associated with health care. The risk of colliding with the responsibilities of Russian authorities will be less.

•

Some of the projects are primarily research oriented (although it may in some instances be difficult to distinguish between research and training). Reportedly, among the rejected applications there were several research projects. It would seem reasonable that applications for funding for such projects should be handled through the ordinary research-funding channels.

On the possibility to secure continuation of activities: •

Norwegian vs Russian financing: There is a risk that essential services in the Northwest Russian health sector become dependent on external (Norwegian) financing. A more comprehensive evaluation of funding possibilities on the Russian side, as well as for continued activity (where warranted) upon expiry of Norwegian project financing, should be included in project applications. Where activities are supposed to continue without Norwegian financing, the Norwegian contribution should be phased out gradually.

On increasing efficiency of project implementation: •

Some of the problems encountered in the projects have been caused by lack of knowledge of the institutional setting in Russia. When project proposals are of a certain size, one should require the applicant to give a description of the institutional set-up, i.e. who is in charge, how the Russian partner is financed, and whether the Russian partner is subordinate to local, regional or federal authorities.

•

More effort could be made to check the possibility of engaging NGOs in projects. While Russian NGOs are still generally weak and cannot fill all the functions NGOs have in the West, there have been positive developments in the region in recent years.

•

More should be done to secure co-ordination between projects – both at the programme level and among project participants. This includes demanding more openness on the Russian side and better co-ordination on the Norwegian.

Evaluation of the Barents Health Programme

30

Annex 1: Evaluation form (questionnaire) HEALTH CO-OPERATION PROGRAMME IN THE BARENTS EURO-ARCTIC REGION 1999 - 2002 EVALUATION Part 1: General information 1.

Information about recipient Project number: __________________________ Date: ________________________ Name:

________________________________________________________

Address:

________________________________________________________

Telephone:

________________________________________________________

Telefax:

2.

E-mail:

________________________________________________________

Person responsible:

________________________________________________________

Information on other project partner(s) (information on Russian partner see point 3) Name:

______________________________________________________

Address:

______________________________________________________

Telephone:

______________________________________________________

Telefax:

2.1 a)

________________________________________________________

______________________________________________________

E-mail:

______________________________________________________

Person responsible:

______________________________________________________

Please give a brief description: On the scope of the co-operation between the partners:

b) How the co-operation has been organised:

Evaluation of the Barents Health Programme

31

6.

Information on Russian partner(s): Name:

_____________________________________________________

Address:

_____________________________________________________

Telephone:

_____________________________________________________

Telefax:

3.1 c)

_____________________________________________________

E-mail:

_____________________________________________________

Person responsible:

_____________________________________________________

Please give a brief description: On the scope of the co-operation with the Russian partner:

d) How the co-operation has been organised:

7.

The geographical area of the project – please state name of town/area/region:

8.

Major goal for the project:

d) What was the main goal for the project?

e)

Has the project contributed to attain this? (Please specify):

f)

Is the project’s goal still relevant?

13.

The result of the project:

d) What was the expected result?

e)

What was the actual result?

f)

Please describe a possible difference between the expected and the actual result:

Evaluation of the Barents Health Programme

32

14. Overview on the financing of the project: a) The recipient’s own contribution

NKr ……………………………..

b) Other participants or partner’s contribution

NKr ……………………………..

c) Financial support

NKr ……………………………..

i) The Ministry of Health and Social Affairs NKr ……………………………..

If the project has received other allocations, please specify: 1)…………. 2)…………… 3)………………. NKr …………………………….. ii) Other Ministries, please specify ………………………………………………… NKr ……………………………..

Other d) Total expenses

NKr ……………………………..

NKr ……………………………..

e) Out of this: Purchase of Russian products and services

NKr ……………………………..

15. Type of project – please put a mark at the relevant alternative: a) Pilot project b) Main project

c) Joint venture d) Other (please specify)

16. Project activities: c)

Please define or describe the activities which are financed by the allocation granted from the Health Programme:

d) Have these activities been organised in line with the original plan? Please specify:

17. Implementation of the project: d) Please state the starting and the closing date of the project according to the application: e)

To what extent has the project been implemented according to the original time schedule? Please state the actual time of implementation of the project and the date for the submission of the account/final report:

f)

What conditions have been significant for the partners concerning i) the implementation of the project? ii) difficulties in the implementation of the project?

Evaluation of the Barents Health Programme

33

18. Training/improvement of competence: c)

Has the project included training/improvement of competence? Yes/No

d) If yes, please specify:

19. Extension of the project: c)

Will the project continue? Yes/no If yes, please specify:

d) If no, please specify:

Part 2: Supplementary information that has not been presented in part 1

________________________ Date

Evaluation of the Barents Health Programme

__________________________ Sign

34

Annex 2: List of persons interviewed during visit to Arkhangelsk and Murmansk: • • • • • • • • • • • • • • • • • • • • • • • • •

Agapitova, Galina, Chair, Murmansk Red Cross Badanina, Valentina, Chair, Arkhangelsk Red Cross Buzinov, Roman, Head, State Sanitary and Epidemologic Control Centre of Arkhangelsk Oblast Emmanuilov, Sergey, Director General, Health Department, Arkhangelsk Oblast Administration Endourova, Larisa, Head of Laboratory, Regional Tuberculosis Dispensary, Murmansk Oblast Gnevasheva, Tatyana, Senior Specialist (pediatrics), Health Department, Arkhangelsk Oblast Administration Gusev, Konstantin, Deputy Head, Federal Prison Administration (Ministry of Justice), Murmansk Oblast Khaltugina, Elena, Arkhangelsk Red Cross Khlebnikova, Nadezhda, Head, Agency for Social Information, Murmansk Kabakov, Vyacheslav, Head of Division, Health Department, Arkhangelsk Oblast Administration Kondakova, Nina, Department of Neonatology, Northern State Medical University, Arkhangelsk Kudyra, Lyudmila, Health Department, Arkhangelsk Oblast Administration Lukicheva, Elena, Deputy Head, State Sanitary and Epidemologic Control Centre of Murmansk Oblast Nemkov, Vladimir, Co-ordinator of International Programs, Murmansk Red Cross Nikishova, Elena, Project Co-ordinator, Regional Tuberculosis Dispensary, Arkhangelsk Oblast Nizovtseva, Nina, Head, Regional Tuberculosis Dispensary, Arkhangelsk Oblast Opeshelov, Sergey, Deputy Chairman, Murmansk Oblast Health Committee Popova, Olessya, HIV/AIDS Co-ordinator, Arkhangelsk Red Cross Presnova, Svetlana, Head, Regional Tuberculosis Dispensary, Murmansk Oblast Sokolova, Lyubov, State Sanitary and Epidemologic Control Centre of Arkhangelsk Oblast Sumarokov, Yuriy, Senior Project Specialist, TACIS, Arkhangelsk Schepenikova, Marina, Murmansk Oblast Health Committee Schulga, Aleksandr, Head of Medical Service, Federal Prison Administration (Ministry of Justice), Murmansk Oblast Toichkina, Tatyana, Deputy Head Physician, Regional Tuberculosis Dispensary, Arkhangelsk Oblast Veko, Galina, TB Co-ordinator, Arkhangelsk Red Cross

Evaluation of the Barents Health Programme

35

Annex 3: List of projects in the Barents Health Programme financed by Norway 1999-2002

Grants up to December 2001 No Y9710 Y9711 Y9713 Y9720 Y9724 Y9725 Y9726 YO378 YO381 B005 B006 B102 B103 B106 B115

Y9712 Y9714 Y9715 Y9716 Y9717 Y9722 Y9723

Project

Grant (NOK)

Area 1: Combatting new and re-emerging infectious diseases TB control in Arkhangelsk 8 390 000 TB control in Arkhangelsk: Improved 1 800 000 Diagnosis and Epidemiology Tuberculosis Project in Murmansk Prisons 1 450 000 Immunization in Arkhangelsk 2 650 000 Translation of textbook on modern infectious 90 000 diseases into Russian Health in NW Russia and the Baltic countries 345 000 –an expert conference Hepatitis B prevention in the Republic of 372 500 Karelia Youth Peer Education on HIV/AIDS and 650 000 prevention of other sexually transmitted diseases Preventative work against drug abuse and 250 000 HIV- and Hepatitis infection in schools and military camps in Murmansk Prophilactics of HIV/AIDS and other 100 000 Sexually Transmitted Diseases Russian Red Cross against tuberculosis 1 000 000 Collaboration with Nenets SEC – Regional 108 000 State Surveillance and Epidemological Centre Rubella prevention in the republic of Karelia 304 000 Competence network for fighting tuberculosis 200 000 in Arkhangelsk oblast Rubella prevention in the Murmansk Region 410 000 Sum 18 119 500

Geographical focus Arkhangelsk Arkhangelsk Murmansk Arkhangelsk Barents Barents Karelen Barents Murmansk Arkhangelsk Murmansk, Arkhangelsk Nenets Karelia Arkhangelsk Murmansk

Area 2: Supporting reproductive health care and child health care Recent advances in Ultrasound 1 050 000 Murmansk Safe Motherhood 2 469 000 Barents Development programme for Monchegorsk 2 510 000 Murmansk home for children with disabilities Breastfeeding groups in the Barents Region 1 560 000 Barents Dental health co-operation between Apatity 70 000 Murmansk and Finnmark County 96-99 Healthy nutrition for women and children in 3 050 000 Murmansk, Arkhangelsk the Barents region "…a full and decent life" 400 000 Arkhangelsk

Evaluation of the Barents Health Programme

36

YO373

YO377 YO379 B002 B101 B107 B108 B110 B113 B114 B116 B117

Y9718 B118

Y9719 YO383

Y9727 YO370 YO374 YO375 YO376 YO380

2 400 000 Murmansk Intervention and improvement in the care of pregnant women and reduction of the perinatal mortality and morbidity in the industrially exposed population of Monchegorsk and the indigenous population of Lovozero Treatment of children with intersex in 290 000 Arkhangelsk Arkhangelsk Activity and training centre in Kirovsk 1 100 000 Murmansk Women and Cancer - Recent advances in 500 000 Barents operative techniques (Conference) Asthma problems under control 90 000 Karelia Childrens health in the Barents region 90 000 Barents conference Clinical cell culture laboratory in 50 000 Arkhangelsk Arkhangelsksk. Competence building and reproductive health.

Competence building of pediatric nurses 150 000 Arkhangelsk Protection of pregnant women and fetal health 50 000 Karelia in the Republic of Karelia Creation of an electronic database for 50 000 Karelia monitoring of life-threatening complications of pregnancy and delivery Exchange of competence in child- and youth 200 000 Karelia related social work in the Rep.of Karelia and Troms County Pregnancy and Infectious diseases 450 000 Barents Sum 16 529 000 Area 3: Counteracting life style related health problems Lifestyle and Health in the Barents Region 562 500 Murmansk Cross-cultural alcohol and drug prevention 100 000 Arkhangelsk family intervention initiatives Sum 662 500 Area 4: Improving services for indigenous people Medical development in Lovozero 400 000 Murmansk Alcohol and drug abuse program for 350 000 Murmansk indigenous people 750 000 Area 5: Quality improvement of medical services Used medical equipment to Northwest Russia 4 200 000 11th International Congress on Circumpolar 150 000 Health Co-operation within the nursery field in 230 000 Arkhangelsk and Tromsø regional hospitals Primary Health Care Project in Arkhangelsk 500 000 Further development of heart surgery and 500 000 circulatory lab International summer school ”Current trends 61 000 in ultrasonograhy diagnostics and diapeutics”

Evaluation of the Barents Health Programme

37

Barents Barents Arkhangelsk Arkhangelsk Arkhangelsk Karelen

YO382 YO384 B001

B003 Y9745 B104 B105 B109 B111 B112 Y9721

276 000 Karelen

Pulmonary Diseases in the Republic of Karelia Tranport of medical equipment to Murmansk Elaboration and introduction of the optimal system of medical consistent rehab. of children and young adults with the disturbances of the locomotary apparatus Student exchange Tromsø-Arkhangelsk Database Workshop for technical rehabilitation aid in Murmansk Organising technical rehabilitation aids centre/workshop in Kirovsk for the Southern part of Kola peninsula Developing competence in electromyographics in NW Russia Quality improvement of the psychiatric services in Arkhangelsk Regional Hospital Suicide intervention training program in Arkhangelsk University of Tromsø: Four different projects Sum

950 000 Arkhangelsk, Murmansk 10 476 320

Total for all projects

46 537 320

Source: Ministry of Health - Project database.

Evaluation of the Barents Health Programme

38

150 000 Barents 483 320 Karelia

250 000 Arkhangelsk 400 000 Barents 395 000 Murmansk 1 150 000 Murmansk 32 000 Arkhangelsk 250 000 Arkhangelsk 200 000 Arkhangelsk