HandBook | Impact Assessment Indicators

Developing impact assessment indicators to evaluate Erasmus+ projects.

HANDBOOK

BB&R EDITIONS

Published: 2019 Š BB&R Editions Developing impact assessment indicators to evaluate Erasmus+ projects. Project 2017-3-ES02-KA105-010400 implemented by Biderbost, Boscan & Rochin (BB&R) in collaboration with Human Rights Youth Organization (HRYO) and Academy of Entrepreneurship (AKEP).

!

https://impact-assessment.eu/

!

Biderbost, Boscan & Rochin John Dalton, 38-52, Bajo J 37007 Salamanca, Spain bbyr@bbyr.com www.bbyr.com

This project has been funded with support from the Erasmus+ Programme of the European Commission. This publication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein.

definitions

IMPACT ANALYSIS (EX ANTE)

IT INVOLVES DOING A PROSPECTIVE ANALYSIS OF WHAT THE IMPACTS OF AN INTERVENTION MIGHT BE

IMPACT ASSESSMENT (EX POST)

IS PART OF THE EVALUATION AND MANAGEMENT ACTIVITY OF PROJECT FOCUSING ON THE EFFECTS OF THE INTERVENTION

DIFFERENCE BETWEEN EVALUATION AND IMPACT ASSESSMENT

EVALUATION COVER ISSUES SUCH AS THE INTERVENTION DESIGN, THE COST AND EFFICIENCY OF THE INTERVENTION

EXPERIMENTAL GROUP AND CONTROL GROUP

COMPARISON BETWEEN TWO GROUPS SIMILAR IN SOCIODEMOGRAPHIC AND PSYCHOSOCIAL TERMS, BUT DIFFERING IN THE RECEPTION OF THE INTERVENTION.

BRAIN TEASERS

! !

TYPES OF INDICATORS FIND THE CORRECT DEFINITION AND EXAMPLE OF THE MOST TYPICAL INDICATORS

EFFICACY INDICATOR

Reduction in the rate of people who do not work or study

EXAMPLE OF A QUALITY INDICATOR

Measures attributes, capabilities or characteristics that the goods or services must have to adequately meet the objectives of the Action/ Project/Programme.

EFFICIENCY INDICATOR

Measure the degree of fulfilment of an objective.

QUALITY INDICATOR

Number of requests processed by an official or public servant.

EXAMPLE OF AN EFFICACY INDICATOR

Measure the relationship between the products and services generated with respect to the inputs or resources used (i.e. average productivity of the different factors of production).

EXAMPLE OF AN EFFICIENCY INDICATOR

Degree of user satisfaction with the Erasmus+ project.

! ! !

! !

TYPES OF INDICATORS FIND THE CORRECT DEFINITION AND EXAMPLE OF THE MOST TYPICAL INDICATORS

EFFICACY INDICATOR

Reduction in the rate of people who do not work or study

EXAMPLE OF A QUALITY INDICATOR

Measures attributes, capabilities or characteristics that the goods or services must have to adequately meet the objectives of the Action/ Project/Programme.

EFFICIENCY INDICATOR

Measure the degree of fulfilment of an objective.

QUALITY INDICATOR

Number of requests processed by an official or public servant.

EXAMPLE OF AN EFFICACY INDICATOR

Measure the relationship between the products and services generated with respect to the inputs or resources used (i.e. average productivity of the different factors of production).

EXAMPLE OF AN EFFICIENCY INDICATOR

Degree of user satisfaction with the Erasmus+ project.

! !

!

STEPS TO BUILD AN INDICATOR ORDER THE STEPS THAT YOU MOST FOLLOW TO BUILD AND INDICATOR

! !

Establish frequency of calculation, baseline and goals.

Validate the indicator.

Formulate the indicator.

Analyse the objective you want to measure.

! !

2. SELECT RELEVANT ASPECTS TO MEASURE.

4. ESTABLISH THE MEANS OF VERIFICATION.

! !

!

STEPS TO BUILD AN INDICATOR ORDER THE STEPS THAT YOU MOST FOLLOW TO BUILD AND INDICATOR

! !

Establish frequency of calculation, baseline and goals.

Validate the indicator.

Formulate the indicator.

Analyse the objective you want to measure.

! !

1. ANALYSE THE OBJECTIVE YOU WANT TO MEASURE.

2. SELECT RELEVANT ASPECTS TO MEASURE.

3. FORMULATE THE INDICATOR (IT MUST BE CLEAR, PRECISE AND SELFEXPLANATORY).

4. ESTABLISH THE MEANS OF VERIFICATION.

5. VALIDATE THE INDICATOR.

6. ESTABLISH FREQUENCY OF CALCULATION, BASELINE AND GOALS.

MATCH-UP

INDICATOR

1

IS A SIGN, A WARNING, THAT DELIVER US INFORMATION, QUANTITATIVE OR QUALITATIVE, ABOUT WHETHER A CERTAIN CONDITION EXISTS OR IF CERTAIN RESULTS HAVE BEEN ACHIEVED OR NOT. 1

EFFICACY INDICATOR 2

MEASURE THE DEGREE OF FULFILMENT OF AN OBJECTIVE.

2

EXAMPLE OF AN EFFICACY INDICATOR 3

REDUCTION IN THE RATE OF PEOPLE WHO DO NOT WORK OR STUDY

3

QUALITY INDICATOR

4

MEASURES ATTRIBUTES, CAPABILITIES OR CHARACTERISTICS THAT THE GOODS OR SERVICES MUST HAVE TO ADEQUATELY MEET THE OBJECTIVES OF THE ACTION/PROJECT/ PROGRAMME. 4

EXAMPLE OF A QUALITY INDICATOR 5

RESPONSE TIME OF THE HOSPITALS TO THE REQUIREMENTS OF THE BENEFICIARIES.

5

EFFICIENCY INDICATOR 6

MEASURES THE RELATIONSHIP BETWEEN THE PRODUCTS AND SERVICES GENERATED WITH RESPECT TO THE INPUTS OR RESOURCES USED (I.E. AVERAGE PRODUCTIVITY OF THE DIFFERENT FACTORS OF PRODUCTION).

6

EXAMPLE OF AN EFFICIENCY INDICATOR

7

NUMBER OF PARTICIPANTS THAT FULFIL THE FINAL EVALUATION OF THE PROJECT.

7

INDICATORS HELP TO ANSWER KEY QUESTIONS AT VARIOUS STAGES IN THE STRATEGY OF OUR ACTION/PROJECT/PROGRAMME, SUCH AS:

8

WHERE ARE WE NOW? WHERE DO WE WANT TO GO? ARE WE TAKING THE RIGHT PATH TO GET THERE? ARE WE THERE YET? 9

MEANS OF VERIFICATION 10

THE TOOLS USED TO INDICATE WHERE AND HOW THE INFORMATION ABOUT THE INDICATORS CAN BE OBTAINED (ACCORDING TO THE OBJECTIVES).

10

EXAMPLES OF MEANS OF VERIFICATION 11

BASELINE FOCUS GROUPS INTERVIEW WITH PARTICIPANTS EVALUATION REPORT PRESS RELEASES SURVEYS AND QUESTIONNAIRES FOLLOWERS AND LIKES IN SOCIAL MEDIA 11

EVALUATION

Page 1 of 1

! ! !

! !

IMPACT ASSESSMENT INDICATORS | FIRST CONTACT

This survey will allow us to explore your initial sensations regarding Impact Assessment Indicators and ErasmusPlus that we will address during this training course. The final aim of this document is to really know if this course is effective. Answer the following statements, ‘0’ being ‘little or completely disagree' and ‘4’ being ‘a lot or completely agree'. Please, answer honestly. There are not correct or incorrect answers.

!

STATEMENTS Do you feel European or European Citizen?

0

1

2

3

4

Do you feel Greek/Italian/Spanish?

0

1

2

3

4

Which is your predominant identity: national or European?

National

European

1 How much do you know about Impact Indicators?

0

1

2

3

4

How much do you know about Impact Assessment?

0

1

2

3

4

How much do you know about Erasmus+ Programme?

0

1

2

3

4

Do you think that knowing about impact assessment indicators can help you in your daily work?

0

1

2

3

4

Do you consider that participating in a Erasmus+ experience allow you to grow-up professionally?

0

1

2

3

4

!

Page 1 of 1

! ! !

! !

IMPACT ASSESSMENT INDICATORS | FINAL CONTACT

This survey will allow us to explore your final sensations regarding Impact Assessment Indicators and ErasmusPlus that we will address during this training course. The final aim of this document is to really know if this course is effective. Answer the following statements, ‘0’ being ‘little or completely disagree' and ‘4’ being ‘a lot or completely agree'. Please, answer honestly. There are not correct or incorrect answers.

!

STATEMENTS Do you feel European or European Citizen?

0

1

2

3

4

Do you feel Greek/Italian/Spanish?

0

1

2

3

4

Which is your predominant identity: national or European?

National

European

1 How much do you know about Impact Indicators?

0

1

2

3

4

How much do you know about Impact Assessment?

0

1

2

3

4

How much do you know about Erasmus+ Programme?

0

1

2

3

4

Do you think that knowing about impact assessment indicators can help you in your daily work?

0

1

2

3

4

Do you consider that participating in a Erasmus+ experience allow you to grow-up professionally?

0

1

2

3

4

!

partners

PARTNERS PARTNERS BIDERBOST, BOSCAN & ROCHIN

!

B i d e r b o s t , B o s c a n & Ro c h i n ( B B & R ) i s a professional, international consulting firm located in Salamanca (Spain). BB&R specialises in many distinguished fields of expertise including social responsibility in the field of youth. BB&R’s team consists of ten young and qualified professionals from all over the world, with various backgrounds and proficiencies in different subject matters. BB&R has collaborated with different notable organisations (European Union, EU-LAC Foundation, InterAmerican Development Bank, United Nation Development Programme, World Bank, International Organization for Migration), NGOs and government agencies (Spain, Brazil, Norway, Canada, Bulgaria, UK) in an effort to train European youth to be more qualified in today’s work force, and to live a more active civic life.

!

http://bbyr.com/en/

HUMAN RIGHTS YOUTH ORGANIZATION

!

Human Rights Youth Organization (HYRO) is a nonprofit NGO founded in Palermo, Italy in 2009. HYRO aims to facilitate and improve the quality of human rights throughout the world by creating a network of organisations with similar human rights oriented goals. HYRO actively works towards promoting peace by regularly taking part in, and promoting non-violent actions towards liberation of any form of oppression.

!

http://www.hryo.org

ACADEMY OF ENTREPRENEURSHIP

!

The Academy Of Entrepreneurship (AKEP) is a nongovernmental, nonprofit organisation, with an innovative and authentic character, trying to meet the new challenges of the modern labour market as its first priority. AKEP, on the other hand, is determined to cope with the new reality created within the economic crisis in an effective and consistent way, on a national, European and global level, and aims to provide answers at a rapid but constant pace and on a theoretical and practical level, regarding the challenges that arise everyday and make entrepreneurship seem uncertain and risky.

!

http://akep.eu

FUNDED BY FUNDED BY ERASMUS+ PROGRAMME

!

Erasmus+ is an EU funded program that aims to support education, training, youth and sport in Europe. With a budget of ₏14.7 billion, the program will provide opportunities for over 4 million Europeans to study, train, and gain experience abroad. Erasmus+ also aims to promote sustainable development of its partners in the field of higher education, and to contribute to achieving the objectives of the EU Youth Strategy. Specific issues tackled by the program include reducing unemployment, promoting adult learning, encouraging young people to take part in politics, supporting innovation, and promoting cooperation and mobility with the EU’s partner countries.

! !

https://eacea.ec.europa.eu/erasmus-plus_en

thank you very much!

Elias Triantafillidis Andrei Mancas Vasileios Papaioannou Anastasios Giannaros Simona Marano Francesco Paolo Riotta Sara Cadoni Alessandra Puccio Eduardo Muñoz Suárez Daniel Gonzalo Salinero Justine Frechette María Fernanda Meléndez

DISCLAIMER - CREATIVE COMMONS This handbook is under the licence AttributionNonCommercial-NoDerivatives 4.0 International (CC BYNC-ND 4.0) This means that you must give appropriate credit and provide a link to the license, but not in any way that suggests the licensor endorses you or your use (Attribution). You may not use the material for commercial purposes (NonCommercial). If you remix, transform, or build upon the material, you may not distribute the modified material (NoDerivatives).

Developing impact assessment indicators to evaluate Erasmus+ projects. Project 2017-3-ES02-KA105-010400 implemented by Biderbost, Boscan & Rochin (BB&R) in collaboration with Human Rights Youth Organization (HRYO) and Academy of Entrepreneurship (AKEP).

!

https://impact-assessment.eu/

!

Biderbost, Boscan & Rochin John Dalton, 38-52, Bajo J 37007 Salamanca, Spain bbyr@bbyr.com www.bbyr.com

BB&R EDITIONS