16. The network also helped creating standard job titles, competencies framework, and terms of reference for M&E specialists in IFC to facilitate closer communication within the network and professionalization of the Results Measurement career stream within IFC. 17. IEG validated 100 percent of PCRs prepared for IFC Advisory Services during FY08-09. Beginning in FY10, IEG moved to a sampling approach, selecting a stratified (by business line) random sample from the population of PCRs (51 percent sampling rate three year rolling average). The sampling rate is set at a level sufficient to make inferences about success rates in the population at the 95 percent confidence interval with a sampling error of +/–5 percent or less. Among the sampling criteria used is the indicative development effectiveness self rating from the PCR. IEG excludes from its population PCRs prepared for non-client facing and knowledge products. 18. IEG’s criteria for rating quality dimensions as “some extent” is based on evaluators’ findings of important shortfalls, akin to Partly Unsatisfactory. 19. IEG rated TSTJ where insufficient achievement in outcomes made it impossible to assign a development effectiveness rating by completion, but where there was evidence of clear momentum and a likelihood that with a bit more time results may emerge. 20. IEG rated development effectiveness in cases where outcomes achievements were assessed, even in instances where it was too soon to judge impacts. However, if outcomes were assigned TSTJ or CV, IEG for the most part also assigned TSTJ and CV for development effectiveness. In this sense, outcome achievement was likely the most important dimension in considering the development effectiveness rating, while strategic relevance, efficiency, and outputs and impacts also played a role in determining the overall development effectiveness rating. 21. This refers to projects for which impacts were not deemed “not applicable” by having met IFC’s Impact Exclusion Criteria. 22. IEG’s approach to assessing financial intermediaries’ sustainability and impacts beyond the financial intermediaries on sub-borrowers and overall market development for Advisory Services projects in the PCRs reviewed is based largely on concepts derived from IFC’s XPSR framework. 23. Among the mature projects, MIGA selects few for self-evaluation every year (up to eight) and IEG independently evaluates the remaining ones. 24. These are the same as IFC’s disagreements. Three of four disagreements involved rating of additionality as a part of work quality rather than a separate dimension of performance, and one involved methodology of assessing the business success of loans to financial intermediaries. 25. This excludes financial sector projects for which no E&S work was needed beyond initial project screening.
Monitoring and Evaluation in IFC and MIGA
53