Seite - (000251) - in Evaluating Climate Change Action for Sustainable Development

The studymaterial usedwas the evaluation reports of thoseCCAprogrammes that: (1) have been implemented by UNDP and other United Nations agencies; (2) have finished initial implementation cycles; and (3) have been subject to terminal evaluations. One of the unique aspects of these identified CCA programmes is that they represent the first evaluation results of the completed CCA programmes within the UNDP system (as of November 2014). Out of a total of 11, nine CCA programmes were selected based on the criterion that the quality of the evaluation reportswas rated to bemoderately satisfactory or higher by the UNDP Independent Evaluation Office.1 The authors conducted a meta- analysis of those ex-post evaluations by closely examining and comparing the contents of the evaluations by applying the philosophical lens of critical realism. The nine programmes includedwere implemented in the following nine coun- tries:Armenia, Egypt,Malawi,Mozambique,Namibia, thePhilippines, Tanzania, TurkeyandZimbabwe(seeTable13.1forsummary).Asthetableshows,withinthe context ofUNprogramming, theseprogrammesvary inmanyaspects: the funding source (such as Global Environment Facility, Millennium Development Goals Achievement Fund, andUnitedNations internal resources); types of beneficiaries, target audiences andgeographic regions (ranging from local vulnerable communi- ties to inter-ministerialmainstreaming at the government level); and implementa- tion modalities (including UNDP stand-alone, United Nations interagency joint programmingandDelivering asOne2). This paper presents the findings of the meta-analysis conducted of the nine evaluation reports. Although the programmes evaluated vary fromone another in manyaspects,what iscommonis thestructureoftheevaluationreports.Eachreport consistsof fourmajorsections,eachofwhichcoversaspecificevaluationcriterion: relevance, efficiency, effectiveness and sustainability. Theevaluatorswhoconducted thenineCCAprogrammeevaluationsallutilised the definitions of each criterion in Table 13.2, which are based on the OECD evaluation criteria adaptedbyUNDPand its partners (OECD2002).3 1Itwasdone throughUNDPIEO’s quality assurance exercise. It is concernedwith thequality of howevaluation report iswrittenbycheckingwhether the structure of evaluation reports includes thenecessarysectionsandaproperevaluationframeworkhasbeenput inplace.Thus“moderately satisfactory” or above rated evaluation reports do not necessarily mean high quality of project activity results themselves. 2Although there isnounifieddefinitionofDeliveringasOnemodality (UN2012), it shouldentail “Four Ones”, i.e. one leader, one programme, one budget and one office amongst different agencies of the UN system. Joint Programming, is often contrasted with Joint Programmes, where the latter implies a set of discrete but related programmesbyUNagencies and the former implies joint efforts even fromthestageofplanninganddesigningofaprogramme,which is also to be implemented together. 3The authors are aware of criticism pertaining to the rather narrow application of the criteria internally towards interventions (for instance, relevancecould includewhether the intervention is contributing to positive change and the achievement of impact; and sustainability should include not only the continued benefits from the intervention butwhether the intervention contributes to broader sustainabledevelopment).However, as thesecriteriaarewidelyused in theevaluations in thenarrow sense, this understanding is appropriate for our analysis. 13 WhatDoEvaluationsTellUsAboutClimateChangeAdaptation?Meta-analysis. . . 237