Seite - (000116) - in Evaluating Climate Change Action for Sustainable Development

Outreachas across-cutting issue rather thana stand-alonecomponent.Theportfo- lio under review included 57 projects and programmes classified by UNEP as belonging to the CCSP and that were either on-going or had been initiated after 1 January2008.A littleoverhalf (32)of theseprojectswerecompletedat the time of the evaluation, 20were on-going and the remaining 5were inactive or had an unknownstatus.Within thisportfolio, therewereanumberof interventionsknown as “umbrella projects”, which included several, independent sub-projects contrib- uting to the same Expected Accomplishment or (set of) Programme of Work Outputs. If all sub-projectswere counted, the total evaluation portfolio comprised about88 interventions.Their spreadover thedifferent thematiccomponentswasas follows: 60%weremitigation, 23%were adaptation, 5%were REDD, and 9% science and outreach. The remaining combined both mitigation and adaptation objectives. 6.3 Challenges to theEvaluation Arapid assessment of the evaluability of the sub-programmeduring the inception phase had brought to light several challenges the evaluation was bound to face. First, it was expected to assess a large, highly diverse and dispersed project portfolio, spread over four components, managed by various branches across the organization based in different duty stations. Second, a review of strategic docu- ments had revealed serious issues with the results framework of the sub-programme, namely its internal logic, the results levels at which Expected Accomplishments andProgrammeofWorkOutputswere pitched and the changes in results statements, indicators and targets every 2 years. Table 6.1 presents the results framework for the mitigation component as an illustration. Third, the assessment of strategic relevance would prove quite challenging considering the rapidly changingpolitical and institutional context suchasnewdecisions immerg- ing fromUNFCCCCOPs andothers. At thesametime, theevaluationwouldneedtocopewithverylimitedevaluative evidence. For instance, monitoring of progress at the sub-programme level was limited to outputmilestones andweak outcome indicators. Project reporting was donor-specific, incomplete and focused on activities and outputs and, over the period covered by the evaluation, less than one quarter of the projects in the portfolio under review had been independently evaluated due to resource limita- tions and a lack of pressure from senior management and Member States. In addition, thisambitiousevaluationhad tobecarriedoutwithavery limitedbudget, whichallowedtherecruitmentofonly threeconsultants forarelativelyshortperiod of time. These challenges were, however, not specific to the Climate Change Sub-programme evaluation. Similar issues were encountered by previous sub-programme evaluations, requiring the Evaluation Office to develop an 98 M.Carbon