Page - (000066) - in Evaluating Climate Change Action for Sustainable Development

important informationonexpected “progress toward impact” (vandenBerg2005) andwhether the conditions have been set to enable longer term impact. The last columnpresentsexpostevaluations5–8yearsafter theprojecthasended.Theseare almost invariable historical evaluations, using a historical approach to trace whether the results of the project have contributed to observed changes in trends, markets, societies, economies and the environment. These evaluations tend to advertise themselves as “theory of change” oriented and using mixed methods and triangulation of evidence to come to conclusions on impact. They have less ofaproblemtomovebeyondcountries to regionsand theglobal level, notbecause the evidence is stronger after 5–8 years, but because they are more flexible in approach and aremore pragmatic and adaptable in using data sources and linking data where possible. This sounds opportunistic, but there are many scientifically soundmethods and tools that canbe combined and triangulated, as amplydemon- strated byStern et al. (2012) andGarcia andZazueta (2015). 3.6 UsingTimeandScale to IdentifyApproaches Anothercut-throughofthethree-dimensionalmatrixof time,spaceandscalewould betocombinetimeandscale.Figure3.2presents thismatrix.Thetimedimensionis of course the same as in the time-spacematrix, but has been simplified a bit, for example presenting one row for ex-post rather than two. The scale dimension provides various perspectives of scale. From interventions focused on one causal mechanism, such as a project focusing on changing customer behaviour on the energymarket throughpricesetting, tomultiple interventionswithinoneproject,of forexamplepublic-privatepartnerships, social changemovements, capacitydevel- opment efforts, to aperspectiveonanenablingenvironment that through rules and regulations, taxation,knowledgedisseminationandother incentives tries toredirect amarket or change behaviour, tomarket change and transformation, the interven- tions becomemore complex and challenging to evaluate. At the far right I have included climate change, and again this environmental scale overlapswith others, posingaspecialproblemthat twoevaluendsneed toberecognized inanevaluation that includes environmental objectives (seeRowe2012). Againwe see randomized controlled trials and quasi-experimental approaches focusingmostlyononeintervention,as tocontrol forcombinationsof interventions willbecomeverycostly.Exanteresearchwilldelivercounterfactualassessmentsof howdifferent scenarioswill performat all scales.Arelativelynewmethodsuchas QualitativeComparativeAnalysis (QCA) iscurrentlyoftenused forcasestudiesof more complex interventions and the enabling environment.Markets are of course the subject of economic researchand for evaluations especiallymarket research to assesswhether anewproductorapproachhasachanceon themarketdominates in the market columns and ideally before the new intervention starts. At the programmeandpolicy levels, expost impact evaluationsmay lookat triangulation 46 R.D. vandenBerg