|
■ Director’s Corner ■ Recent Evaluations ■ The Evaluator's Perspective ■ Events and Presentations ■ The Sustainable Development Goals (SDGs) adopted
|
|
The Sustainable Development Goals (SDGs) adopted by UN General Assembly in September 2015 create new implications for evaluation in the Post-2015 World. The evaluation community is advocating for building evaluation in SDG processes, moving beyond indicators and monitoring, towards timely evaluative feedback and better understanding of what works, under what conditions, and what causalities foster or impede the achievement of these goals. Watch EvalStory featuring Juha.
The GEF IEO has been contributing to these processes in various fora. During the European Environmental Evaluators Network Forum, we talked about the need to place climate and environmental policy in the big picture of economic and social issues. At the Global Assembly of the International Evaluation Association (IDEAS) and the Fourth International Conference on National Evaluation Capacities (NEC), we reflected on how the SDGs necessitate a fresh look at how environmental sustainability and longer-term inclusive and equitable development are interlinked and can be assessed and evaluated. At the IDEAS/NEC Conference we also brought a specific focus on evaluating sustainable development in Small Island Development States (SIDS) as they are facing unique and often severe challenges to sustainable development. Evaluating sustainable development in the SIDS must take into account the economic, social, and environmental dimensions, while dealing with considerable risk and uncertainty caused by global climate change.
The GEF IEO will continue to adopt a systems perspective to assessing environmental impacts and links to socio-economic aspects, as recently done in the Impact Evaluation of Protected Areas. The benefits of using remote sensing and GIS, combining mixed methods and more traditional approaches were realized in this evaluation and the future evaluations that feed into the Sixth Comprehensive Evaluation of the GEF (OPS6) will be built upon the experience of this innovative evaluation.
I hope you will enjoy reading about this and other IEO evaluations. Happy New Year! |
We bid a fond farewell to Aaron Zazueta who retired last month after 13 years with the GEF IEO. Since joining the GEF IEO in 2002, Aaron has led performance, thematic, and impact evaluations. In 2010 Aaron was appointed as the Chief Evaluation Officer and the team leader on impact evaluations. Aaron led and managed several thematic evaluations, and introduced several new methodological approaches in the evaluations. His most recent contributions include the Impact evaluation of GEF Support to Protected Areas and Protected Area Systems, Impact Evaluation of GEF Support to Climate Change Mitigation, the Impact Evaluation of the GEF in the South China Sea and Adjacent Areas, Assessment of Quality at Entry of Arrangements to Measure Impact, Evaluation of the Scientific and Technical Advisory Panel (STAP) of the GEF, and Progress Towards Impact Study in the Fifth Overall Performance Study (OPS5).
Thank you Aaron for all your contributions to the GEF IEO!! |
The 49th GEF Council Meetings were held in October 2015. Juha Uitto presented the semi-annual evaluation report (SAER) which included the key findings from: the GEF IEO / UNDP IEO Joint Impact Evaluation of the GEF Support to Protected Areas and Protected Area Systems, the Morocco Country Portfolio Evaluation, the Knowledge Management Needs Assessment, and the review of the Climate-Eval community of practice. Separate presentations on the Protected Areas Impact Evaluation by Aaron Zazueta, and Knowledge Management by Geeta Batra, were well received by the Council and stimulated a good discussion. Following the discussion of accreditation of new agencies, the Council requested IEO to conduct a survey across GEF Partner Agencies and recipient countries on the current structure of the GEF Partnership and present a preliminary analysis in June 2016. The concept note for this review is available here.
SAER and supporting information documents
|
The most comprehensive evaluation of its kind, the Impact Evaluation of GEF Support to Protected Areas and Protected Area Systems combines qualitative, quantitative, and spatial evaluation approaches to both data collection and analysis. It draws on evidence from global databases on protected area management effectiveness tracking tools, species population time series, and remote sensing of forest cover, but also draws on in-depth evidence from visits to seven countries (Vietnam, Indonesia, Kenya, Uganda, Namibia, Colombia and Mexico) and 28 protected areas. The evaluation was done jointly by the GEF and UNDP Independent Evaluation Offices in collaboration with the Global Land Cover Facility at the University of Maryland, the Institute of Development Studies, the IUCN WCPA-SSC Joint Task Force on Biodiversity, the National Commission of Biodiversity and Knowledge in Mexico, and NASA.
This impact evaluation concluded that loss of global biodiversity continues at an alarming rate due to multiple development pressures. The GEF support is contributing to biodiversity conservation by helping to lower habitat loss in PAs. The GEF support has helped to build capacities, mainly in areas of PA management, support from local populations, and sustainable financing. However, sustainable financing of PAs remains a concern. The GEF support is contributing to large-scale change in biodiversity governance in countries, including legal frameworks that increase community engagement. The combination of the three key elements allows higher likelihood of broader adoption, including long-term investment, financial sustainability, and creation of links across multiple approaches, stakeholders and scales.
A webinar about the key findings was held on November 10, and generated interest from over 100 participants. The recording, as well as evaluation report and other documents are available by clicking on the links below.
Summary Evaluation Report, Full Report and Approach Paper
Recorded webinar:
▪ Presentation (Juha I Uitto and Aaron Zazueta, GEF IEO)
▪ Panel Discussion ( Gustavo Fonseca, GEF Secretariat; Nigel Sizer, WRI; Aaron Zazueta, GEF IEO)
▪ Questions and Answers (moderated by Juha I Uitto, GEF IEO)
|
The GEF created the Small Grants Programme (SGP) in 1992 with the aim of developing community-led and -owned strategies and technologies for reducing threats to the global environment while addressing livelihood challenges. Implemented by UNDP, SGP has provided over 18,000 grants to communities in more than 125 countries. In 2008, the SGP was jointly evaluated by the independent evaluation offices of the GEF and UNDP. That evaluation was crucial in shaping the way forward for the SGP.
This second joint GEF-UNDP evaluation of the SGP covers the period 2008 to 2014, with a focus on the fifth SGP operational phase, which began in 2011. The evaluation assesses the extent to which the most important recommendations and related GEF Council decisions emerging from the 2008 evaluation have been implemented, the factors that have affected their implementation, and the extent to which these recommendations and Council decisions remain pertinent in light of current and future circumstances. The evaluation also reports on the SGP’s current role and results in terms of effectiveness in achieving global environmental benefits while addressing livelihoods, poverty, and gender equality; broader adoption of grant-level results; the SGP’s strategic positioning; and monitoring and evaluation.
Summary Evaluation Report, Full Report and Terms of Reference |
As part of our efforts to enhance the learning from evaluations and to better engage with stakeholders, GEF IEO conducted Knowledge Management Needs Assessment. The study had two main objectives: 1) an assessment of the use of IEO evaluations by different groups of stakeholders; 2) an assessment of knowledge needs and preferred forms and modes of communication to increase the use and influence of IEO evaluations.
We would like to thank our stakeholders including--GEF Council members, GEF Secretariat, STAP, GEF Agencies, country operational and political focal points, GEF CSO Network, Secretariats of International Environmental Conventions, and external partners-- who participated in the needs assessment. Altogether, 820 people responded to online surveys and 50 people participated in semi-structured interviews and discussions. We were very encouraged by the results on overall satisfaction and use of our evaluations. In terms of overall satisfaction, more than 90 percent of respondents were satisfied with the relevance, quality, usefulness, ease of understanding, and timeliness of our evaluations. In terms of usage, more than 80 percent of respondents reported using evaluations, at least to some extent. The findings indicate that IEO evaluation reports contributed to decision-making, projects/programs preparation/adjustment and for improving understanding of environmental issues, GEF projects, programs or processes.
The results also pointed to areas we need to do more and better. We recognize the need for us to focus on drawing good practice and lessons from evaluation reports in different areas such as focal and cross-cutting areas, project design and implementation, scaling up and impact, and comparisons between integrated and conventional approaches. We will follow through on these suggestions and develop synthesis products that focus on good practice and lessons learned on priority themes identified by the assessment, and use a variety of dynamic and static communication channels to engage with, and inform our partners and stakeholders.
Report and presentation. |
The study, conducted on behalf of the Climate-Eval community of practice, provides an overview of the current methodological challenges in evaluation of NRM interventions that involve climate change aspects. It analyzes the typology and design of evaluations, the issues of time horizon and spatial scale pertinent to such interventions, the issues of M&E systems quality and data quality and credibility, the issues of valuation of natural resources, and evaluation utilization. The study also identifies opportunities to strengthen evaluation methodologies, including the use of the theory of change approach, mixed methods and retrospective stocktaking that offer another way of understanding complex cause-and-effect relationships.
Report
|
The Rio Conventions, for which the GEF serves as financial mechanism, recognize the important linkage between gender-related issues and achievement of International Environmental Conventions’ goals and objectives. The 2014 GEF Gender Equality Action Plan (GEAP) aims to operationalize the GEF Policy on Gender Mainstreaming. The GEF-6 focal area strategies incorporate gender responsive approaches and indicators, and the related Project Information Form (PIF) now requests information on how gender consideration will be mainstreamed in project preparation. Irrespective of the replenishment period, financial mechanism or focal area of a project, program or portfolio of projects and programs, there are some basic evaluative questions that can form the starting point for evaluating gender. The draft “Addressing Gender in Evaluations - Guidance for Approach Papers” provides a selection of guiding questions for consideration.
The depth of focus of GEF IEO evaluations is informed by GEF strategies, policies and processes regarding the topics and focal areas being evaluated. The GEF has already indicated that adjustments may be made to the gender indicators based on initial implementation experiences. Other changes to the GEF Policy on Gender Mainstreaming or the GEAP depend on requests from Council when reviewing progress of GEAP implementation. The IEO’s guidance note on addressing gender in evaluations should as such be seen as a ‘living document’, and will be updated if changes in the GEF gender mainstreaming policy and plans demand such an update.
|
IEO and the Climate-Eval community of practice hosted a panel session at the IDEAS Global Assembly. The session, chaired by Anna Viggh, GEF IEO, focused on Climate Change Monitoring and Evaluation. Christine Woerlen, Arepo Consult, presented the Climate-Eval study on evaluating climate mitigation interventions, which was followed by the presentation by Takaaki Miyaguchi, Ritsumeikan University, Kyoto, of a practical example applying the Theory of No Change. Dennis Bours, GEF IEO, presented the Climate-Eval study on good practice principles for the development, selection and use of adaptation-focused indicators; and Dickson Gumisiriza, Provide and Equip, discussed key policy issues affecting climate change adaptation in Uganda. Presentations were kept concise, providing 45 minutes of discussion between panelist/presenters and the audience, resulting in a lively discussion that also touched upon topics beyond the presentations discussed – like the use and abuse of evidence in policy making and how to develop and apply a theory of change for smaller activities in the climate change arena.
More about GEF IEO at the IDEAS Global Assembly |
On November 24, 2015 Andy Rowe from ARCeconomics presented at an IEO webinar on Rapid Impact Evaluation (RIE). RIE offers the potential to evaluate impacts in both ex ante and ex post settings providing utility for developmental and formative evaluation as well as summative settings. RIE triangulates judgments of three separate groups of experts to assess the incremental change in effects attributable to the intervention. Two methodological innovations are central to the method: the scenario-based counterfactual and a simplified approach to measuring change. RIE was developed by evaluation practitioners to respond to gaps where existing methods are generally too costly slow or challenging for many interventions.
Presentation and recorded webinar. |
|
|
|
|
