Comments
Dear Eddie, dear colleagues from the Secretariat,
We appreciate very much your reaction to the findings of our GIZ study on EITI’s effectiveness and impact – you are preaching to the converted. We totally agree with the great relevance of the EITI to identify risks and weaknesses of extractives regimes. This is exactly why we are keen that this relevance is understood widely across all stakeholders beyond our inner circle.
Hence, based on our and, as the process of the study has shown, the understanding of many other stakeholders of what constitutes a systematic monitoring and evaluation framework we still defend this key finding, which you criticize. I reckon we all agree that a discussion of EITI stakeholders is continuously needed on national level to take the next steps from identifying weaknesses and risks towards actually overcoming them, e.g. ”from reports to results”, from transparency to accountability.
However, if I understand your reaction to the findings of the study correctly we possibly agree to disagree that the EITI needs a (more) robust and systematic M&E (first and foremost at national level – but also at global level). This is an essential instrument to convince stakeholders beyond our “transparency club”, that we are successfully changing peoples’ life for the better. We need to be aware that the EITI is still largely depending on taxpayers’ money from OECD countries. Public funding is limited and if you have to choose between paying for an EITI report or paying 20 teachers legitimate impact data becomes an issue.
We believe the development of common M+E standards (including a joint definition of inputs, outputs, outcomes and impacts as well as their interdependency) across countries will facilitate a global M+E system as the data collected at national level can be aggregated globally. Also we believe that the legitimacy of the evaluation framework will be increased by moving away from the endogeneity of some of the EITI tools you mention. This will go a long way of enlarging our “transparency club” beyond the converted.
You are right, the study GIZ has commissioned and its findings are no more and but also no less than one impulse into that important discussion. We hope that other stakeholders of the EITI and the extractives sector as a whole find the results of the study useful and feel encouraged to have a go on establishing the impact of the EITI – please feel invited to add your views here or contact us directly with any questions/comments!
Looking forward to discuss this further and get more input from all sides. This debate alone is another proof how lively and engaged the EITI movement really is and I am proud to be part of it.
Yours sincerely,
Kristian
Kristian Lempa
Senior Project Director
Extractives for Development – X4D
Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) GmbH
Hi Lutz, Eddie, and Kristian,
Thank you all for starting this discussion and congratulations to BMZ and GIZ for this insightful and thought-provoking report. I also think the EITI Secretariat is to be commended for their openness and responsiveness to external studies, even when their findings are critical of aspects of EITI.
This discussion touches on some very important questions about EITI. Although the issue of EITI’s monitoring and evaluation frameworks is an important one in terms of measuring the initiative’s outputs, I also think the study raises additional questions about the ultimate outcomes of EITI’s work, including:
- What are the overarching goals of EITI? Can—or should—participating stakeholders agree on a set of shared global objectives, or should EITI’s goals instead reflect individual country contexts?
- Can—or should—EITI’s progress towards these goals be measured?
To explore these and other questions raised by the report, BMZ, GIZ, MSI Integrity, NRGI, Publish What You Pay, and the World Bank will be co-hosting a discussion at a side event at the EITI board meeting next Wednesday, October 26th. We hope this event will catalyze further discussion and reflection on EITI’s accomplishments and impacts. Please feel free to post potential questions for panelists to this thread or email them to me by next Tuesday the 25th at ben@msi-integrity.org as we hope to include a broad range of perspectives at the event.
Best,
Ben Collins
MSI Integrity
Dear Kristian (GIZ) and all partner organizations,
I wants to on behalf of the Africa Youth on Mining and Environment (AYME) starts by thanking GIZ and partners organizations for doing such a research on the EITI and coming up with very critical findings.
The AYME also assert to the fact that EITI’s monitoring and evaluation frameworks is a very important issues regarding the impact of EITI in the extractive industries in the implementing countries particularly in Africa. We strongly believe that all efforts of the EITI ranging from transparency to accountability is about improving the prudent management of the extractive industries. AYME we believe that EITI's transparency and accountability goes beyond reporting what is paid, received and explaining how it was used in theory, but instead, realistically reflecting on the livelihood of the people particularly the mining host community people and the national population of the extractive countries. In other words, changing the status from a low wellbeign to a medium or high wellbeign of the people.
In our opinion since the inception of the EITI this aspect has still not reached in most extractive countries, a case study is post war Sierra Leone were the AYME activities are more visible. Instead the "political aspect" of EITI is what has being archived which are pronouncing what is paid, what is received and creation of mining data tracking, EITI countries Champions and Secretariats, MSG, (Structures). But the "economic aspect" of the EITI is still not visible which to us is the most important. Thus, AYME has the following recommendations.
1- EITI’s goals should reflect on individual country contexts as country dynamics in the extractive industries remains varies.
2- Monitoring and Evaluation system goes beyond what is paid and received or how it is used, but also what impact it creates on the economic wellbeign of the population and the governments' budgets.
3- How can Youth participation be visible in the MSG as over the years youth involvement and contribution remains an big gap in the prudent management of the extractive industries of Africa. AYME is making that efforts now but the plain level ground is not given to us as youth promoting the good governance of the extractive industries. So the EITI International should start this process for Youths and it can come down to country context.
Thanks for this interesting report and for letting us contribute to its development. Broadly we agree and are pleased with its conclusions that, although evidence from quantitative data was insufficient, the EITI is highly relevant and much needed, and has contributed to identifying risks and weaknesses of the extractives regimes. However, we find its central argument that “there is no systematic monitoring and evaluation framework in place” to be incorrect. These points echo very much what we submitted in the attached related to the first draft.
The report is right that the demonstration of impact at the international level is a challenge for the whole EITI enterprise. The challenge is set out in a blog by Jonas Moberg. There are limitations in establishing “causality”, measuring moving targets, the challenge of different stakeholders wanting different outcomes from the process, recognising the huge diversity of country processes and objectives, acknowledging the wider impact on the transparency and accountability agenda, etc.
That is why the systematic monitoring and evaluation framework has to be established first and foremost at the country level. A global results framework would be overstretched to be relevant in Dili and DC, or Kinshasa and Kabul, or Myanmar and Mozambique. Therefore, as the report suggests, for such a varied tool as the EITI, the M&E process needs to start at the country level.
At the country level, the EITI has an in-build robust system for monitoring and evaluating progress. Each year each country is required to update its costed workplan annually to reflect the evolving objectives and actions of the EITI process in each country. This is required to reflect how the EITI relates to progress.
At the end of each year, the country is required to produce an Annual Progress Report to reflect on and monitor the progress against the objectives and actions set out in the workplan and any wider impacts.
Every three years, the country is required to undertake a validation. Not only does this process access the country’s progress against the EITI requirements, including whether it is compliant, it also makes an impact assessment – an evaluation of the process.
Collectively, these tools ensure an in-country planning, monitoring and evaluation process designed to improve delivery. Therefore, it is wrong to suggest that there is nothing systematic. It is hard to think of any other international process with such a robust, systematic, in-build M&E system.
The report is obviously more concerned that this is difficult to collate at the global level. Recognising that this diversity and country-led implementation cannot be simply captured by quantitative analysis alone, since 2007, the EITI Board has monitored the results achieved through the implementation of the EITI Standard through:
- its regular implementation and outreach progress reports
- annual progress reports
- the validation process
- its key performance indicators
- overall evaluations.
In 2015, the EITI Standard became fully operational (i.e. all countries report using the Standard) after a period of transition since it was enacted in the 6thGlobal Conference in May 2013. Over forty countries have published over 300 EITI reports. New features such as contract transparency, disclosing information on licensing, budget distribution and expenditures are now regularly included in EITI reporting. Together with other aspects such as timeliness for publishing reports, data readability, subnational reporting, there is now a richer amount of more timely, reliable and accessible information that allows monitoring the work of the EITI more thoroughly. Similarly, the format and ways in which the EITI is delivering outputs, including communication tools, is changing. It is expected, and desirable, that EITI information is increasingly delivered in digital format (open data, portals, and websites).
The progress on this evolving implementation across the 51 implementing countries is captured in the implementation progress reports submitted to the Board about three times a year. Progress on the outreach to another 20 or so countries is captured in the outreach progress reports on the same cycle. These documents are internal to the Board.
In addition, the EITI provides a public annual document which seeks to capture progress internationally and across the 51 countries, as well as other administrative information reflecting the progress against the Secretariat’s annual workplan and budget (which are both approved by the Board).
The International Secretariat has established these sets of indicators aimed at addressing three different aspects:
1. Agency effectiveness (i.e. the EITI). This responds to the “value for money” question. The indicators to be monitored for this relate to the inputs and outputs to/from the EITI as an agency (mostly the
International Secretariat as the supporting unit of the International Board, the decision-making body).
2. Attributable outcomes. This responds to the question of what concrete results have been achieved, totally or partially, as the result of the agency’s activities, products, interventions and steering.
3. Big picture indicators. Although not attributable to any single organisation (not the least, to the EITI Board and Secretariat), selected proxy indicators in areas such as investment climate, human capital spending, corruption, poverty, all related to goals sought by the EITI, are aimed to shed light on general context and the direction of change.
In addition, the EITI has undertaken two major evaluations:
- Rainbow Evaluation of the EITI’s impact on the transparency of natural resource revenues (2009).
- Scanteam evaluation of the EITI in 2011.
Plus an evaluation of the EITI technical assistance and funding mechanisms in 2015. Many other non-commissioned evaluations and research pieces also exist on the research page on the EITI website.
The EITI has made significant contributions to improved governance of the extractive sector in several countries around the world. In countries like the Democratic Republic of the Congo, the EITI has been central to many reforms of the sector. At the international level, debates on transparency in the sector are unrecognisable from ten years ago, and the EITI is seen as being at the forefront of many frontier debates including beneficial ownership, commodity trading, and artisanal and small-scale mining.
It is also clear that the EITI process is one of the only functioning global mechanisms to inform and channel debate in resource-rich countries in a way that includes all stakeholders.
In the development business, there can often be a naïve belief in figures. While it is important to have quantitative information, it is even more important to have the right kind of information and to have it used. We need more serious analysis, in words and numbers, of the information out there. The GIZ report is an important contribution to that, but it is wrong - or at least overly crude - in one of its key findings.
Best
Eddie Rich
Deputy Head
And Regional Director for Africa and Middle East
EITI International Secretariat
Extractive Industries Transparency Initiative