IMPACT EVALUATIONS CRUCIAL FOR EFFECTIVE PUBLIC POLICY:
Courtesy: Asian Development Blog- See more at: http://blogs.adb.org/blog/impact-evaluations-crucial-effective-public-policy#sthash.ZGxzffSZ.dpuf
Asian countries are increasingly turning to investing in dedicated development programs rather than relying entirely on economic growth to deliver better social outcomes. Evaluations of their actual impact have not always accompanied such decision making, but where they have, it has made a key difference.
Impact evaluations can guide policy and investments based on evidence instead of them being driven mostly by special interests.
Evaluations and policy choices intersect in three respects. First, conducting assessments of policies or investments—for example, a social protection initiative—allows them to be expanded on the basis of the results obtained. Second, they bring out the crucial complementary factors that are necessary for development success—for example, road projects can improve inclusion if they are linked with programs addressing education and health care in the same area. Third, there are emerging avenues of action—for example in climate change, where past experience may not provide a sufficient guide for the future.
Careful evaluations in some social areas have given policymakers grounds to expand or wind down a program. For example, assessments of conditional cash transfers have provided a basis to expand these programs in Brazil, Mexico, the Philippines, and elsewhere.
An impact evaluation of the Philippines’ conditional cash transfer program, which started with 6,000 households in 2008, showed gains in reducing stunting in children, increases in school enrollment and attendance, and increased ante- and post-natal care for women—well worth the program costs. As a result, the program was massively expanded to cover more than 3 million households in 2013.
Impact evaluation initially focused on social programs, where information on beneficiaries, and others before and after a program, is relatively easier to obtain. But it is now expanding into other areas of policy, such as transport, energy, and the environment. All these areas present challenges for analysis, such as separating cause and effect, attribution, and assessing the validity of the outcomes across various locations, beneficiary groups, or policy instruments.
Evaluations have often also highlighted the importance of complementary factors. In many instances, while the intervention did produce results, the gains were limited and full impact was not achieved. Impact evaluations help identify the barriers to achieving better outcomes, or the inadequacies in program design.
One consistent theme has been the limits of poverty-oriented efforts to reach the very poor. An impact evaluation of microfinance in Pakistan and Viet Nam found that there were trade-offs between program impact and targeting of the poor. Pakistan’s program was able to target and include the poor, but the program had very little impact on their welfare. On the other hand, Viet Nam’s mostly non-poor clients enjoyed positive direct impact on income and enterprise employment.
A study on Bangladesh pointed to the gains from rural roads maintenance, but again the benefits missed the very poor. In these and other instances, the presence or investment in related areas such as education, health, and financial literacy proved critical. Rural electrification will not only light communities, but can transform lives if education and income opportunities are promoted.
Among emerging issues, biodiversity and the environment have received relatively little attention from impact evaluation. But where they have been considered, the results have been noteworthy.
An evaluation of forest protection worldwide found that establishing protected areas is effective in reducing the loss of forest cover. But the impact of protected regimes—
rates of deforestation before and after intervention—were sharper in areas closer to communities and economic activities than in settings far from economic activities.
Studies in Latin America found a strong correlation between forest protection and indigenous land tenure and property rights. While protecting pristine forests is equally important, protection of land competing with other economic uses can have relatively high payoffs too.
The application of lessons to new challenges remains difficult as there are limits to what can be inferred from historic data. Using an impact evaluation based on past experience to guide future action presents a tough challenge when the problems are new. Climate change is a case in point where climate models yield various projections with varying probabilities.
These are some of the challenges of applying impact evaluation to guide policy. These and many others are to be discussed at an international conference on 5-6 September cosponsored by 3IE, the Government of the Philippines and the Asian Development Bank, that will review the growing body of evaluative work directed at policy.
