Assessing impact by using randomized control trials, is it worth?
Social scientists are always innovating and developing more rigorous scientific tools that help to assess impact, understand social behavior and help policy makers to take right decisions. Although, in agricultural sciences social researchers face different challenges to assess impact. The hardest problem, as in other science fields, is to identify the right counterfactual to assess impact of the intervention. We are not able to see what could it happen with the same beneficiaries if the intervention has not being implemented. Depending on the counterfactual we could have positive or negative impacts (figure 1).For example, distributing an improved variety will increase yield and income for adopters but what could happen if the same group of producers did not adopt the variety, would they be worst-off or better-off. We cannot be sure about that because there are several other factors (climate, social networks, motivation, etc.) that can influence the result. Therefore we are not able to identify the pure impact effect of our intervention.
As a result to these needs a group of scientist is promoting the implementation of Randomized Control Trials (RCT) as an option to assess impact in more scientific rigorous ways. The Poverty Action Lab (J-PAL) is a network of 49 affiliated professors around the world who are united by their use of Randomized Evaluations to answer questions critical to poverty alleviation. Esther Duflo founder and director of the J-PAL has done an impressive presentation resuming RCT importance and key results (Social Experiments to Fight Poverty, worthwhile watching the 15 minutes video).
Several CGIAR’s center like CIAT and CIP are now interested on implement this type of methods in their research. For instance, CIAT researchers participated on the J-PAL workshop done In Bogota that was aim to build capacity to conduct randomized evaluations. Scientist from different Latin American (LAC) countries and several research institutions from private and public sector participate on the workshop allowing CIAT to improve their networking in LAC and benefit from new methodologies. Also an internal seminar at CIAT was carried out with several social scientists of the center to spread the tools and information of randomized evaluations. The internal seminar was useful to criticize and discuss the main restrictions on applying this methodology on the Agricultural Research. For example, cost effectiveness on carry on this type of exhaustive evaluation, the ethic implications of not providing the intervention benefits to all producers, the difficulties to keep a pure control sample without the influence of the intervention and its spillovers and several more.
Therefore there are still several doubts about applying this type of methodology and a lot of work to be done to implement this rigorous scientific evaluation. We thank the Abdul Latif Jamel Poverty Action Lab and their initiative to translate research into action. The material presented here is part of the workshop develop in Universidad de Los Andes.