Managing evidence: a new actor connecting evaluators and policy makers
Version en Español (Link)
Last Wednesday at the ReLAC Conference we received a reality check on what is the information that is most relevant to policy makers when reading an impact evaluation. According to the Ministry of health of Peru, Aníbal Velásquez Valdivia, the only thing read from an impact evaluation: recommendations. He told us about his former experience as an evaluator, writing big introductions and long methodology sections because he thought that made his research credible in front of the reader, the policy maker. However, once on the other side he realized that the only thing he needs from a study of this type is a solid and useful recommendations section. Once he understands that the programme being evaluated is not going as it should and is flawed, he is only interested in evidence-based information on how to make it better. He concluded that there is no need to put all the effort in identifying the problem, as the focus should also be on finding solutions.
Despite its importance, the recommendations section is often the weakest part of an evaluation study, where the researcher sometimes makes big leaps from the results of the study or recommends very general things like ‘the programme needs to be refocused’. Partly joking and partly serious, he touched on a very relevant issue which he calls “the need for a new actor”: a bridge between researchers and policy makers, a group of people that can process scientific information and turn it into useful recommendations for policy makers. Velásquez Valdivia explained that scientific recommendations often are not practically viable. It is the role of this new actor, then, to take the recommendations and analyse them through a socio-economic and political lens. These need to be people who understand not only public policy but also rigorous research and who can strengthen the presently weak communication link. The ministry calls for the creation of a knowledge management actor who makes scientific information from impact evaluations viable for policy makers who have to make decisions.
Velásquez Valdivia also stressed that different types of studies (therefore different evaluations) are needed at different times of the political cycle: a policy maker at the beginning of a programme needs different information from what he needs during programme implementation, or when he is close to the end of his mandate. The political cycle therefore has to be taken into account by evaluators who want to give relevant information to policy makers.
In programs or projects which are looking to influence politics, it seems a priority to adapt how we produce and communicate information to policy makers with appropriate timing to achieve this objective.
These issues pose new challenges for evaluators who aim to influence decision-making:
– How to reconcile the slower pace of rigorous research with the timing of decision-making in public policy?
– How to create effective groups that work together to provide evidence, recommendations and propose solutions?
By Fanny Howland and Genowefa Blundo Canto