We assume this will work… because we assume it should
Approaches to development, and the methods that flow from them, are profoundly shaped by assumptions about people… Assumptions are also made about processes, such as how change happens or how learning takes place. Assumptions are made about what can and cannot be done. All of these shape the nature of the approach and the choice of methods. Where do these assumptions come from? Some are based on experience or sound research and evidence from elsewhere. Others are based on beliefs and values – some of which are based on stereotypes and misinformation. — Rowland 2003
I recently participated in the American Evaluation Association 2013 conference, in Washington DC. The AEA yearly meets have an amazing range of professional development sessions at the beginning and at the end: this year the offer was 60 different 1-day, 2- day or ½-day workshops, in a variety of evaluation-related topics. Needless to say, it was hard to choose just one (they are also quite costly). I finally decided on a 1-day workshop on Assumptions, “Working with Assumptions: Key Concepts and Tools for Program Design, Monitoring and Evaluation”, presented by Apollo Nkwake and Nathan Morrow, from Tulane University’s Disaster Resilience and Leadership Academy. Both presenters have extensive background in global research, evaluation and capacity building.
The objectives of the workshop were for participants (mostly evaluators and program designers who commission evaluations, from diverse fields) to improve our work with assumptions in the elaboration, monitoring and evaluation of projects and programs. My objective was to extract from the session the things that were most applicable to our work in CIAT and CRPs with M&E, Theory of Change and the “search for outcomes”. The definition of assumption used was “something that is taken for granted-or advanced as a fact. To assume is to suppose, to take it as given, to take for granted”. My first “learning moment” came early on, when I had to remember that assumptions are not only about possible obstacles, but also very much about where opportunities for change lay.
The importance of surfacing assumptions
Feyerbend, the “philosopher of science” argues that one cannot do research, or collect any data, without the existence of some theory (explicit/ formal or (often) informal and culturally implicit) or theoretical concepts. Each of these theories is full of assumptions of how we think the world works: assumptions are, according to Nwake “the glue that holds all the pieces [of a theory of change] together”. Theory-based approaches to planning and M&E elaborate the sequence of changes/mini-steps that connect activities to final impact, but they often fail to help program managers, designers and evaluators to articulate key assumptions clearly. Challenging also is the fact that, in complex programs and projects such as ours, there are as many of these different, and often conflicting, ideas about how change happens, as there are stakeholders. This lack of clarity on how the change processes are underpinned by our many assumptions pose a significant threat to program success, confounds the learning processes, and makes it challenging to monitor and evaluate them. A first step to smooth the way is to ensure discussing assumptions becomes a standard in our theory of change/ PIPA/ Outcomes mapping and similar processes.
The presenters discussed their assumptions typologies, or ways for a project to organize and classify assumptions. A few of them are below:
- By paradigm- the main categories here are whether they are ontological or epistemological (based on our perception of reality and of what constitutes a “correct” description of reality), or axiological/ prescriptive- all about values– (based on what we think is “right”, what “should be”) or causal/ predictive (based on how we think different parts of the world work and about the conditions under which these can be changed).
- By scale of articulation- whether they are ambiguous, tacit, informally explicit, explicit or explicit and tested. By the way, one of the objectives of doing theory of Change/ program theory exercises is to move assumptions from ambiguous and tacit to formally explicit, and hopefully at the end of a program phase, to tested.
- And the one we most commonly use, one by “stage” of theory of change.
This model (figure 1) was adapted from John Mayne’s Contribution Analysis. For theory of change, program theory, PIPA, Outcomes Mapping or project planning exercises, this approach is very valuable, as it facilitates eliciting and organizing assumptions at the causal/ transformational/ impact pathways (including design of strategies and products) and the external levels. And although we do already make use of this model, it was good to remember the articulation of assumptions is all around the “arrows” of a theory of change model, that is, the ways in which activities reach end results, or impact. We regularly make so many assumptions around how things like “reach”, “enhance”, “contribute to change”, “influence” happen!
Inquiry into Assumptions
This basic and necessary level of inquiry can be well complemented by looking into other important assumptions we miss sometimes:
At the “problem diagnostic” level:
- What are the Root Causes of the focus problem?
- What factors interact in creating/ sustaining the problem?
- For whom is this a problem? Is it a problem for all involved?
- What levels of the problem will this intervention address?
At the stage of design of an M&E framework:
- Are these indicators valid and reliable measures of outcomes?
- What design will be the most appropriate to measure?
- How will the data be used?
- How precise and objective is the outcome measurement expected to be?
And, more in general, questions to surface assumptions regarding a whole theory of change:
- When you look at the total picture, do you believe that the theory makes sense?
- Is there anything going on in the real world that may make it difficult to get this theory off the ground the way we’ve planned it?
- Is this theory of change PLAUSIBLE? Have we created a compelling story about the pathway of change that would lead to the long-term goal in this community?
- Is this theory of change FEASIBLE? Do we have the capacities and resources to implement the strategies that would be required to produce the outcomes in the pathway of change?
- Is this theory TESTABLE? Have we specified how success will be measured clearly enough that we can recognize progress toward our goal when we see it? Have we defined indicators for each outcome in clear terms that a researcher or evaluator can use to produce a research plan?
Some remaining questions
How far can we/ should we elaborate assumptions in the field of agriculture, complex, interconnections, paradigms, etc? How far can we extrapolate assumptions?
Whose assumptions matter? Which assumptions matter?
When is it essential, and when only useful, to discuss assumptions?
And my last bit of callousness: is making assumptions explicit actually useful in a pragmatic sense? That is, even if we find serious assumptions that will affect the probability of (egalitarian, sustainable) impact of a project- will that change drastically the formulation of the project? What aspects can change?