Highlights

Introduction

The field of evaluation is a diverse field that spans different disciplines. Disciplines such as education, psychology, sociology, political science, clinical sciences, and others have contributed with different epistemological perspectives, methodologies, and methods. Some have contributed to niche methodologies, others have contributed to debates across disciplines and advanced researchers’ thinking and practices in their respective fields. Dozens of books exist which present varied evaluation models, evaluation approaches that place emphasis on different aspects, questions, and engagement processes. It would be difficult to find two evaluators who share the same perspective on evaluation’s role and on how to conduct a given evaluation project. This diversity contributes to the richness of evaluation as a field and as a set of practices. Beyond this diversity, evaluators feel they belong to a group (a meta discipline) having a role which consists of offering a critical perspective on interventions with a focus on making the work they do useful.

As an evaluator, I greatly value this diversity. I appreciate being exposed to a multitude of approaches, ideas, and positions, and having the privilege to debate with those who share a willingness to improving our practices and societies—enhancing agility and reflexivity in refining evaluation methods. This book proposes an evaluation model based on the foundation established by many authors who have significantly contributed to the field. I won’t impose a specific role to evaluation or a definition, as I value the richness of the current diversity. Instead, I hope to offer flexible avenues for integrating impacts on planetary health in all our projects, while respecting this diversity of perspectives.

Definitions

Evaluation is probably as old as the human race, dating from the time humans first made a judgment about whether building campfires and using weapons helped them to survive. Indeed, evaluation is an essential human activity that is intrinsic to problem solving, as humans (a) identify a problem, (b) generate and implement alternatives to reduce its symptoms, (c) evaluate these alternatives, and then (d) adopt those that results suggest will reduce the problem satisfactorily. As humans, we will always be faced with problems whose solutions require evaluation so that effective action can be taken. (Shadish & Luellen, 2005, p. 183)

The word evaluation has the same etymology as value. Evaluation is about assessing/appraising the value of things. The Merriam-Webster’s defines ‘evaluation’ as “to judge the value or condition of (someone or something) in a careful and thoughtful way”.

The field of evaluation is diverse and offers different definitions. Some of these are more focused on systematic analysis principles, while others are more versed on identifying shared solutions. For example, the Encyclopedia of Evaluation (Mathison, 2005) offers the following definition:

Evaluation is an applied inquiry process for collecting and synthesizing evidence that culminates in conclusions about the state of affairs, value, merit, worth, significance, or quality of a program, product, person, policy, proposal, or plan. Conclusions made in evaluations encompass both an empirical aspect (that something is the case) and a normative aspect (judgment about the value of something). It is the value feature that distinguishes evaluation from other types of inquiry, such as basic science research, clinical epidemiology, investigative journalism, or public polling. (Fournier, 2005, pp. 139-140)

This definition contrasts with that provided by House and Howe in their book Values in Evaluation and Social Research (1999): 

The aim is for evaluators to use procedures that incorporate the views of insiders and outsiders, give voice to the marginal and excluded, employ reasoned criteria in extended deliberation, and engage in dialogical interactions with significant audiences and stakeholders in the evaluation. (House & Howe, 1999, p. IX introduction)

I personally also like the definition from Mark, Henry and Julnes (2000):

Evaluation assists sensemaking about policies and programs through the conduct of systematic inquiry that describes and explains the policies’ and programs’ operations, effects, justifications, and social implications. The ultimate goal of evaluation is social betterment, to which evaluation can contribute by assisting democratic institutions to better select, oversee, improve, and make sense of social programs and policies. (Mark et al., 2000, p. 3)

These definitions show how much the representation of the evaluative exercise can vary according to the evaluator’s approach and values.

The definition used in the previous book I co-edited and co-wrote (Brousselle et al., 2011b) elaborates on this point:

To evaluate is basically to make a value judgment on an intervention by implementing an inquiry design capable of providing scientifically valid and socially legitimate information about that intervention or any of its components, the objective being to ensure that the various actors concerned, whose perspectives can be different, are able to take a stand on the intervention so that they can construct, individually or collectively, a judgment that can be translated into actions. (Free translation, Champagne et al., 2011c, p. 38)

These definitions are still valid, but are they enough for our new world context? As emphasized in the following collective reflection:

Evaluators need to adopt an open mindset, question predetermined myths, value diverse cultures, and adopt a relational, inclusive, equitable, and respectful approach. Evaluators need to engage with communities, to adopt community-driven approaches, while bringing a holistic perspective in the evaluation (Evans et al., 2020). Evaluators are seen as adopting a broad role where projects, programs and policies, and systems are questioned, where barriers to change are targeted, and where inequities and injustices are identified. They are also people who facilitate conversations in communities, gather and analyze data, and give back results to the community for dialogue and deliberation. They empower communities, support their self-determination, and, within communities, elevate more marginalized voices. They also need to recognize that there will be occasions when it is not appropriate for them to lead an evaluation, that it may be led entirely by community leaders, and instead they are an ally. Furthermore, evaluators are responsible for sharing with the larger community of peers and contributing to social change. Evaluation should be used to initiate and foster social movements, working across isolated evaluation experiences. In this role, evaluators engage in highlighting harm, environmental impacts, and injustices and in taking positions to support action not only for the public good—as was the original goal of the field—but for planetary health (Brousselle & McDavid, 2021). This marks an important change in the ontology of evaluation as it enlarges the purpose of evaluation practice beyond responding to commissioners’ interests, beyond the public good, to an even larger conception of our role, and with a responsibility toward nature, while being accountable to communities. Adopting such an ontology of practice fundamentally implies that the evaluator is standing for the value of evaluation, which can no longer be an instrument of the system but a way to reinforce democratic and community processes: Only then will evaluators have the capacity to contribute to transformative change. (as cited in Brousselle et al., 2024a, p. 188)

Another challenge to consider is the growing polarization in society, which often leads to irreconcilable perspectives. In a post-truth era where facts are contested, can we reach a collective understanding and agreement on what the problem is and its potential solutions? If not, how can evaluators navigate post-truth contexts, and what role do they play? The evaluator’s ability to document and expose various positions remains important. Although the topics covered by the evaluation may not always be directly affected by the polarized context, building some form of shared understanding is still possible in most cases. But what if the topic involves issues where certain groups have irreconcilable positions? How will this impact the role of evaluators? Will their legitimacy itself be contested? These are some of the many questions raised by the growing context of organized disinformation and polarization. We are entering an era of significant uncertainty. Definitions of what evaluation is have evolved over time alongside the field itself. We can expect the role of evaluation to continue changing in the coming years, with new practices shaping future definitions of evaluation.

As a researcher, I value scientific processes, grounded in the belief that a reality exists, and can be partially understood using scientific methodologies. I am not willing to compromise on the existence of certain phenomena that are widely supported by scientific evidence, such as climate change. At the same time, I acknowledge that doubt is a core concept in the construction of scientific knowledge. Additionally, debate plays a natural and important role in knowledge construction. However, I also recognize the existence of irreconcilable perspectives and the need to shape processes accordingly. My position is pragmatic, as I adopt the view that evaluation processes are largely bricolage.

Bricolage goes beyond the notion of mixing methods, or mixed-methods, which is focused on combining different types of data. It includes combining elements from different paradigms and enquiry traditions, and adapting methods and tools as needed. It includes attention to the context of the evaluation in terms of the nature of what is being evaluated and its setting, and the nature of the evaluation and its questions and purposes. Bricolage is more likely to be used in an emergent evaluation design where initial findings and understandings can be used to inform subsequent design. (BetterEvaluation, 2025)

As Margaret Hargreaves (2021) explains:

I see bricolage as a transdisciplinary, mixed paradigm approach for reconciling the different ontological (“real world”) and epistemological (“theory of knowledge”) perspectives that complexity science and systems thinking bring to evaluation. I use bricolage to strengthen the design of complexity- and systems thinking-informed evaluations by weaving multiple paradigms into all four elements of evaluation design: (1) conceptual frameworks, (2) inquiry frameworks, (3) methods and metrics, and (4) evaluation values and valuing. (Hargreaves, 2021, p. 114)

Later in her article, she notes that evaluation bricolage serves an important function, creating value:

A new evaluation approach that is emerging is not about determining value (providing an objective appraisal of the value or worth of a program model based on predetermined criteria), but about contributing to change by creating value (engaging proactively in ongoing communication with multiple stakeholders about evaluation findings while paying attention to the organizational, institutional, and community contexts in which the evaluation decisions are being made, and actions are being taken. (Hargreaves, 2021, p. 122)

In a post-truth era and within the current environmental and social context, which demand radical changes, evaluation can be defined as an engaging process that combines various complementary inquiry approaches to support decision-making, while also contributing to sense-making, value creation, positive ecosystems, and social change. Evaluation aims at providing credible information to make informed and socially acceptable decisions about products, projects, programs, and policies –called interventions in this book. Evaluation can help align interventions with community needs, support evidence-informed design of interventions, provide information on what works and what doesn’t according to the context, inform on the larger impacts of interventions, support informed decision-making on allocation of resources, and monitor progress.

Evaluation Approaches

Over the past century, evaluation practices have expanded and diversified a lot. Approaches and methods have evolved. It is a field where one will find a variety of definitions and evaluation questions reflecting the perspectives of those contributing to the field. Evaluation experts often use the same terms but with different meanings, organize and frame evaluation questions in varying ways, and have various ways of representing their evaluation approach. The field has also engaged in major philosophical and methodological debates, gathering thinkers in quantitative, qualitative and mixed methods research, and has made strong contributions to research designs. There are many ways to evaluate interventions, and one needs to be prepared to make some explicit choices and to discuss them. It is a field where debate is welcome. All evaluators consider themselves as constant learners in methodologies, methods, and engagement.

Shadish et al. (1991) contrasted (then) existing evaluation approaches and identified five core dimensions that any evaluation model addresses, albeit differently and with more or less emphasis: use, social programming, knowledge, values, and practice. Inspired by this work, Alkin (2004)’s work titled Evaluation Roots where he offered a tree representation to position and group well-known evaluation approaches. The tree had three main branches (use, methods, and valuing) and two roots (accountability and control, and social inquiry). This representation attempted to classify evaluation approaches in a simple way; however, it lost the fact that each evaluation approach positions itself according to the five dimensions identified by Shadish and colleagues (1991).

In 2023, Mertens published a critique of the tree, emphasizing that Alkin’s (2004) tree did not include Indigenous’ voices and those of people of colour (Mertens, 2023). Mertens proposed the inclusion of Indigenous and transformative approaches that pursue “social, economic, and environmental justice through evaluation” (Mertens, 2023, p. 12). She proposed that evaluation approaches be compared on their assumptions about axiology (assumptions related to the nature of ethics and values), ontology (the nature of reality), epistemology (the nature of knowledge and the relationships between the evaluator and stakeholders), and methodology (the nature of systematic inquiry) to make explicit the believes and values underpinning our approaches (Mertens, 2023).

In 2023, M.Q. Patton emphasized that each evaluation theory addresses values, methods and use, with contrasted configurations of approaches for each of these component, proposing that evaluation theories shouldn’t be conceptualized as a tree, but rather as a forest (Patton, 2023a, 2023b), somewhat echoing the original analysis of Shadish, Cook and Leviton (Shadish et al., 1991).

Interestingly, in the rich discussions following the publication of the evaluation tree, the roots of the evaluation tree (accountability and control, and social inquiry) have attracted less attention. Instead, the focus has gravitated toward the branches (values, use and methods), as though these were more important, overlooking the full set of five core dimensions identified by Shadish and colleagues in their early work (Shadish et al., 1991).

In practice, evaluators tend to apply similar approaches in comparable contexts, reflecting their training and their specializations in the field. Fewer evaluators adapt the choice of approach to the evaluation context. Contandriopoulos and Brousselle (2012) have shown that there seems to be a fit between the characteristics of the context and the appropriateness of the choice of approach. Evaluators seem to be drawn to practice in contexts of similar characteristics, hence using and advocating for the same family of approaches. However, the emerging context of higher polarization, widespread disinformation, and the de-credibilization of scientific expertise creates a need for evaluators to adapt their practice, becoming skilled facilitators capable of using bricolage in the design of their evaluation projects. This new expectation underscores the need to train evaluators in developing their ability to mobilize different approaches according to the characteristics and dynamics of their specific evaluation contexts.

Exposure to diverse approaches occurs through engagement in the field—via readings, participation in training events, attendance at conferences, and practical application—broadening the range of possibilities in the evaluation process. Understanding the principles of diverse approaches—and their suitability for different contexts of evaluation—enhances effectiveness and adaptability as an evaluator. This comes with time and experience by reading different authors, being exposed to different currents of thought, and working with different evaluators on diverse projects.

When to Evaluate and for What Purpose?

Evaluation can be used for a multitude of purposes and at different stages of an intervention.

Evaluation is often presented as a step in the policy cycle, taking place after the implementation of an intervention, intended to provide information to guide revisions as the cycle begins again. This representation is outdated and misleading. It doesn’t account for the vast potential of evaluation to contribute to policymaking at all stages of an intervention. Instead of sitting in the policy cycle between implementation and revisions, it is more appropriate to locate evaluation in the centre of the cycle, informing policy and program design, implementation and revisions.

Before an intervention is developed, evaluation can be used to identify needs and find the best fit for the contextual characteristics. Or, it can support the design stage of an intervention by identifying what would work best in a particular context, based on the scientific literature (logic analysis). Evaluation can be a process to optimize the design and implementation of the intervention in dynamic contexts, as in Developmental Evaluation (Patton, 2011), or to achieve transformational objectives as empowerment, emancipation, or advocacy (Brousselle & Champagne, 2011; Mertens, 2023). It can be used as a formative exercise to improve an intervention or its implementation, or as summative exercise to decide whether a program merits investment (Mark et al., 2000). It can guide decisions on what interventions to collectively support or stop funding, as in spending reviews or economic evaluations, for example). Evaluations “can be undertaken for the express purpose of supporting or building the image of a program” (Palumbo, 1987, p. 12). They can also contribute to the generation of generalizable knowledge (Brousselle et al., 2011b; Mark et al., 2000; Patton, 1997). These evaluative purposes are not exclusive, and an evaluation project can contribute knowledge for different purposes at different stages (Mark et al., 2000).

Evaluative Questions

Evaluation is at the intersection of research and policy making/decision-making. A distinction is generally made between evaluative research and monitoring (Brousselle et al., 2011b). Evaluative research analyzes causal relations, specifically the links between the components of the intervention, including the results. Monitoring compares components and aspects of the intervention to some objectives or targets and to record progress. Monitoring provides information on the resources dedicated to the intervention and its structure, the activities (also called process), and the results.

Several questions exist in evaluation. Although authors present and regroup evaluation questions in different ways, most models include the same core questions. The typology and definitions used in this book are inspired and adapted from what was suggested in L'Évaluation: Concepts et Méthodes (Brousselle et al., 2011b). Planetary health considerations have been introduced in each one of them (see Figure 4.1 and the following chapters).

Figure 4.1 Evaluation Questions

Evaluative Research Questions

Evaluative research includes questioning in five categories (see Figure 4.2):

Figure 4.2 Evaluative Research Questions

Source: Adapted from Champagne, F., Contandriopoulos, A., Brousselle, A., Hartz, Z., & Denis, J. L. (2011c). L’évaluation dans le domaine de la santé : concepts et méthodes. In A. Brousselle, F. Champagne, A.-P. Contandriopoulos, & Z. Hartz (Eds.), L’évaluation dans le domaine de la santé : concepts et méthodes (2nd edition ed., pp. 35-56). Presses de l’Université de Montréal: 62.

Monitoring Questions

Monitoring is used to assess the fidelity of the intervention in comparison to the original plans (which includes the costs), the reach, the quality of the intervention, whether the goals were achieved, and the impacts on planetary health. Table 4.1 summarizes the different options for monitoring the intervention.

Table 4-1 Possible Focus for Monitoring

Source: Adapted from Champagne, F., Hartz, Z., Brousselle, A., & Contandriopoulos, A. P. (2011d). L’appréciation normative. In A. Brousselle, F. Champagne, A. P. Contandriopoulos, & Z. Hartz (Eds.), L’évaluation: concepts et méthodes. Deuxième édition mise à jour. Les Presses de l’Université de Montréal: 98.

Note that monitoring questions focus on the intervention’s components, whereas evaluative research questions examine the relationships or links between those components (see Figures 4.2 and 4.3).

Figure 4.3 Focus of monitoring

Source: Adapted from Champagne, F., Contandriopoulos, A., Brousselle, A., Hartz, Z., & Denis, J. L. (2011c). L’évaluation dans le domaine de la santé : concepts et méthodes. In A. Brousselle, F. Champagne, A.-P. Contandriopoulos, & Z. Hartz (Eds.), L’évaluation dans le domaine de la santé : concepts et méthodes (2nd edition ed., pp. 35-56). Presses de l’Université de Montréal: 62.

Be aware that assessing the effects in a monitoring exercise is different from an effect analysis. An effect analysis will analyze the causal relationship between the intervention and the observed effects. In monitoring, no causal relationship is assessed; effects are monitored but could potentially be due to another element than the intervention, such as another competing intervention or a change in the characteristics of the context.

Understanding the distinction between evaluative research and monitoring is important. However, in real life evaluators will often mix the use of the two approaches as they are different but complementary ways of analyzing an intervention. For example, in implementation analysis, the first stage may include an assessment of the program’s implementation (fidelity and reach) before explaining the implementation with a process analysis (Patton, 1997; Scheirer, 1994). The distinction between evaluative research and monitoring is mostly a heuristic approach to learning evaluation.

Conclusion

Coherence is key when conceiving an evaluation project. Coherence refers to the right alignment between evaluation context, questions, approaches, and methods. There is no real starting point to align these pieces and the final proposal is often the result of many adjustments and negotiations.

Finding the right combination requires a deep understanding of the context and of the expectations of the commissioners and various groups involved. Being knowledgeable about different evaluative approaches will increase the evaluator’s agility for choosing among the many possibilities. Once the evaluation questions have been identified in collaboration with the commissioners, the evaluator will need to identify the research design for answering the question and then identifying the methods to use. The research design will depend on the evaluation question. For example, if you plan to conduct an effect analysis, you will likely choose between experimental designs, quasi-experimental designs, or contribution analysis. If you plan to conduct a logic analysis, you will use a knowledge synthesis approach as your study design. Evaluation questions are key in determining the study design and the study design will be key in choosing appropriate methods. In some contexts, the choice may be decentralized or co-determined with multiple parties; in such cases, the evaluator acts a facilitator offering guidance on what approaches would work or not work for addressing potential questions.

Given the advocacy, sense-making and value-creation roles that evaluation can play, the evaluator also needs to remain mindful of the broader context and consider which parameters can be influenced to ensure the evaluation has a positive impact. In particular, participative and collaborative processes, culturally responsive practices, or engagement processes can be important components of the project design, beyond evaluation questions. However, the ability to choose what would be an adequate approach for a given context depends on the evaluator’s ability to use different models and their awareness and depth of understanding the context. As evaluators, we would like this first phase to be rational and straightforward. The reality is much messier.