Highlights

Introduction

We all work within specific contexts, each with their own sociological characteristics, ecological boundaries, challenges, and opportunities. In addition, collectively we need to make choices to sustain social programs in a resource-constrained environment. How do we prioritize the right interventions for the right people, according to the place and time? How do we determine if an intervention meets the needs of individuals and the community’s commitments? How do we make sure we fund and implement interventions that will have beneficial outcomes for society (Robinson, 2018)? All these questions speak to the intervention’s relevancy and the need to examine the alignment between the socio-environmental context and the intervention.

The question of relevance is part of many different evaluation exercises. It is related to needs assessment and it is often a criterion in spending reviews (Robinson, 2018; Rossi et al., 2019).

In a context where life is threatened, the environmental conditions and their relationships with interventions should be integrated into needs assessment exercises. Further, recognizing that disruptive climate change events will happen more frequently and affect the risks communities will face, a prospective view needs to be implemented when assessing an intervention’s relevance.

Defining What is, What should be and For Whom

Needs assessment is a type of evaluation that provides information on the relevance of the intervention before implementation (ex-ante) or information about whether an existing intervention is still relevant, based on the analysis of socio-ecological characteristics of its implementation context (ex-post).

From a program evaluation perspective, needs assessment is the means by which an evaluator determines whether there is a need for a program and, if so, the nature and extent of that need and related implications for program services appropriate to address that need. Needs assessment is critical for the design of new programs, but it is also relevant for established programs when it cannot be assumed that the programs continue to meet the need or that the need has not changed. (Rossi et al., 2019, p. 31)

Needs assessment is a fundamental exercise to make sure programs contribute positively to solving a problem.

Needs assessment is a process or a systematic set of procedures undertaken for the purpose of setting priorities and making decisions about programs and organizational improvement or allocation of resources. The priorities come from identified needs, which are measured discrepancies (gaps) between the current (what is) state of affairs of a group or organization and the desired (what should be) state in regard to variables of interest. (Witkin & Altschuld, 1995, p. 276)

Defining “What is” and “What should be” involves identifying the target population (For whom?) and determining important elements from the context to take into consideration.

Relevance is a relative concept and is determined by taking into consideration different elements, an important one being the population the intervention is intended to serve. Needs assessments can be conducted across country populations, large communities, small groups or more narrowly within organizations (Gupta et al., 2014). In this chapter, for simplicity, the group the needs assessment will include is called “community”. It could include small communities defined by the population an organization serves, or larger communities up to a nation or the population of a country. Clarifying early who the relevant community is will help narrow down the scope of the evaluation.

Contextual Elements and Boundaries

The context is constantly changing, affecting the relevance of interventions: risks and needs change, and new, more effective and/or efficient interventions emerge. Furthermore, when thinking of socio-ecological challenges, meaningful and significant action is urgent calling for the systematic integration of socio-ecological considerations into this exercise. Needs assessment involves defining the context, in particular the elements which will matter the most and the contextual boundaries. How is the context defined? Is it geographically? Is it defined based on other criteria, for example, according to boundaries of communities of practice or organizations?

Ex-ante and Ex-post

The process for conducting the needs assessment will differ depending on whether the intervention already exists or is new. For a new intervention, one will want to characterize the context. Given the pace of change spurred by climate change and other environmental threats and their impact on health and social well-being, evaluators need to adopt a prospective perspective in their analysis. This involves not only conducting a diagnostic of the current context but also anticipating future risks and community needs.

• What challenges is the community facing?
• What major problems will the community face given the impacts of climate warming and other environmental and social changes?
• Which of the important problems identified can be addressed through collective action? Some problems may have no feasible solutions and can be discarded. The feasibility of addressing a problem—considering available technologies and resources—needs to be taken into consideration.
• What should the objectives of the intervention be? What causes should be targeted?
• Which population should be targeted by the intervention?
• What partnerships would be relevant? (adapted from Champagne et al. 2011).

If the program already exists, one may want to question the relevance of the intervention if it feels like the needs or context may have changed (Rossi et al., 2019). Considering the current environmental state and the increasing disruptions impacting communities that require rapid and profound changes, and given that previous needs assessments often took for granted or overlooked environmental conditions, it is pertinent to reconsider interventions with regard to their socio-environmental impacts. The context has changed and continues to evolve fast.

Conducting a needs assessment of existing interventions is an exercise which consists of deciding whether interventions should be sustained, modified, or terminated. When the program already exists, the question of its relevance includes understanding its contribution in relation to the changing context; questions of relevance then extend to other considerations of existing alternatives and whether there are more effective or efficient options.

Other important Considerations

Needs assessment steps can appear to be straightforward; however, they raise other important ethical dilemmas and questions. Several of these are outlined below.

First, how can the highest priority be determined? The importance of a problem can be assessed according to different criteria: its scope, its gravity, and the characteristics of the people affected. For example, a problem affecting a large population segment but with minor consequences may be considered less important than a life-threatening condition affecting a small population group. There is no rule for determining what makes a problem more important than another one—it comes down to weighing values. That is why engagement activities ensuring consultation and buy-in from different groups is important. Background information identifying a community’s main problems can be collected by different means: reports presenting a state of the situation, census data, surveys, key informants, interviews or focus groups, world cafés, brainstorms, etc. (Rossi et al., 2019). To gain social acceptance of the priorities identified, engagement activities with different community groups are crucial.

Second, how can needs be forecasted (Rossi et al., 2019)? A needs assessment based on the historic data will always lag the current situation. Given the time needed to design and implement an intervention, there is a risk that an intervention based only on existing (retrospective) data will be outdated when fully implemented. That’s not a certainty, of course, yet it is a risk. A prospective analysis (perhaps a consideration of scenarios) should be conducted to analyze what the risks will be.

Third, how can socio-environmental considerations be integrated? Compared to current needs assessment practices, new steps need to be introduced for considering planetary health and creating conditions that contribute to positive eco-systems, offer protection from climate events, and increase resiliency. In particular, when conducting a diagnostic on the context to identify needs, the evaluator should adopt a holistic approach in their forecast of the context’s evolution. International commitments to reduce greenhouse gas emissions and preserve and promote biodiversity (United Nations, 2015, 2022) should be included in priority-setting activities and decision-making, as should the critical dimensions for all kinds of life to thrive on the planet (Richardson et al., 2023; Steffen et al., 2015).

Furthermore, with the climate already warming significantly, climate events such as flooding, heatwaves, droughts, and fires will happen more regularly and more severely and create new risks for communities and organizations. It is our responsibility to be responsive to this new evolving state of the world, which requires envisioning what these risks will be and consider both mitigation and adaptation measures, as well as ways to increase community and population resiliency (Brousselle et al., 2020). Additionally, given the linkages between environmental injustice and environmental crises, socio-environmental conditions should be considered in all needs assessments, not just environmental conditions alone. Finally. the evaluator should consider the intervention’s impacts on natural and human systems when contemplating the intervention’s potential to meet community’s needs. The Planetary Health Framework can offer support for integrating these dimensions into a holistic implementation approach to relevance analysis (Brousselle & McDavid, 2021).

Next Steps?

Needs assessments analyze the context’s characteristics to identify the intervention that would be the best match for solving a perceived problem affecting a specific context and community. Needs assessment will often be conducted using a combination of criteria and evaluation methodologies. In spending reviews the intervention’s effectiveness and efficiency are criteria used in complementarity to the relevance criteria to determine if the intervention should continue to be funded.

Other evaluation questions and methodologies can help conducting needs assessments. For example, an ex-ante needs assessment can naturally lead to conducting a reverse logic analysis to identify interventions suited to a specific context that are promising for achieving the objectives. A reverse logic analysis strategy could also be pertinent to inform ex-post needs assessment, as the expectation is to implement knowledge-informed priority-setting and decision-making.

A Political Exercise

As mentioned earlier, relevance is a relative concept. Its meaning varies according to different groups’ perspectives or who sponsors the analysis. In an ideal world, one would engage with different community groups to elicit and unpack what a relevant intervention should be, or engage with a diversity of actors throughout the prioritization process to align with a broader definition of relevancy: “Is the program trying to achieve an outcome for society that is important?” (Robinson, 2018, p. 306). “Needs assessment requires carefully analyzing a situation and building support for action” (Gupta et al., 2014, p. 33). Priorities may differ according to the groups involved (Gupta et al., 2014). Implementing the right engagement activities is a way to make sure decisions will be socially legitimate and accepted. They can also contribute to making programs known and building meaningful relationships with a diversity of actors (Gupta et al., 2014).

Needs assessments, because they can support priority-setting and budgetary decisions, can be very political exercises (McDavid et al., 2019). Needs assessment is, for example, one important element of spending reviews. In spending reviews, relevance is generally one criterion among others. For example, Robinson identifies six criteria employed during spending reviews to identify savings options: relevance, duplication, effectiveness, efficiency, equity, and market failure (Robinson, 2018, p. 306). Marchand and Brousselle note that in the spending review exercise which took place in the Province of Quebec in 2014, decisional criteria were (1) relevance (the objectives relate to the needs of the targeted populations and the government priorities; a program that could have been created in the current context; a program that, if dismantled, would create important negative social, economic, and environmental consequences), (2) effectiveness, and (3) efficiency (Gouvernement du Québec, 2014; Marchand & Brousselle, 2018, p. 399).

The official rationale for spending reviews has been that they ensure that the most relevant interventions are funded and others are discarded; however, spending reviews are often used when the political priority is drastic cuts in public spending. Determining relevance is more than just a technical exercise. It also involves weighing competing values and their reflections in political priorities.

It is not unusual to observe direct opposition and barriers to data access when programs are threatened by the exercise. As noted earlier, many choices must be made during a needs assessment. Its validity depends, at least in part, on whether the process is recognized as legitimate by the different parties—something that cannot be assured if the process is contentious from the outset. An evaluator’s ability to facilitate discussions and communicate effectively will be critical to the success of the process.

In scoping a needs assessment, one consideration is the diversity of values and interests in a community. Generally, homogeneous communities will present fewer challenges in terms of assessing needs. Smaller communities can also be more straightforward for the same reasons. The evaluator should anticipate that social media and other sources of information will influence community preferences in setting priorities and their understanding of the problem. Disinformation about climate change related risks can divide communities.

Finally, including socio-environmental considerations in the intervention’s analysis of relevance is likely to completely switch how priorities are identified. As these criteria have not been included in prior evaluations, their integration could lead to eliminating some interventions and choosing others with more positive impacts. Conducting evaluations on an intervention’s relevance using planetary health lenses could be an important lever towards implementing the kind of transformation that is needed for truly addressing the current socio-environmental challenges.

Summary

Table 11.1 summarizes the main questions and steps for conducting a needs assessment.