Vous l’avez sans doute déjà repéré : sur la plateforme OpenEdition Books, une nouvelle interface vient d’être mise en ligne.
En cas d’anomalies au cours de votre navigation, vous pouvez nous les signaler par mail à l’adresse feedback[at]openedition[point]org.

Précédent Suivant

Chapter 2. Research questions in PHIR

p. 35-49


Texte intégral

1PHIR, like any scientific approach, starts with a research question that the study must try to answer. In this context, the demand expressed by the stakeholders (decision-makers, population, research funders, practitioners, researchers…) is often and legitimately focused on results (effectiveness of the intervention). However, it is misguided to consider that effectiveness is the only, or even the main, research question in PHIR. If we accept the social utility purpose of PHIR (see Chapters 1 and 6), there are other equally important research questions to consider. Clearly, the choice of the question should be the subject of a consultation process among all stakeholders and take into account multiple issues related to data, budget, relevance, time, etc. (Bamberger & Mabry, 2007). This choice is, in fact, not neutral. For example, as in other types of research, the same PHIR can lead to different conclusions depending on whether the choice is made to base them on a broadly formulated population-based result or on a result that considers differences between social groups.

2The aim of this chapter is to present and illustrate the main research questions in PHIR. These questions are neither exclusive nor exhaustive; the same study can sometimes address several questions. In addition, a PHIR can be one step in a multi-step research programme, and each step can contribute to answering one or more of the questions. The list of potential questions could be very long; Patton (2008) has proposed more than 100 in one of his books on evaluation. In order not to overload this chapter, we will limit ourselves to the main research questions most often encountered in PHIR projects and in our experience. There are several ways to categorize these research questions. We have chosen to present them according to the stage of the intervention development process in which they most often feature (see Figure 3), even though we are well aware that a stepwise or linear process does not sufficiently capture the complexity and customisation of this development process. Finally, this diagram applies to de novo interventions developed within the study context. Thus, some steps do not apply to existing interventions that may be the subject of study.

Figure 3. Stages in the development of a population health intervention and evaluative questions.

Image 100002010000049A0000022E55072EEA20F06652.png

Designing the intervention: constructing, refining and validating the intervention theory

3The design of the intervention may be an object of study in a PHIR project. This object may, moreover, be quite complex to grasp, if the intervention is considered not as separate from the context, but rather as part of an interventional system (Cambon et al., 2019), i.e. assuming, on the one hand, permeability between the interventional and contextual components and, on the other, a complex dynamic of action and effect.

4To structure this process, the theory-based approach offers several advantages: it explains and provides arguments for the strategies and activities mobilised by the intervention; it describes the cascade of expected effects (by which causal mechanisms each activity or strategy and their interactions contribute to the outcomes); and it takes into account the population or contextual factors that may interact with these activities. Thus, an intervention design process should begin with the conception of an intervention theory. This can only be done by paying attention to the definition of what is called theory (Cambon & Alla, 2021; Moore et al., 2019), which should not be limited, as is often the case, to so-called classical causal theories, such as protection motivation theory (Maddux & Rogers, 1983) or social cognitive theory (Bandura, 1989). These theories, in fact, generally take a decontextualised and often monodisciplinary view of a subject. On the contrary, the theory must integrate concrete elements linked to the implementation context. The intervention theory must thus integrate multiple constructs describing the constitutive hypotheses of the interventional system to be studied during the PHIR. Table 3 pre-sents, for each type of theoretical approach, an example of a definition.

Table 3. Theories and four models of causality.

1- Theory of intervention

“Hypotheses on which people, consciously or unconsciously, build their interventions.” (Weiss, 1998)

2- Framework                              

“A structure, overview, outline, system or plan consisting of various descriptive categories, e.g. concepts, constructs or variables, and the relations between them that are presumed to account for a phenomenon.” (Nilsen, 2015)

3- Middle range theories

“Theories that lie between the minor but necessary working hypotheses that evolve in abundance during day-to-day research and the all-inclusive systematic efforts to develop a unified theory….” (Merton, 1968)

4- Grand theory

“Theory that will explain all the observed uniformities of social behaviour, social organization and social change.” (Merton, 1968)

Source: Ridde, Pérez et al. (2020).

5In line with Chen’s work, Cambon & Alla (2021) define intervention theory as the combination of:

  • the causal theory, which explains the mechanisms (Box 1) of the effects activated by the intervention components and their hybridisation with all the contextual determinants likely to act as obstacles or facilitators of the expected (or unexpected) outcomes;

  • the action model, which provides concrete elements for implementing the intervention components used to guide the process in order to achieve the objectives. The key feature of the action model is that it focuses not only on activities related to the outcomes, but also on the sequences, resources, actors, and preconditions necessary for their implementation.

Box 1. Mechanisms: what are we talking about?

The notion of mechanism has various definitions, depending on disciplines and epistemological approaches. Machamer et al. (2000) define mechanisms as “entities and activities organized such that they are productive of regular changes”. Others define them more as preconditions for results, as in the realist approach, where a mechanism is “an element of reasoning and reactions of agents in regard to the resources available in a given context to bring about changes through the implementation of an intervention” (Lacouture et al., 2015). In the field of health psychology, they are defined as the processes by which a behaviour change technique regulates behaviour (Michie et al., 2013). This may refer, for example, to how practitioners perceive the utility of an intervention or how individuals perceive their ability to change their behaviour.
In combining contextual and interventional components, the change process produces mechanisms, which in turn produce effects (final and intermediate results). For example, we could envision that a motivational interview for smoking cessation might produce different psychosocial mechanisms, such as intention to quit in the short term, a perception of the usefulness of quitting, and the feeling of being able to do so (perceived self-efficacy). These mechanisms influence smoking cessation. This constitutes a causal chain, defined here as the way in which each event in an ordered sequence causes the next event in the chain. These mechanisms, as a system, can also affect their own contextual or interventional components. For example, the feeling of self-efficacy could influence the choice of smoking cessation aids.
Source:
Cambon et al. (2019).

6Figure 4 presents the intervention theory for the interventional system.

Figure 4. Interventional System Intervention Theory.

Image 1000020100000569000002D1BF07068B379B1FF4.png

Source: Cambon & Alla (2021).

7Developing the intervention theory is a partnership-based process by nature (Box 2). It can only be developed with the relevant stakeholders by mobilising several sources of information: 1) existing theoretical frameworks (e.g. behavioural change models); 2) evidence from the literature or from previous or concurrent studies (e.g. on causal links, the effects of interventions, the influence of context); and 3) stakeholder expertise (what change is likely to occur by implementing this activity, and how and why it will occur).

8The subsequent steps (Figure 3) can help refine and empirically validate the intervention theory in the pilot study. This validation may also be carried out during the evaluation with a view to analysing, using qualitative and quantitative methods, the effectiveness of the intervention in real-life conditions and the mechanisms leading to this effectiveness (see Chapter 3).

Box 2. Design of a Covid-19 vaccine hesitation intervention theory

In the Covamax study (Sponsor: University Hospital of Bordeaux, ANR Funding), the first step was to develop an intervention theory prior to the implementation of a Covid-19 vaccination campaign.
To develop it, the research team, with the help of a group of healthcare stakeholders, carried out a cross-analysis of the CoVaPred survey on the acceptability of anti-Covid measures (
Schwarzinger et al., 2021) and a review of meta-analyses and systematic reviews of factors involved in the vaccination decision (in particular for seasonal influenza) and predictive theories of behaviour.

On this basis, the team was able to develop an intervention theory that specified the key elements of action for a future vaccination campaign:
- Interventional components: 1) positive communication based on collective immunity and not on individual vulnerability for people not at serious risk of severe illness; 2) mobilisation of local relays and actors in an outreach strategy.
- Implementation methods, resources, and sequencing: 1) health professionals (doctors, nurses, midwives, pharmacists) playing pivotal roles in the vaccination campaign in communication and in the act of vaccination, rather than vaccination centres; 2) training of health professionals to respond to patients’ concerns and questions.
- Contextual conditions for success: 1) individual access, free of charge and without delay; 2) a vaccination procedure performed at the time when information is provided.

Activating the following mechanisms: confidence in the vaccine and the word of the professional, sense of control, perception of vulnerability to the disease, adherence to the norm.

9This work of constructing the intervention theory can also be carried out after interventions have been implemented. This is the case when evaluations are conducted for existing interventions. In these situations, having an intervention theory could also make it possible to explore questions on the conventional elements of intervention evaluation, which are relevance (are the intervention and its components appropriate for addressing health needs in this given context?) and especially coherence (are the activities consistent with the intervention objectives and with each other?) (Contandriopoulos et al., 2011).

10This intervention theory can be used to develop a theory-based intervention or to evaluate a posteriori a vaccination campaign that was actually implemented.

Assessing viability

11Viability (described by Chen as “viability validity”, to supplement internal and external validity) is defined as the extent to which an evaluation provides evidence that an intervention is a success in the real world (Chen, 2010). This notion of success refers to the following dimensions of the intervention from the stakeholders’ standpoint:

  • Useful: do stakeholders perceive the intervention as useful for mitigating the problems or improving well-being?

  • Affordable: do the funders see the intervention as feasible in terms of their capacity to fund it?

  • Practicable: are the relevant professionals in the field able to implement it with their resources and expertise?

  • Workable: can existing organisations routinely coordinate and implement activities related to the intervention?

  • Evaluable: is it possible to evaluate the outcomes of this intervention (i.e. is there a hypothesis regarding the link between intervention components and potential health effects)?

12Viability thus goes beyond feasibility (the possibility of implementing an intervention) to focus on its capacity to be implemented, sustained, and scaled up, in routine conditions, by the usual actors (and not, for example, as might be seen in a research scenario with specific funding, an adapted legal framework, ad hoc actors, etc.).

13Assessing the viability of an intervention should be a prerequisite to any effectiveness study in order to avoid the risk of concluding that an intervention will be effective, when ultimately it will not be viable in the real world at the end of the research process (Chen, 2010). Thus, the viability assessment should be carried out as early as possible when developing an intervention. This is one of the purposes of pilot studies (Thabane et al., 2019) in PHIR (Box 3).

Box 3. Viability study in the 5A-QUIT-N controlled trial

5A-QUIT-N is an organisational innovation designed to manage local resources available in the Nouvelle-Aquitaine region (France) to support smoking cessation among pregnant women.

The research team developed the first version of the organisation based on the scientific literature, recommendations from institutions and scholarly societies, interviews with stakeholders, and analysis of the current organisation of smoking cessation services for pregnant women and existing resources. Based on this first version of the organisation, the viability was studied:
- first, a priori (before implementation in the field), based on interviews with professionals and a review of the scientific literature on factors limiting and facilitating the implementation of such organisations;
- then, during a pilot study in one health region, which served to describe the implementation and analyse its obstacles and levers, as well as to gather the opinions of stakeholders, professionals, and pregnant women.

For example, from the perspective of midwives, utility was shown in the concrete outcomes seen in their patients (e.g. smoking cessation); affordability, in the fact that the programme was fully covered by health insurance; practicality, in the ability to integrate the programme into their normal work structures; and adaptation, in the programme’s integration into existing regional organisations.

Evaluating effectiveness

Efficacy, effectiveness: effectiveness in theory versus in real-life conditions

14English has two words to express effectiveness: efficacy, which refers to effects achievable under ideal conditions (or theoretical conditions, where the influencing factors are highly controlled), and effectiveness, referring to effects achieved under real-life conditions (Porta, 2008). Effectiveness under ideal conditions is generally determined by a “classic” randomised controlled trial, while effectiveness under real-life conditions is determined by a pragmatic controlled trial, a quasi-experimental study, or an observational study (natural experimentation). The recent Covid-19 crisis familiarised us with the difference between the two through the evaluation of vaccine effectiveness, in which efficacy was assessed using randomised controlled trials and effectiveness was assessed by real-life studies carried out, in particular, from medico-administrative databases.

15In PHIR, studies are most often de facto studies of effectiveness in real-life situations. Indeed, due to the nature of the interventions, researchers very rarely insert themselves into an experimental situation that involves relatively simple, technical interventions, such as the act of vaccinating.

16The effectiveness of a population health intervention is judged by health outcomes (morbidity, mortality, well-being, etc.). This effectiveness is what corresponds to the overall objective or purpose of an intervention in the planning process. By default, it can be judged on distal outcomes related to health determinants (decrease in environmental exposure, behavioural change, etc.) or on proximal outcomes of seeking care or using the activities and services offered by the intervention. This is what corresponds to the specific intervention objectives in the planning process. It is often impossible, or irrelevant, to assess the effects of an intervention on health outcomes, especially when they are long-term (e.g. an intervention to strengthen children’s psychosocial skills can take decades before producing observable health outcomes). In cases where the results are not health outcomes, arguments are needed to support the relevance of the results in relation to the determinants (i.e. factual elements indicating that a change in exposure to a particular determinant will ultimately lead to a change in terms of health). This can also be one of the uses of the intervention theory discussed in the previous paragraph.

Impacts

17The term “impact” can have several meanings (see example in Box 4):

  • Long-term outcomes, when they are not measurable in the study and it takes a very long time to see their occurrence; for instance, in a programme to reduce smoking among adolescents, the outcome may be smoking prevalence, and the impact (not measurable but modellable), a reduction in cancer incidence.

  • Health outcomes, when the study’s objective is not formulated in these terms. This is particularly the case in projects based on the Health in All Policies (HiAP) approach. Each policy has its own non-health-related goals that can have indirect effects (or impacts) on health.

  • Unintended beneficial or adverse consequences of an intervention (which are not the same as outcomes, which are the intended consequences corresponding to the research question) (Turcotte-Tremblay et al., 2021). An example of beneficial unintended consequences might be that of a physical activity programme aimed at increasing the well-being of seniors (expected outcomes), which may have both a positive health impact, i.e. a decrease in cardiovascular morbidity, and a positive social impact, i.e. a strengthening of community bonds. An example of an adverse unintended consequence might be when increasing green spaces in a city to promote physical activity leads to an increased incidence of allergy symptoms in children. Box 5 provides 12 questions that can guide reflections on unintended consequences.

Box 4. The impact of managing the COVID-19 crisis

One example of these impacts is seen in the consequences of anti-Covid measures worldwide. To reduce mortality among people vulnerable to Covid-19, universal measures were taken to reduce human physical interactions. It is now known that these measures had a negative impact on people’s mental health and led to increased social inequalities in health, delayed treatment for other pathologies, increased food insecurity in vulnerable households, and increased domestic violence, among others (Cambon et al., 2021).

 

Box 5. Considerations for taking into account the unintended consequences of interventions

Based on several empirical studies and a literature review, 12 considerations have been suggested to help research and intervention teams pay closer attention to unintended consequences (Turcotte-Tremblay et al., 2021).

1. Set an explicit objective or research question that addresses unintended consequences
2. Choose and define your terminology
3. Adopt a theory or conceptual framework
4. Determine the study’s perspective
5. Clarify the intervention theory
6. Anticipate potential unintended consequences
7. Focus on desirable, undesirable, and even neutral unintended consequences
8. Include flexible, exploratory methods
9. Cast a wide net when collecting data
10. Track the evolution of unintended consequences over time
11. Take equity issues into account
12. Validate the classification of desirable versus undesirable consequences with stakeholders

Efficiency

18Effectiveness can be expressed in terms of the resources mobilised to achieve it (human, financial, etc.). This is referred to as efficiency. This is the domain of medico-economic studies, with three main approaches (Le Pen & Levy, 2018) corresponding to three types of results (Box 6):

  • The cost-effectiveness approach uses an outcome indicator expressed in terms of health or health determinants (e.g. the cost of a heart attack avoided).

  • The cost-utility approach uses a generic and composite outcome indicator that includes the intervention’s impacts in terms of quantity and quality of life. The Quality-Adjusted Life Year (QALY) is the reference indicator in this regard. This approach has the advantages of integrating all the consequences of an intervention and of allowing different interventions to be compared with different themes.

  • The cost-benefit approach also uses a generic indicator, but expressed in monetary rather than health terms.

Box 6. The Assessing Cost Effectiveness (ACE) Prevention synthesis

ACE Prevention was a massive synthesis funded by the National Health and Medical Research Council (NHMRC). The final report was presented on September 8, 2010. The overall objective of this project was to provide a comprehensive analysis of the incremental cost effectiveness of preventive intervention options for non-communicable diseases in Australia. The experts involved evaluated 123 disease prevention measures to identify those that would prevent the most diseases (cost effectiveness) and premature deaths (cost utility) and those with the highest cost-benefit ratio. The report is available from: https://public-health.uq.edu.au/files/571/ACE-Prevention_final_report.pdf
Source: V
os et al. (2010).

Implementation

19The description of an intervention’s implementation is an important focus of evaluation (Ridde & Turcotte-Tremblay, 2019). This description can help explain the results and their heterogeneity. For example, it can help determine whether negative outcomes are linked to the intrinsic ineffectiveness of the intervention (intervention theory failure), inadequate or incomplete implementation (implementation failure), or lack of fidelity or adherence to the intervention. The implementation analysis focuses on the role of social actors, power issues, and the internal and external dynamics of the intervention. Social science theories are often used to properly comprehend these implementation processes.

20Numerous implementation indicators (“implementation outcomes”) are used, including coverage (proportion of the target population actually reached or participating), intervention dose (“quantity” of intervention delivered), fidelity, organisational quality (compliance with standards, benchmarks, best practices), stakeholder satisfaction, etc. (Hoffman et al., 2014; Proctor et al., 2011).

Analysing the mechanisms

21For complex interventions (which is the case for most population health interventions), analysis of the mechanisms is required to understand how a result was obtained: how it was produced, under what conditions, for whom, and how (Craig, Dieppe et al., 2012). This understanding is crucial not only to perceive the implementation, but also to support the intervention’s sustainability, transfer, or scaling up. It refers directly to the social utility dimension of PHIR.

22This analysis consists of characterising the causal chains (e.g. why and how a particular activity produces a particular mechanism and outcome), analysing the factors linked to the implementation of an intervention (e.g. what makes an actor take it up or not), and finally, understanding the contribution of contextual and population factors to the intervention and results (e.g. to explain the differential effect that some interventions may have on social and territorial inequalities in health). This contribution analysis of mechanisms can be done in particular within the context of a theory-based evaluation (see Chapter 3).

Analysing the dissemination factors of an intervention

23An intervention is evaluated at a given time, in a given context. One challenge, and this is the social value of PHIR, is to be able to support a process of disseminating the intervention when it has proven to be effective, whether such dissemination is done by perpetuation (over time), transfer, or generalisation (in space). Thus, PHIR can, and indeed should, incorporate research questions that focus on describing and analysing:

  • the transfer of intervention from an experimental context to real life and factors related to its adoption by actors and decision-makers;

  • the applicability and transferability of the intervention to contexts other than the one in which it was evaluated (Cambon et al., 2013; Schloemer & Schröder-Bäck, 2018) (Box 7);

  • the processes and conditions for scaling up;

  • the factors related to maintaining the intervention (sustainability, durability).

24It should be noted that this analysis of dissemination factors is informed by all the previous steps: developing an intervention theory, analysing viability, analysing processes and mechanisms in order to understand and predict the intervention’s capacity to be disseminated, under what conditions and, if necessary, with what adaptations. Thus, while conceptually the dissemination of an intervention is seen as occurring after it has demonstrated its viability and effectiveness, the elements for this analysis are collected throughout the research process.

Box 7. Transferability criteria

Transferability differs from applicability in that it is focused on results, whereas applicability is focused on implementation criteria. Transferability criteria take into account elements that pertain to the intervention itself (the components and conditions for the implementation) as well as contextual elements that can influence not only the implementation, but also the results more directly. These elements are the characteristics of the populations, stakeholders, and the context in which the intervention is implemented. Thus, this notion of transferability fully integrates that of the interventional system developed earlier, in that it assigns a significant role to context in contributing to outcomes.
As part of an alcohol risk reduction programme tested in an association in Marseille, France, an analysis of transferability showed, for example, that the support programme put in place was effective in terms of recovery indicators if, and only if, certain contextual conditions were met, such as the layout of the premises where the interviews were conducted (in the form of a salon having a reception area with alcohol), the support processes (at home and/or on the premises), and the training and support provided to the workers (social work rather than health work, regular psychological support for the workers). These conditions are important to consider because they strongly influence the potential for implementing this support elsewhere (applicability) and its success, as it directly affects specific mechanisms (e.g. deconstruction of the feeling of shame).

Conclusion

25This chapter shows that there are numerous PHIR research questions. They are not mutually exclusive and can be articulated concurrently and/or sequentially. The research questions in the following table are provided for illustrative and indicative purposes. Indeed, each PHIR project is a particular case, and the questions it addresses depend on the research context, time and data constraints, and stakeholders’ views.

Table 4. Research questions.

Evaluation dimensions

Examples of sub-dimensions

Examples of research questions

Designing the intervention

Intervention components
Contextual elements to be considered
Effect and process mechanisms
Intended result

How are the elements of the intervention likely to have an effect?
Under what conditions can this intervention have an effect?

Assessing viability

Utility
Affordability
Practicality and adaptability
Evaluability

Useful: e.g. do individuals and other stakeholders see the intervention as useful for mitigating problems or improving well-being?
Affordable: e.g. do decision-makers consider the intervention to be financially sustainable?
Practicable and workable: e.g. can the intervention be integrated into the organisational practices of professional communities?
Evaluable: e.g. is it possible to evaluate the intervention?

Assessing effectiveness and efficiency

Effectiveness
Efficiency
Impacts

How effective is this intervention?
What are the unintended consequences of this intervention?
What is the cost-benefit ratio of this intervention?
What is the cost effectiveness of this intervention?
What is the efficiency of this intervention?

Evaluating the implementation

Coverage (proportion of the target population actually reached)
Intervention dose (“quantity” of intervention delivered)
Fidelity
Mechanisms
Quality of delivery (compliance with standards, benchmarks, best practices)
Stakeholder satisfaction

Under what conditions does this intervention have an effect?
What does this intervention bring to stakeholders?
By what mechanisms does this intervention have an effect?
How does this intervention have an effect?

Assessing transferability and/or dissemination conditions

Transfer of intervention from an experimental context to real life, factors related to adoption of the intervention by actors and decision-makers
Applicability and transferability of the intervention to other contexts than the one in which it was evaluated
Processes and conditions for scaling up
Factors for maintaining the intervention (sustainability, durability)

To what extent can this intervention achieve the same results in this other context?
How can this intervention be adapted to new contexts without losing its effectiveness?
How can this intervention be implemented in another context?
What are the conditions for maintaining this intervention and what is the degree of sustainability?

Précédent Suivant

Le texte seul est utilisable sous licence Creative Commons - Attribution - Pas d'Utilisation Commerciale - Pas de Modification 4.0 International - CC BY-NC-ND 4.0. Les autres éléments (illustrations, fichiers annexes importés) sont « Tous droits réservés », sauf mention contraire.