Vous l’avez sans doute déjà repéré : sur la plateforme OpenEdition Books, une nouvelle interface vient d’être mise en ligne.
En cas d’anomalies au cours de votre navigation, vous pouvez nous les signaler par mail à l’adresse feedback[at]openedition[point]org.

Précédent Suivant

Chapter 4. Methods of data production and analysis in PHIR

p. 77-90


Texte intégral

1Population health intervention research is essentially empirical (see Chapter 1). Thus, this approach is based on collecting and analysing primary or secondary data, regardless of their nature or source.

2The types of data to be mobilised and the methods for analysing them are defined based on the research question (see Chapter 2). They may also be a function of research traditions and disciplinary approaches (there are several ways to address the same question). Finally, the identification, production, transformation, and analysis of data must respect methodological, ethical, and legal principles and best practices.

3This chapter will review these principles (Figure 9) and describe some of the methods of data collection and analysis most commonly used in PHIR, without presuming to cover all practices exhaustively.

Principles

Problematisation

4The first of the principles to be considered in PHIR is problematisation, i.e. the scope of the questions about the intervention that the research is intended to answer. As we have seen in previous chapters, the aim of PHIR is to understand how actions work rather than to produce universal results, leading to multiple possible and interdependent questions.

5PHIR attempts to propose, use, validate, or adjust explicit or implicit theories (Ridde, Pérez et al., 2020), such as: 1) intervention theories (i.e. hypotheses upon which people, consciously or not, construct their programme plans and actions); 2) explanatory frameworks (i.e. structure, overview, schema, system, or plan made up of various descriptive categories; it describes empirical phenomena by bringing them into a set of categories without explaining their interactions); 3) middle-range theories (i.e. theories linking hypotheses that are detailed and clear but highly evolving, with an attempt to generalise and abstract); and 4) grand theories (i.e. theories that explain all observed uniformities in social behaviour, social organisation, and social change).

Figure 9. Some key principles.

Image 100002010000040B000003C52E546995DFEACCB8.png

6Thus, the diversity of the questions that PHIR seeks to answer requires a combination of relevance (which methods are most appropriate to answer the question?), rigour (how to apply these methods as rigorously as possible to obtain valid data?), and sensitivity to unanticipated effects and contexts (Turcotte-Tremblay et al., 2017). These characteristics must therefore be taken into account when choosing data collection and analysis methods.

Rigour

7Whatever the data collection approach, its implementation requires rigour. First of all, it should be made clear that, contrary to what disciplinary controversies have been suggesting for too long, there are no methods that are inherently more rigorous than others (i.e. the conventional opposition between quantitative and qualitative methods) (Olivier de Sardan, 2008). Nor is one approach more objective than another, or one methodological approach more influenced by values than another? (Hassall et al., 2020). It is the entire process of conducting a PHIR that must be as rigorous as possible, regardless of the method or approach used, in order to yield results that are meaningful, valid, and validated, regardless of the scientific criteria applied (see Chapter 3). As we have previously explained, pragmatism is a core feature of PHIR approaches, and those who follow this process consider that “there is now room for both subjectivity and objectivity to be useful at different points within an evaluation or applied research study” (Donaldson et al., 2009). Thus, in each approach, there is a rigorous way of conducting a research process.

8If we were to summarise, rigour in PHIR could be considered to have the following dimensions:

  • researcher/team's competence with respect to the nature of the data being used;

  • respect for ethical principles and legal frameworks;

  • ability to substantiate all the methodological steps (e.g. how a questionnaire or interview guide was developed, from what sources, how it was pretested, by whom);

  • compliance with best practices intrinsic to the type of research (e.g. double entry for quantitative data, transcription of interviews and prolonged field immersion for qualitative data);

  • justification of the methods used to meet the objective.

9For example, it is often thought that quantitative methods are more rigorous because they are more objective, while qualitative methods produce results more susceptible to the analyst’s subjectivity. However, the reality is that the subjectivity of which qualitative analysts are accused can be found, in quantitative methods, in the orientation towards often restricted data collection tools (e.g. reduced number of questions, closed response modalities). These restrictions are likely to limit the scope of possible responses, thereby inducing subjectivity upstream, whereas it is seen downstream in qualitative study designs. The issue of rigour, therefore, does not involve comparing methods but rather refers to each method’s compliance with specific and detailed criteria to ensure its soundness and validity. These are the criteria of scientificness, which include:

  • the transferability criterion, which refers to the reproducibility of the results and their applicability in other situations;

  • the reliability criterion, which refers to the reproducibility of the results if the same conditions for collection are followed;

  • the confirmability criterion, which realigns the notion of neutral judgement in accordance with facts and not the researcher’s values;

  • the credibility criterion, which ensures that what is produced reflects the nature of the data and not the researcher’s interpretation.

10There is a great deal of literature on this subject, looking at quantitative (Olivier de Sardan, 2008), qualitative (Bujold et al., 2018), or mixed methods (Laperrière, 1997).

Ethics

11Ethics is not just a legal matter of what steps to follow. Ethics is, first and foremost, a set of fundamental principles for health research, including research based on questionnaires and interviews. Indeed, a data collection process may not be innocuous. In addition to the methodological issues it raises by its influence on behaviours (of both researchers and respondents), it can also, like any intervention, raise ethical questions, as it can cause negative feelings, contribute to stigmatisation, or create expectations that cannot be met.

12As such, research should only be undertaken: 1) if it is potentially useful (hence, a precondition is that the state of the art justifies the relevance of the research question); 2) if it meets the criteria for rigour so that the results can be used; and 3) if the risks generated by the research are either minimal or counterbalanced by benefits for those taking part in it. The ethical or non-ethical nature of the research should not be determined based on the researcher’s personal interpretation, but rather on an independent collegial process.

Some methods of data collection and analysis

Quantitative, qualitative and mixed methods

13Generally speaking, there are different approaches (quantitative, qualitative, mixed) and a variety of data sources, which can be secondary (e.g. routine data) or primary (e.g. questionnaires, interviews, observations). It is not our intention here to go into detail about the wide range of methods, which have been extensively developed in numerous reference works (Campbell & Stanley, 1966; Candy et al., 2011; Creswell, 2009; Malterud, 2001; Mason, 2005; Patton, 1990; Sandelowski et al., 2009), but rather to summarise their broad outlines (see Table 5).

Table 5. The different research methods.

Quantitative method

Qualitative method

Mixed methods

Philosophical stance

Social worldview:
The social world has an existence independent of humans. Social facts are objectifiable, in that they follow rules/laws that can be apprehended through measurement.

Research objective:
To define the laws that govern this social world. This scientific knowledge is based on the core principle of the reproducibility of facts, which is not very compatible with PHIR.

Reference paradigm:
positivist approach

Social worldview:
Reality is socially constructed, i.e. based on the meaning given to it by the actors.

Research objective:
To gain an understanding, without presuming to generalise, of the systems of values, beliefs, and culture that underly the behaviours, forms of action, and thinking of people in society.

Reference paradigm:
interpretive or constructivist approach

Social worldview:
In PHIR, the objects of research are often complex. This precludes studying only a small part of the system in isolation or trying to oversimplify it. The researcher must develop a research design and collect data that will make it possible to answer all the research questions.

Reference paradigm:
pragmatic approach

Interpretative approach

The analysis of phenomena is based on a deductive approach of interpretation and of hypotheses to be tested.
It is based on causal relationships between variables characterised by quantifiable and measurable attributes.
Objectivity in the research is asserted and concretised by controlling for the roles among these different variables when testing hypotheses, in particular through comparisons and by applying the principle of “all else being equal.”

The analysis of phenomena is based on interpretation of the meaning given to those phenomena by the actors.
Actions within these phenomena are not reduced to quantifiable, measurable attributes.
Understanding them requires a connection between the actor and the researcher, since the researcher seeks to mediate the meaning attributed to them by the actors.

Methodological choices are determined by the research question rather than by epistemological hypotheses.
Combining methodologies allows for innovative ways of understanding and studying the world: it makes it possible to monitor and measure phenomena while taking into account the context in which they are rooted and which they shape.

The analysis process

Statistics are the tool of excellence for measuring the association between an exhibition (e.g. an intervention) and an effect.
The study designs are based on methods of experimentation structured on the principle of comparison, which reinforces the mechanics of causality in accordance with the principle of “all else being equal.”

It is the meanings given to actions and behaviours by actors (including their own actions and behaviours), and not their iterations or recurrences, that are sought to uncover the complexity of the processes through which the facts are constructed.
The objective is to disentangle the social phenomena from the ways in which actors make sense of them through different techniques: participant observation, interviews, focus groups, life stories, etc.
The analysis focuses on singularity and dissimilarity.

Mixed methods are generally classified according to three dimensions (Box 8): time frame (concurrent or sequential), weighting (equivalent status or dominant status), and procedure for combination (merger, integration, and connection) (Box 6).

14We present the main characteristics of the different categories of data collection methods according to three dimensions: 1) epistemological stance (i.e. each method produces different accounts of the same reality, reflecting a particular conception of the world); 2) the interpretative approach (i.e. the data interpretation principles applied); and 3) the analytical approach (i.e. the means, process, or approach for accessing what is thought to represent reality). For the sake of clarity, this presentation assumes an inevitable Manichaeism. The debates and controversies, as well as the middle courses and nuances, are numerous and continue to fuel scientific writings to this day. An example of a mixed methods approach is presented in Box 11.

Box 11. Types of mixed methods

In the sequential option, the researcher seeks to explain or expand on the results of one method through another method used subsequently; for example, a qualitative study (exploration) could be followed by a quantitative analysis (generalisation of results). The different types of data are collected one after the other.
In the concurrent or convergent option, quantitative and qualitative data are brought together to provide a comprehensive analysis of the research question. Both forms of data are collected at the same time and integrated into the interpretation of the overall results. For example, in a project to evaluate a support system for vulnerable people, we developed a convergent multi-case approach. The objective was to assess, within a facility that serves a highly precarious, marginalised, and stigmatised drug user clientele, how it could help realign the users’ position within their health, social, and civic environment by applying approaches such as empowerment and community building. In particular, this involved identifying the processes and mechanisms at play (transferability and sustainability). To do this, the study deployed a series of investigations combining user and stakeholder interviews, observations, documentary analyses, and questionnaires. The data were analysed together to formulate a conclusion regarding the effect and the conditions for this effect.
In the third option, known as the quasi-mixed method or conversion design, the researcher collects only one type of data (qualitative or quantitative) and transforms it into another; for example, qualitative data (interviews) might be transformed into quantitative data (word counts).

Modelling and methods using existing data

15The development of large databases whose use is facilitated by the expansion of computer processing capabilities is an opportunity for PHIR. Such data can be integrated into empirical studies and can even replace them. This is important for PHIR researchers to consider, because these models can help answer questions that were difficult to answer without these tools. This can also reduce the costs and logistical constraints of research (primarily through using existing data rather than generating new data).

16Underlying these generic terms, modelling and databases, are many objectives and potential uses. We list the main ones here.

To construct and validate intervention theories

17Literally, modelling refers to the construction of a model to describe or explain a complex system. Developing and validating an intervention theory thus, by definition, constitutes a modelling exercise. As we have seen, a variety of sources can be used to develop a theory, and its validation can be based on both qualitative and quantitative methods.

To use existing data in various experimental and non-experimental research designs

18In health, there are many routine data, related, for example, to the production of care (reimbursement databases, monitoring of activities), to the sociodemographic characteristics of the population, to environmental factors, etc. These data can be individual (e.g. a person’s vocational status) or collective (e.g. a region’s deprivation score). Before generating new data, which is complex, costly, and can pose ethical or legal issues, it is advisable to consider using existing data. This use may take several forms. It could be to describe a population; for instance, in a randomised controlled trial, data can be matched based on population characteristics available in sociodemographic databases (such as Insee in France). Or it could be to supplement a specific data collection; for example, health events in the medium term may be monitored passively (i.e. without a specific survey) by using reimbursement databases. Or again, it could be to replace a specific data collection; for example, several evaluations of the effectiveness of Covid-19 response interventions were based entirely on existing data, i.e. using pre-existing surveys on the implementation of these measures in different territories and correlating these data with epidemic-related health data (incidence, hospitalisations, deaths) from hospital bases and health surveillance systems.

To model non-existent data

19It is not possible to collect all the outcomes of an intervention over the course of a study, particularly for the long-term effects and impacts of an intervention. These can be estimated by modelling based on previous knowledge (Box 12). For example, the WHO has developed the Heat tool, which allows a regional community to estimate the number of lives saved based on behavioural data (cycling and walking)1.

Box 12. How many lives were saved through an intervention?

As in many countries, numerous decision-makers in West Africa still believe it is important to charge patients when they visit health centres. Yet this point-of-service payment imposes an insurmountable barrier on most people, especially the poorest. Interventions have thus been implemented to show the effectiveness of abolishing direct payment in expanding healthcare use. Many studies have shown that this has led to increased use of public health centres and reduced family healthcare spending. However, before scaling up this intervention, decision-makers wanted evidence that it reduces mortality and thus saves lives. Yet in these types of interventions it is virtually impossible to obtain this type of data, especially when we know that the causal pattern between access to care and mortality is long (5 to 10 years?) and circuitous, given the many determinants of health. However, to inform the debate, a team decided to model the effects of this intervention on children’s health in order to provide decision-makers with data without waiting for the long-time frame of these studies. Using the Lives Saved Tool (LiST) approach, which includes, among other things, measures of changes in coverage of essential services and several scenarios of their impacts, the study showed that scaling up the intervention nationally, assuming the same effects, would save 14,000 to 19,000 children’s lives and reduce mortality by 16% to 17% for children under five (Johri et al., 2014).

To use modelling in a research design as an alternative to data collection

20These methods were developed in health economics to address questions of effectiveness or efficiency when direct empirical observation was not available. In particular, these are decision-support models in which all available information is modelled statistically to estimate an intervention’s effect on health, to compare two strategies indirectly with each other, or to transpose results to another population with different characteristics. These techniques are mainly used to carry out a priori evaluations to support decision-making.

Methods for data production through consensus-building

21The objective of understanding the complexity of PHIR interventions, besides invoking the pragmatic approach called for in mixed methods, also strongly implies considering stakeholders and the public as actors in the research. In fact, whether in formulating interventions and their underlying hypotheses, or in producing data to refine and validate them, the actors’ perspectives are a core issue, and specific methods must be implemented to take them into account (see Chapter 5).

22PHIR projects therefore incorporate a variety of methods for collecting and pooling these perspectives to advance the research project. These include, for example, methods to construct and validate the intervention theory, to prioritise the strategies to be implemented, to share findings on the factors that facilitate or limit programme implementation, etc. Various methods can be applied, such as the Delphi process and the nominal group technique, and these two main methods have many variants. Moreover, technological advances and cross-fertilisation between the qualitative and quantitative aspects of these two techniques enable the development of other methods, such as concept mapping (Péladeau et al., 2017). These three methods of consensus-building can, but do not necessarily, fit into participatory research approaches in which all the processes relating to PHIR are carried out in a co-construction approach (see Chapter 3). In this chapter, we present these methods as means of facilitating, with quantitative analyses, stakeholder involvement. Our aim is to show that stakeholder involvement is not limited to participatory action research approaches nor to qualitative methods, and that figures and numbers can also be useful when we make room in the process for the intervention’s stakeholders.

The Delphi method

23The aim of the Delphi method (Ab Latif et al., 2016; Booto Ekionea et al., 2011; Niederberger & Spranger, 2020; Williams & Webb, 1994) is to determine the extent to which people agree on a given matter in order to obtain a consensus opinion. It can be used for different purposes, such as to identify the current state of knowledge, to refine the hypotheses of an intervention, to resolve controversial situations, to formulate operational recommendations, and to develop monitoring tools or indicators.

24The Delphi method is usually conducted using questionnaires. It can be conducted in person or remotely by mail or text message. While focus groups deliberately use group dynamics to spark debate on a topic, the Delphi method preserves the anonymity of participants and limits debates during the different data production rounds (see Box 13). The Delphi method can follow four steps: 1) preparation (i.e. developing criteria for participant selection, assigning an anonymous number, contacting the selected persons); 2) administration of questions, with each person receiving a series of questions on the subject of the study (e.g. on the purpose for evaluating the intervention); 3) consolidation of responses in order to draw up the report for each round until consensus is reached; and 4) classification of subtopics (if necessary), which is helpful for producing the final report and having it validated by the participants.

Box 13. The OCAPREV Example

As part of a research project to develop an intervention theory of the conditions for effectiveness of nutrition-based applications to support behaviour change, a hybrid Delphi method (online and in-person) was used with two groups of experts: professionals and patients.
The aim was to obtain, at the end of the consensus process, a list of behaviour change techniques (e.g. information on the advantages and disadvantages of physical activity) and one or more mechanisms of effects (e.g. feeling motivated, feeling able to put it into practice).
The work was conducted in four stages: 1) In three series of online questions, the professional group was asked to assign one or more mechanisms (e.g. becoming aware of one’s vulnerability, activating the intention to change, strengthening perceived self-efficacy) to each technique (e.g. presenting the benefits of change, encouraging the person to modify their environment to make it conducive to change; anticipating problems and solutions to sustain the change over the long term). On this basis, the mechanism-technique pairing(s) chosen by at least half of the experts were selected. 2) In a second round, the group was asked to validate or adjust this list. Again, the pairings chosen by half of the respondents were selected. 3) At an in-person seminar of patients, they were asked to validate or adjust the mechanism-technique pairings produced by the professionals. They validated them all and added several comments to the e-Delphi software. 4) In the third round of the e-Delphi process, the group of professionals validated the list of pairings thus amended by the patients.
Source:
Aromatario et al. (2019).

25This method presents several advantages: rapid consensus, free expression without group influence, low cost of administration and analysis, and the possibility of obtaining large amounts of data. However, it can be long and cumbersome (several survey rounds) because only those who deviate from the norm must explain their position. Also, potential interactions among the hypotheses under consideration are not taken into account.

The nominal group

26The nominal group process (Delbecq et al., 1975) consists of generating proposals on an issue that concerns a small number of people without necessarily reaching a consensus. It is used to address open-ended questions that involve a point of view or opinion, or to gather suggestions or solutions. The nominal question is the question presented to the group. It must be precise, clear, and unambiguous. It must generate responses with the same degree of specificity.

27This technique requires intense preparation and precise structuring. The nominal question must absolutely be validated in advance by persons with the same characteristics as those invited to the meeting, so as to generate information based on a common understanding of the question.

28It may consist of five steps: 1) production of statements, in which each participant individually produces a list of statements responding to the nominal question (one answer = one idea); 2) data collection, in which the facilitator invites each person to share with the group just one of their ideas and reformulates them, taking care to number them (each participant must propose a statement that is not already listed or else explain how it differs); 3) clarification of the statements, which involves revisiting them and making them concise, clear, and understandable for everyone, so as to be understood by the whole group, but without seeking agreement; 4) individual anonymous voting, in which each person chooses the statements they think best answer the nominal question and orders their choices from most to least important; and 5) analysis and presentation of results, in which the statements for the whole group are cumulatively weighted by adding up the weights given to each statement (it is useful to include for each statement the number of participants who chose it, as well as the total weight assigned by the group).

29The advantage of the nominal group process is that it allows everyone to express themselves and it produces a collective view without the participants having to reach consensus (thereby avoiding power plays). Proponents of empowerment evaluation, such as Fetterman (2000), have often suggested using a similar approach to encourage free expression and discussion among intervention stakeholders.

Concept mapping

30Concept mapping (Kane & Trochim, 2006) aims to produce a collective consensus around the answer to a question that can be helpful not only in developing a conceptual framework to guide the intervention planning or evaluation (Box 14), but also in obtaining empirical data to evaluate an action (e.g. what factors help to sustain the intervention). This method combines creative brainstorming (as with the nominal group) and multivariate statistical analysis to collectively develop statements in response to the question, which are then grouped into categories. This mixed-methods approach confers strong credibility on the consensus results. It was designed to enable a panel of people, using a participatory and iterative process, to identify the key components and dimensions of a concept and to describe how each component is related to the others. With this method, qualitative data can be not only analysed inductively, but also studied using multivariate statistical analyses that combine the ideas expressed by participants into categories and in the form of concept maps. Weights can be applied, and results can be presented graphically. It can be conducted entirely online or in alternating in-person and online phases.

Box 14. Illustration – The research project that produced the ASTAIRE grid

To develop a health promotion analysis tool, known as Astaire, a conceptual mapping method was used to define the criteria to be taken into account when analysing the transferability of interventions (the term having first been defined by a literature review and shared with the experts).
The process involved 18 experts and led, in the first stage, to the generation of 234 criteria, which were then reformulated, standardised, and sorted. Then, through the process of rating and categorisation, two grids were developed. The first grid, which consisted of 18 criteria and 56 sub-criteria, was to be used when designing and describing a primary intervention. The second, with 23 criteria and 69 sub-criteria, was intended to be used when considering the transfer of a primary intervention to a different context, or when assessing
a posteriori what caused a difference in effects between the primary intervention and the intervention ultimately implemented in the new context.
Source:
Cambon et al. (2013).

31The process can be grouped into six main steps: 1) preparation, which includes selecting participants and developing the objective of the conceptualisation; 2) generating statements by brainstorming about the question being asked (one answer = one idea; e.g. what are the challenges of implementing the intervention?); 3) structuring the statements by weighting and categorising them; 4) representing the statements in the form of a conceptual map by using multivariate and cluster analyses and/or by calculating the average relevance score for each criterion; 5) interpretation of the maps by the people involved; and 6) use of results.

32The advantage of this method is that it arrives relatively quickly at an answer to the question and provides a graphic representation of all significant ideas and their interrelationships.

Conclusion

33The objective of this chapter has not been to produce an exhaustive list of data collection and analysis methods that can be used in PHIR, but to present the broad outlines of those most commonly used and to highlight their advantages and limitations. It is also essential to consider PHIR objectives from a mixed-methods standpoint and to think about potential ways to innovate in our analyses. In particular, we have presented some key principles of PHIR methods, and in addition to rigour, the emphasis must be primarily on their relevance. In contrast to the usual practice, the research questions should guide the choice of methods, and not the other way around.

34Thus, methods should be chosen based on their capacity to answer the research questions and their fit with the type of research design (see Chapter 3) being used. Given the multiplicity of PHIR questions, and assuming a pragmatic approach, this implies using methods either in combination or sequentially in different phases of the research project. Moreover, the methods presented here should serve not only to collect and organise information for the purpose of the research project, but also to create a climate of trust and reciprocity among the stakeholders of a research project (e.g. participatory methods) and foster the interdisciplinarity needed for this field of research.

Notes de bas de page

Précédent Suivant

Le texte seul est utilisable sous licence Creative Commons - Attribution - Pas d'Utilisation Commerciale - Pas de Modification 4.0 International - CC BY-NC-ND 4.0. Les autres éléments (illustrations, fichiers annexes importés) sont « Tous droits réservés », sauf mention contraire.