1The ex-post case studies impact assessment method is based on the assumption that the effects of research on society are the result of complex interactions between a plurality of actors, scientific or otherwise, who produce knowledge and disseminate, adapt, transform and use that knowledge. The impacts of research thus appear as the result of this network of actors taken as a whole. In this context, it is difficult to attribute an impact to a specific actor. The case study method seeks to document the actors' contributions to the generation of an impact (Text Box 1).

2To do so, the method uses impact pathway analysis. This tool is a key element of the approach, which makes it possible to identify the actors, assess their contributions, and shed light on the processes by which the research performed manifests materially as impacts on society. These processes take place over the long term1.

3This chapter first discusses the guiding principles of the method, then turns to the manner of their implementation in the Miriades project.

Guiding principles

Definition of case studies and their scope

4The method is based on the performance of ex-post case studies, i.e., assessing the impacts of research that has already been completed.

5This is done by starting from a known impact (social, health, environmental, etc.), and then moving back in time, approaching any work2 whose results are likely to have contributed to this impact. The work thus approached constitutes the scope of the case study.

6The identification procedure for a case study selected in Miriades, moving from downstream to upstream, helps provide a sense of how the research has contri-buted to a given impact. This contribution is the result of multiple investments and is built over time, most often through several shorter-term scientific projects. Societal impact is therefore rarely the product of a single, time-limited project.

Identification of actors and their contribution

7The method is based on an analysis of the role of the actors contributing to the impact pathway of the case under consideration, from the start of research to the materialization of impacts. It identifies them and describes their material and immaterial contributions, as well as their respective roles in carrying out research, circulating knowledge or transforming research products. It thus highlights the interactions between actors, as well as the causal links between the different stages of the impact pathway.

8This approach uncovers the network of actors who participated in building the impact and positions their contributions within the impact pathway.

Accounting for the diversity of impacts

9The effects of research on society and its environment affect various different fields (economic, health, cultural, etc.). These may be positive or negative, direct or indirect, expected or unexpected, established or developing, in the short-, medium- or long-term. The ex-post case study method integrates the multidimensional nature of the impact into its standardized assessment tools.

Standardized tools for the assessment and presentation of results

10For the performance of a case study and the presentation of its results three standardized assessment tools are used: the impact pathway diagram, the dimensions of impact, and the timeline.

11These tools make it possible to report on the particularities of each case, put the cases into perspective, and thus to draw generic lessons. The results can thus be observed at different scales, from the laboratory to the institution.

• The impact pathway diagram is a chart that summarizes impact production in five phases: the actors in the research process and their contributions (actors' contributions), the products resulting from this research (research products), the actors that contributed to the circulation and adaptation of these products and the resources applied to such end (dissemination of knowledge and intermediaries), and lastly the impacts observed in the form of two phases (initial impacts and widespread impacts). The diagram also includes contextual elements.
• The dimensions of impact refer to the primary areas (environmental, social, economic, etc.) in which research impacts are likely to materialize. The tool promotes a systematic exploration of the nature of the impacts observed and facilitates their identification and characterization. These broad areas can be broken down in various ways, in particular based on the different research institutions and their missions, their objectives, and the fields of application of their research work.
• The timeline presents a picture of the case under consideration in chronological terms, from the start of research to the impacts observed. It temporally situates the impact, the diversity of the actors, the salient elements of the case, and the key moments in the impact pathway: the contextual events (scientific, social, etc.) that influenced the progress of the research and the materialization of impact; engagement of actors; primary products of research; and materialization of the initial impact.

12The use of these tools is detailed in the chapters on the construction of the impact pathway (diagram of impact pathway: see figure 1 and on the practical realization of a case study (timeline: see figure 2).

13The case studies are informed by documentary information and interviews with stakeholders in the impact pathway. Identified impacts are documented and supported by factual evidence collected from impact pathway actors and impact recipients.

14The study results in a report prepared according to a standard framework. This report is submitted to the interviewees for their opinion.

Adaptation of assessment tools for IRD

15Based on these guiding principles, the methodological group made its choices in terms of impact characterization, established the criteria and procedure for case selection, and framed the conduct of the studies.

Dimensions of impact selected

16In Miriades, six dimensions of impact were selected to characterize the societal impact to which the institute's research has contributed:

• Academic;
• Economic;
• Environmental;
• Institutional;
• Public policy;
• Sociocultural, health and educational.

17The impact in terms of capacity development cuts across all these dimensions. Initially identified as an independent dimension, it ultimately seemed more coherent to report on it in the context of each of the six dimensions. This choice also makes it possible to better differentiate the capacity deve­lopment process as a driver of impact generation from capacity development as impact.

18The choice of this frame of reference results from a consideration of the criteria associated with IRD's missions and the contexts in which it pursues them. It thus takes into account:

• IRD's primary mission – to produce science focused on the intertropical and Mediterranean zone – based on an equitable scientific partnership with the higher education and research communities of the concerned countries and regions. This mission has a dual objective: to contribute to advances in scientific knowledge in matters of sustainable development, and to help give development policies a better grounding in scientific knowledge;
• the development challenges in the countries with which the institute works;
• the diversity and characteristics of the partnership contexts in which IRD research teams work;
• the potential areas of application of the impacts of the research conducted at the institute. These are structured around major themes addressing global challenges such as improving the health of populations, reducing inequalities, managing and conserving biodiversity, understanding global changes, quantifying hazards and risk reduction, or sustainable management of ocean resources.

The choice of a qualitative approach

19Research impact analysis may be qualitative or quantitative.

20Unlike the Asirpa and Impress methods, which combine both qualitative and quantitative analyses, the Miriades project opted instead to take a qualitative approach, for two reasons. The first is to avoid the risk that the tool would tend to be misunderstood by those who might see a quantification of impacts as a new research assessment metric. The second involved the complexity of building a tool to perform such assessment, which lends itself more to a qualitative than to a quantitative approach.

Case selection procedure

21The choice of criteria and the selection process used for the chosen cases were guided by three requirements:

• the availability of a diversified set of research themes to test the method;
• involvement in the process by the scientific departments in charge of coordinating the institute's scientific policy;
• identification of volunteer researchers to lead the case studies.

22A set of twenty-six subjects deemed likely to serve as case studies was compiled from several sources: a review of IRD scientific news sheets published between 1996 and 2010 (consisting of 360 news sheets), proposals from the scientific departments and proposals from the project's methodological group.

23Cases were selected based on the following criteria:

• diversity and extent of the impacts observed;
• response to development challenges;
• significant contribution of IRD and its partners to the impact;
• academic results;
• representativeness of thematic and disciplinary fields;
• type of partnership established (higher education and research, businesses, non-governmental organizations [NGOs], community organizations, non-academic institutions, etc.);
• ways of disseminating results (patent, expertise, tool, method, etc.);
• diversity of geographical and cultural contexts;
• human resources available to conduct these studies.

24After this initial selection, the selected cases were the subject of a feasibility analysis by the case's potential scientific rapporteur and project team. This resulted in the formulation of an impact hypothesis; an initial definition of the scope of the case; an assessment of the accessibility of the data necessary for case documentation; and a draft timeline and impact pathway diagram.

25At the end of this selection process, nine cases were selected for the Miriades project (Figure 1).

Composition of case study teams

26Each case study was led by a scientist, serving as scientific rapporteur and responsible for the study. This rapporteur had individual support in regard to methodological, editorial and documentary aspects from a consulting member of the project team trained in the method.

27Each of these studies was also monitored, depending on the case, by one or two members of the methodological group, based on the themes addressed and the diversity of the impacts observed. They provide an external perspective and advice throughout the study.