Research Assessment and Open Science in a Diverse World
p. 121-128
Texte intégral
1I would like to start by recalling that the Open Science project has been in existence for decades, although we can say that it has entered a consolidation phase in recent years. The Unesco Recommendation for Open Science1 is a step in this direction. It provides a general framework, a definition that tends towards a global consensus, a set of principles and values and, finally, a list of actions to face. Among the latter, it is important to mention the need to align the incentives of the research evaluation systems with the progress of open science. How can we promote open science through the evolution of research evaluation modes?
2Among the various points concerned, it is fundamental to consider all the missions of universities, beyond publication activities, which we assimilate to research activity. I am referring to teaching, but also to what we in Latin America call extensión universitaria (third mission). When working on the Unesco Recommendation, the experts realised that there was not a general consensus on the instruments of promotion and recognition of open science in research evaluation. It is therefore essential to recognise the plurality of incentives that exist, and to consider them in context.
3The Unesco member countries that participated in the making on this recommendation also discussed at length one critical issue in the road to open science: inequalities. There are indeed visible inequalities today between what is commonly called the North and the South, between countries that could be described as peripheral or semi-peripheral and central countries. There is a significant digital divide, particularly in terms of infrastructure. The increase in these inequalities can also be seen in the transition to open access as it is currently taking place in European journals, particularly in connection with the move to the Article Processing Charges (APCs) model. In Latin America, this model represents a huge difficulty, due to the very high amounts involved. The Recommendation also defends multilingualism because English is unfortunately often the only code for interoperability between and across databases.
4I would also like to mention another asymmetry that is important for the transformation towards openness in research assessment, namely the very notion of excellence. I have tried to represent in a diagram the way in which the differentiated vision of the North and the South can be seen in the power relations of the academic system. At the top is academic excellence, also more appropriately called 'research quality', which has been identified with mainstream science, frequently identified with the main stream of scientific publishing. At the very bottom of the hierarchy is the so-called peripheral science, i.e. everything that falls outside the circuit of what is rewarded in the traditional research evaluation system. Alongside this, there are various transnational circuits (notably the Latin American system), which were born directly out of open access, with a desire to open up the academic dialogue, as well as national circuits, which continue to exist. In these peripheral or semi-peripheral countries of the South, different levels of prestige conferred by publication have thus arisen: a national level of prestige (which I am observing here mainly for Latin America), a regional or international level of prestige, with different evaluation cultures for each.
Fig. 7: Traditional representation of the academic system

Source : Beigel, “Research Evaluation in the Southern Road to Open Science” and Fernanda Beigel, “Circuits and fields. The multi-scale process of academic prestige-building”, in Keim, W. et al., Handbook On The Circulation of Academic Knowledge. Forthcoming.
5There is a boundary between the different evaluation cultures, which distinguishes between what might be called national prestige, but which is in fact only recognised within the institution of origin and in a particular country, and what is called international prestige, but which, due to inequalities between countries, is in fact reduced to the scale of prestige specific to the country concerned. In Latin America, there is also a certain type of international prestige, which is truly shared within a group of countries, and which can therefore be described as regional, in connection with a very specific publishing circuit.
6The left arrow on my diagram represents the fact that when we ask what academic excellence is, we are in fact asking what the best science is: the one that has the most impact? Given that this notion does not originate in this region of the world this is how academic excellence has been defined in the countries of the South. I have shown it separately from the diagram, because it is partly imposed from outside, due to the internationalisation of the academic research system developed from the 1990s. Open science places a precise challenge to this inequality in terms of the autonomous definition of the research agenda.
7Let us mention the case of China, in relation to the notion of imposed excellence. In the reforms currently underway in China, we recognise the perverse effects that this type of heteronomous imposition can have, notably through the very rapid adaptability that Chinese researchers have shown, with the generalisation of the use of English and the advent of a fracture between research concerns and societal concerns, all in the name “impact”, but eventually embodied in the Impact factor. It is therefore important to emphasise that the reform underway in China shows the need to support national policies to assist local scientific publication.
8If we now look at the other relevant concerns of experts and Member States when developing the Recommendation for open science, we see that these were essentially related to global inequalities in access to resources and infrastructures.
Fig. 8 : Open Access Repositories

Source: Beigel, “Research Evaluation in the Southern Road to Open Science”. Based on data extracted from re3data.org - Registry of Research Data Repositories. https://doi.org/10.17616/R3D, accessed January 2022.
9To go further in this study of global inequalities, let us look at the global landscape of repositories with scientific production in Southern countries. It can be seen in Figure 8 that there are not yet many publication repositories in Africa, but that Latin America is among the best equipped countries in the South. On the other hand, if we look at Figure 9, that is open data repositories, we can see a glaring difference in the number of repositories between the North and the South, including in Latin America, even if the contrast is still greatest with Africa.
Fig. 9 : Open Data Repositories

Source: Beigel, “Research Evaluation in the Southern Road to Open Science”. Media used during the presentation (available on the conference's website). Based on data extracted from re3data.org - Registry of Research Data Repositories. https://doi.org/10.17616/R3D, accessed January 2022.
10Other inequalities can be observed in the current global distribution of research information systems or CRIS (Current Research Information System), which allow for the fully integrated administration of all kinds of information that are currently managed independently within the countries of the South, i.e. interfaces for the management of scientific policies, human resources or research projects on the one hand, and open archives or open data warehouses on the other. CRIS systems link the repositories and information databases, providing highly interconnected and interoperable open science data. There are about 950 CRISs listed in EuroCRIS2, most of which are located in Europe and northern countries.
11Next is another type of global asymmetry, affecting to publications, which can be seen when looking at the distribution between Diamond Open Access journals and open access journals with APCs. South America has 2,612 diamond journals listed in DOAJ and only 130 APCs journals, which is a peculiarity of this region. Western Europe has nearly 4,300 journals, of which 1,907 are open access with APCs. The latter are generally the most prestigious, and therefore the most sought-after by scientists in the South. But how will they pay individually for an average 2.000 dollars to publish a paper?
12I would now like to focus more on the linguistic face of the global inequalities in science. There are a number of tangible initiatives around the main challenges facing our continent, namely multilingualism and the hypercentrality of English. There are initiatives such as the Helsinki initiative3, the Latin American Forum for Research Assessment4, which launched CLACSO, or the Jussieu Call5, which insists that to have a real impact on society, research must be multilingual. Let's look at the particular strengths and weaknesses of Latin America in terms of bibliodiversity and open access. There is a strong historical regionalism since the 1950s, with the early development of national information management systems, although, as mentioned earlier, these systems were disconnected from other bases such as open access repositories. Regional indexing systems have also been set up, allowing the professionalisation of thousands of diamond open access journals. Latin America also has a very strong and dynamic publishing industry, with books published since the 1930s, and a very long tradition of non-commercial open access.
13In addition to these strengths, there is legislation in several countries in favour of open publications and open data, and a federation of repositories with more than 790 institutions in more than 10 different countries. An initiative led by CLACSO, which lists the different actions carried out in Latin America, gathers interesting data on this theme, and specifies that there are 57 instruments relating to open science policies, the majority of which are based on collaborative infrastructures. I also would like to briefly delve in an important tradition of citizen and participatory science, the so-called extension universitaria (or third mission of universities), which has a long history of co-production of scientific knowledge in collaboration with the social sector. All these interactions must be highlighted in the research assessment systems but also revalued in the transition to open science.
14One of the key points in this respect is multilingualism. The hyper-centrality of English is now absolutely clear in the research world. However, although English is a dominant reality, it is not the only reality, and in South America we have a number of channels for the circulation of knowledge that bypass the use of English. For example, if we search Scopus or the Web of Science, we find Spanish and Portuguese articles as marginal shares. In contrast, the OLIVA6 database, which combines entries from SciELO and Redalyc and has 345,391 entries in Spanish, more than the Web of Science (270,632 entries) and almost as many as Scopus (373,419 entries). The same observation can be made in BIBLAT7 and LA Referencia8, the latter database containing many more references in Portuguese than Scopus and the Web of Science.
15Well now, how “aligned” to these strengths are the rewards promoted by the research assessment systems in Latin America? we can observe a contradiction, even a form of “alienation” between these regional potentials for open science and the orientations led by the evaluative policies regarding individual careers, institutional accreditation and research funding. There is indeed a certain dichotomy between robust and highly developed public infrastructures and an evaluation system, which relies largely on the impact factor and other criteria imposed by the North, but which are supposed to represent a universal standard, that of the large mainstream journals. In effect, the national classification for researchers and the national funding agencies use the impact factor or the h-index to assign positions or resources. It is necessary to start changing our perspective. We used to represent our arrow as going from North to South: now let's go from bottom to top, focusing on the quality of the research and its relevance to society. We can start to see that there are different possible production and circulation channels, which promote research and interaction with society, and can contribute to citizen science. It is this diversity of practices that research evaluation must address.
Fig. 10: To invert the path to defining research quality

Source: Beigel, “Research Evaluation in the Southern Road to Open Science” and Beigel, “Circuits and fields. The multi-scale process of academic prestige-building”.
16Eventually one of the main challenges in South America is to invert the path towards defining research quality autonomously in accordance with a public ecosystem with a robust infrastructure and highly developed resources, which supports diamond journals, as well as many alternative circuits, especially at the local level. The current evaluation system pushes researchers to look up to mainstream criteria, and distances them from regional ecosystems of scientific production and communication. This is why we need to set up a regional CRIS, which already exists in Peru, Brazil and Argentina. Datasources are a fundamental issue to broaden up the landscape of scientific production under assessment, as is the need to really stimulate the multilingualism of local productions by giving them more visibility in international databases such as SciELO, Redalyc or BIBLAT.
17In South America, there are many small-scale journals published by small research teams or universities. At the opposite end of the spectrum, mainstream bibliographic databases include very few Latin American journals. Latindex, SciELO, Redalyc and BIBLAT show it is possible to publish quality journals, with a level of excellence defined collectively in the academic community with a necessary balance between local standards and international criteria, a balance that must be pursued through horizontal and not vertical imposition.
Notes de bas de page
1 UNESCO, “UNESCO Recommendation on Open Science”, 2021, https://unesdoc.unesco.org/ark:/48223/pf0000379949.
2 “euroCRIS | The international organisation for research information”, accessed 16 March 2022, https://eurocris.org/.
3 Federation Of Finnish Learned Societies et al., “Helsinki Initiative on Multilingualism in Scholarly Communication”, 2019, https://figshare.com/articles/Helsinki_Initiative_on_Multilingualism_in_Scholarly_Communication/7887059.
4 “FOLEC - Foro Latinoamericano Sobre Evaluación Científica”, CLACSO (blog), accessed 16 March 2022, https://www.clacso.org/en/folec/.
5 Serge Bauin et al., “Jussieu Call for Open science and bibliodiversity”, 10 October 2017, https://jussieucall.org/jussieu-call/.
6 Fernanda Beigel et al., “OLIVA: una mirada transversal a la producción científica indexada en América Latina. Diversidad disciplinar, colaboración institucional y multilingüismo en SciELO y Redalyc” (SciELO Preprints, 19 July 2021), https://doi.org/10.1590/SciELOPreprints.2653.
7 “Biblat - Bibliografía latinoamericana”, accessed 16 March 2022, https://biblat.unam.mx/fr.
8 “LA Referencia”, accessed 16 March 2022, https://www.lareferencia.info/es/.
Auteur
-
Fernanda Beigel
CONICET National University of Cuyo-Mendoza, Argentina

Le texte seul est utilisable sous licence Creative Commons - Attribution 4.0 International - CC BY 4.0. Les autres éléments (illustrations, fichiers annexes importés) sont « Tous droits réservés », sauf mention contraire.
Livre blanc — Une Science ouverte dans une République numérique
Direction de l’Information Scientifique et Technique - CNRS
2016
White Paper — Open Science in a Digital Republic
Scientific and Technical Information Department - CNRS
2016
Livre blanc — Une Science ouverte dans une République numérique — Guide stratégique
Études et propositions en vue de l’application de la loi
Direction de l’Information Scientifique et Technique - CNRS
2017
White Paper — Open Science in a Digital Republic — Strategic Guide
Study Review and Proposals for Implementing the Act
Scientific and Technical Information Department - CNRS
2017
Connecting the Knowledge Commons — From Projects to Sustainable Infrastructure
The 22nd International Conference on Electronic Publishing – Revised Selected Papers
Leslie Chan et Pierre Mounier (dir.)
2019
Actes des Journées européennes de la science ouverte
Open Science European Conference – OSEC 2022
Open Science European Conference
2022
Proceedings of the Paris Open Science European Conference
OSEC 2022
Open Science European Conference
2022