Versión clásicaVersión móvil

Cyberidentities

 | 
Leen D’Haenens

Part Three: Communities in Cyberspace

Policy-Making and the Value of Electronic Forms of Public Debate: Underpinning, Assumptions and First Experiences

Cees Leeuwis

Resumen

This paper investigates the potential value of electronic forms of debate in governmental policy processes. For this purpose, some strengths and weaknesses of conventional forms of political debate are discussed. On the basis of insights derived from communication science and social psychology, a number of preliminary assumptions are generated. These assumptions address the way in which electronic forms of debate may compare with conventional forms of debate, keeping in mind the strengths and weaknesses of the latter. It emerges that – under specific conditions – electronic forms of debate may have various advantages over conventional forms of debate. These potential advantages are, amongst others: (a) a larger number and variety of participants; (b) less occurrence of time pressure; (c) more egalitarian participation; (d) more extensive articulation of arguments; (e) more flexible information provision during the debate; (f) better opportunities for in-depth debate along a greater diversity of discussion lines; and (g) greater freedom and openness during the debate. A major weakness of electronic forms of debate may lie in their limited capacity to facilitate a group identity, thereby having counterproductive effects on consensus formation and conflict resolution. On the basis of some initial research experiences, the paper sets out to critically reflect on the extent to which the assumed advantages and disadvantages have materialized in actual practice

Texto completo

I INTRODUCTION

1One of the ways that new information and communication technologies are expected to contribute to the renewal of democratic processes is through an enhancement of societal debate within governmental policy processes (Van Dijk, 1991; Percy-Smith, 1996). This type of “teledemocracy” usually takes place in the form of electronic debates that are organized and operated by means of e-mail discussion lists or more advanced debating software on the World Wide Web. This article investigates the potential value of electronic forms of public debate in policy processes. Thereby, its scope is limited to those forms of debate which are purposefully initiated by governmental institutions in an effort to involve citizens in policy-making.

II ELECTRONIC DEBATES FROM A COMMUNICATION SCIENCE PERSPECTIVE

2A key question to be addressed is whether or not electronic media have specific properties which may lead to an improvement of the quality or effectiveness of public debate within policy processes. However, this question is hard to address in its present form. First, notions like “greater quality” and “effectiveness” have multiple dimensions (e.g., access, degree of participation, depth, relevance for policy-making, etc.), and can only be assessed on the basis of a normative framework. Second, the notion of “specific properties” is problematic. Even apart from the fact that different forms of electronic debate exist, various authors have stressed that the characteristics and specific properties of a particular electronic medium may depend on both the social context in which it is used (Fulk, Schmitz & Steinfield, 1990), and on the content of what is being communicated (Leeuwis, 1996). Rafaeli (1988: 119-20), for example, points out that “interactivity” should not be looked at as a medium-characteristic. In his view, the level of interactivity is constituted by the extent to which later communicative acts address the relation between two or more preceding statements (Rafaeli, 1988: 111). In this vision, media may pose preconditions and boundaries to interactivity, but they certainly do not determine the actual level of interactivity in a specific social setting. A face-to-face conversation offers ideal opportunities for interactivity but may in fact take place in a totally non-interactive manner (e.g., if people are not prepared to listen to each other), while the opposite may be true for an exchange which is mediated by a medium which is less optimal in this respect (e.g., a letter). A similar mode of reasoning can be applied to other so-called “media-characteristics” like speed, user-control, information richness (Trevino, Daft & Lengel, 1990), and the like. These “characteristics” must rather be looked at as features of communication processes that are shaped by active human agents in a specific setting.

3The above discussion is relevant because it makes clear that the appropriateness of electronic media for “bridging the gap” between citizens and administrators will probably depend more on the attitude of the parties involved than on the medium itself. When governmental staff or citizens are not genuinely interested in a constructive debate, the use of ICT will certainly be useless. However, when there is a serious intent to communicate, the parties involved can make use of such a variety of media and discussion techniques that the added value of electronic debates is not immediately evident. The above discussion also teaches us that a study of the potential added value of electronic forms of debate should not start primarily from the “media characteristics” of electronic media, but first and foremost from the features (and problems) of specific communicative practices and settings: in this case, public debates in policy processes. This is why this chapter will first address the strong and weak points of conventional forms of (face-to-face) public debate, like the ones initiated by Dutch governmental institutions (see e.g., Rathenau Institute, 1994; Van de Poll & Glasmeier, 1997), and only then reflect on possible contributions of electronic forms of public debate in a similar setting. In this manner, we may get an idea of the potential added values and/or undesirable consequences of such electronic debates.

2.1 Characteristics of public debates in policy processes

4As indicated above, an analysis of strengths and weaknesses of conventional or electronic forms of public debate in policy processes can only take place against the background of certain normative considerations. The following three normative criteria are used:

  1. In a public debate all relevant and valid arguments must be raised, and considered, in the generation of conclusions.
  2. A public debate must take place in an “open” fashion; participants must express a certain freedom of speaking, listening and thinking. In an “open” debate, arguments (rather than interests, power and positions) must be decisive (see Habermas’ [1981] notion of “communicative action”).
  3. A “good” public debate has an influence on both the governmental policies generated and the public acceptance of these policies.

2.1.1 Selective participation due to place and time constraints

5Because conventional forms of debate take place at specific places and times, a high degree of coordination and mobility is required from the participants, who are often representatives of societal interest groups and laypeople. This means that certain selection processes will take place. In a recent debate on nature development (Van de Poll & Glasmeier, 1997; Aarts & Te Molder, 1997), for example, it was difficult to find farmers who could spare the time to participate for several weekends during their busy season. Moreover, due to the physical presence of the participants, it is often considered impractical to accommodate more than 40 or 50 participants.

6At first glance, time and place play a less dominant role in electronic forms of debate. During a debate over several months people can contribute at any time, and at any place where Internet facilities are available. Also, it seems easier with electronic debates to accommodate a higher number of participants. However, it is clear that other forms of participant selection will take place relating to computer access and skills, as well as to writing capacities.

7Especially if one assumes that access to Internet facilities will increase significantly in the future, it seems that electronic forms of public debate can develop some advantages over conventional forms:

Assumption 1:
For those policy issues where a relatively large number of interested people and/or stakeholders have access to Internet, electronic forms of public debate can accommodate a larger number and wider variety of participants than conventional debates (given a certain amount of organizational input).

Assumption 2:
The larger the number and wider the variety of participants in a public debate (to a certain degree of saturation), the greater the chance that (a) relevant arguments and perspectives are raised; (b) policy makers will incorporate some of the conclusions in their policies; and (c) citizens’ acceptance of policies is increased.

2.1.2 The “peak” rhythm: Concentration of the debate in time

8Conventional debates tend to be concentrated in time; that is, they take place at a series of well defined meetings of variable length (ranging from hours to days). Dining these periods the intensity of the debate is high, while communication tends to very limited in the long intervals between meetings. During the meetings, the verbal mode of communication generally results in many short statements being generated at a high speed. Although this “peak” rhythm provides participants with a clear sense of the amount of time to be invested in debate, we have seen that it may also result in excluding particular categories of people. Moreover, the peak rhythm tends to go along with a lot of time pressure during the meetings, thus limiting the number of issues that can be thoroughly discussed. Also, the rapid pace of exchange tends to allow fast thinkers and verbally strong participants to dominate the debate.

9Electronic “meetings” can be much longer in duration than conventional ones. The rhythm of electronic debates is not only more even (i.e., less concentrated), but also slower. In a lively electronic debate one can expect some 10 or 20 contributions per day, instead of some 100 speech-acts per hour in a conventional debate (with fewer participants). The nature of these contributions tends to be different as well. In addition to short reactions, some participants tend to write long and well prepared analyses, based on careful study and editing. A disadvantage of all this may be that participants have to spend more – if fragmented – time in order to follow and participate in the electronic debate. At the same time, such debate puts the easy writers in an advantageous position, even if less capable writers have more time for editing. A possibly relevant research finding in this respect is that participation in electronic debates tends to be more equally distributed than in conventional debates (Dubrovsky, Kiesler & Sethna, 1991; Kiesler & Sproull, 1992). While these authors attribute this phenomenon to a reduced influence of power and status (see below), others have suggested that (time-)technical dimensions play a role (Spears & Lea, 1994).

10These considerations lead to the conclusion that electronic debates may have a couple of additional advantages over conventional debate:

Assumption 3:
Because of the “(time-)technical” conditions that go along with electronic forms of debate, time pressure will play a less significant role than in conventional debates.

Assumption 4:
A reduction of time pressure during debate will result in (a) a more egalitarian use of participation opportunities; and (b) better prepared and more thoughtful contributions by the participants. Hence, the reduction of time pressure will lead to an increase in the number of active participants and to a more extensive articulation of relevant arguments, both of which may in turn result in increased policy influence and policy acceptance (see assumption 2).

2.1.3 The timing of collective memory and information provision

11For purposes of preparation, participants in conventional public debates are usually provided with a selection of articles, reports and/or presentations by experts before the actual debate starts. During debate, it frequently happens that lines of argument develop which had not been anticipated in the preparatory information provision. Due to the peak rhythm of conventional debates, it is often impossible to correct this. At such points the debate is either interrupted or continues on the basis of sub-optimal information. The information provision about the debate itself usually takes place through note-taking by individual participants (a somewhat distracting exercise), and/or through summaries by a discussion leader, or minutes.

12In many ways information provisions preceding electronic debate resemble that in conventional debates. If written sources are supplied in electronic form, it may be easier to provide a somewhat greater selection of preparatory material from which participants can choose. During the debate itself, however, information provision may differ significantly. Moderators or participants can at any time bring in new documents or resources, and these can be immediately incorporated into the electronic debate. Also, the debate itself is completely documented; all original contributions are available to everybody.

13Again, we can assume a potential added value of electronic debates:

Assumption 5:
The relatively slow rhythm of electronic debates – when compared with conventional ones – makes it possible for the provision of information into the debate to be adapted and extended while the debate unfolds.

Assumption 6:
As the flexibility of the information provision during a debate increases, participants feel better-equipped to arrive at a well-considered conclusion.

2.1.4 Sequential organization and the role of the discussion leader

14Plenary meetings and parallel sessions are discussion methods used in conventional debates, and the latter make it possible to discuss different topics simultaneously. An important limitation, however, is that each participant can participate in only one line of debate at the same time. In that sense conventional debates are sequential in nature. Even if there is some space to maneuvre for participants hoping to influence the agenda in a meeting, we often see that in order to reach a conclusion within a certain time-frame, discussion leaders tend to be rather strict in ensuring that the discussion does not deviate into “side-lines” which are deemed “irrelevant.” In doing so, however, potentially relevant topics may not be sufficiently dealt with. A bad chairperson can do a lot of damage in this respect.

15Existing debating software allows participants to participate in several discussion lines (or “meetings”) at the same time. Moreover, it is relatively easy to open a large number of new discussion lines if desired. In order to arrive at a conclusion, discussion leaders (moderators) can provide summaries and proposals to which the participants can react. In order to prevent a debate from spreading like an oil slick, a certain amount of control remains necessary. Given the fact that going into separate “side-lines” does not necessarily prevent progress within “main-lines,” such control can be less stringent than in a conventional debate. Here too, bad moderators can be detrimental. However, the rhythm of electronic debates probably makes it easier to correct discussion leaders if necessary.

16In all, it seems that electronic debates can be structured more flexibly than conventional debates. This leads to the following potential advantages:

Assumption 7:
The opportunities for structuring a debate more flexibly make it possible for in-depth discussion to take place along a larger number of discussion lines, and with a greater number of participants per line than in conventional debates. This increases the chances that participants “have their say,” which in turn contributes to the identification and evaluation of relevant arguments.

Assumption 8:
In electronic debates, ineffectual discussion leaders are more likely to be corrected than in conventional debates.

2.1.5 Social presence and convergence

17In conventional debates participants not only use written or verbal language, but also non-verbal forms of communication. Moreover, the exchange of ideas can be supplemented with social events like eating, walking, drinking, etc. Thus, there is a high degree of “social presence” (Short, Williams & Christie, 1976). Social psychological studies point out that in such conditions less polarization may take place (Kiesler, Siegel & McGuire, 1984; Siegel, Dubrovsky, Kiesler & McGuire, 1986; Spears, Lea & Lee, 1990) and that there may be a greater willingness to compromise (Morley & Stephenson, 1970; Spears & Lea, 1992) than in more “anonymous” computer-mediated conditions. This phenomenon is not only attributed to a greater degree of group identification, but also to differential power and status processes. As mentioned earlier, the finding that electronic communication often goes along with more equal levels of participation has been connected by some to a supposedly reduced influence of power and status (Dubrovsky et al., 1991; Kiesler & Sproull, 1992). In this view, participants in electronic forms of communication tend to speak out more freely as they are less constrained by the social influence of others, as the latter’s social presence is reduced. This, then, goes along with phenomena like “flaming” (Siegel et al., 1986). In contrast, power and status processes would play a more significant role in conventional debates, and provide the necessary social pressure for participants to accept compromises. However, research findings on the willingness to compromise are not consistent (see Spears & Lea, 1992), and the thesis of reduced status and power influences has been contested. Spears and Lea (1994) argue that more equal participation in electronic forms of communication has more to do with time-technical issues (see 2.1.2). According to them, participants can take on a variety of identities in a discussion (see also Aarts & Te Molder, 1997), which implies that the question of whether or not an agent conforms to others can only be answered against the background of the specific identity that a participant takes on. They show that social influence always plays a role if one takes into account the “active identity.”

18Although we can conclude that power and status are influential in both conventional and electronic forms of debate, it remains likely that a group identity is formed more easily in conventional debates. Hence, it is plausible that in the latter it is more easy to reach compromise on eventual conclusions. Assuming that some degree of compromise and consensus about conclusions is a precondition if one wishes to influence and/or direct policy, we can formulate the following assumptions;

Assumption 9:
The reduced social presence in electronic debates (when compared with conventional debates) hampers the formation of group identity, and reduces the chances that the participants reach agreement.

Assumption 10:
The larger the degree of agreement in the outcomes of a public debate, the greater the chance that the debate has an influence on policy formation and policy acceptance.

2.1.6 Negotiation, openness and flexible identities

19The fact that power plays a role in all communication implies that in many ways we are dealing with a negotiation process (see Leeuwis, 1996). The negotiation metaphor is even more appropriate for public debates since stakeholders and their representatives often participate. At the same time such debates imply a learning process. During the debate participants gain insight into diverging views, perspectives and interests. This merging of communication, learning, and negotiation goes along with several complications. Participants tend to forward those arguments which are in line with their position, role and interest (see Aarts & Te Molder, 1997), and find it threatening to give up an argument, adopt another, or engage in a new line of thinking for which the consequences cannot be overseen. Moreover, putting forward particular views and arguments may damage or disturb social relations. A discourse analytical study of a public debate (Aarts & Te Molder, 1997) suggests that participants make use of a repertoire of identities (e.g., group member, stakeholder, layperson, expert) in forwarding and/or combating particular views and arguments. Thus, different identities are not used to present diverging views and arguments, but rather to construct a coherent story. Of course, the repertoire of identities that participants may take on sensibly is limited, even if it can be expanded by speaking on behalf of others (i.e., on enrollment or recruitment, see Latour, 1996).

20In all, stakeholders often hold back and stick to their views, even if – from yet another identity – they may feel sympathetic to other positions. We frequently see that such risk-avoiding behavior reduces the openness within conventional debates, and thereby hampers learning and progress on joint policy recommendations. Both the limitations in the repertoire of identities, and the tendency to stick to particular arguments and views (i.e., the lack of openness in the debate) are connected to the fact that in a conventional debate all statements can be traced to particular individuals whom other participants expect to show a certain degree of consistency. In other words, participants can suffer negative consequences from daring statements and identity shifts because they can be identified as individuals.

21In principle, similar processes to those described above are likely to take place in electronic debates. An important difference may be (depending on the set-up of the debate) that the participants get to know each other less well, and may be less aware of one another’s societal positions. This may make it less clear why people put forward particular views and arguments. Despite this relative “anonymity,” it is quite common in electronic debates (at least those in policy processes) that contributions can be traced to specific individuals. Participants usually have a recognizable electronic address, and often sign contributions with their own names; both in principle create risks comparable to conventional debate. However, in other types of debate on the Internet (e.g., chat-boxes and newsgroups) we see that people have a greeter freedom to play around with their identities (e.g., in terms of their sex, race, and location, see Turkle, 1995), precisely because they are anonymous and non-traceable. From a technical point of view, it is quite possible (at times) to give participants in electronic public debates the opportunity to communicate in a non-traceable fashion. In such conditions, the risk of suffering negative consequences from one’s statements is reduced, so that participants may be more prepared to engage in an “open” debate. That is, they may speak out more freely and explore new modes of thinking. A potential disadvantage is that this may lead to more polarization, flaming, and bending of facts. Nevertheless, non-traceable communication may be an attractive option for individuals who find themselves in a sensitive position due to media attention, the interests that are at stake and/or their function (e.g., government administrators).

22The above considerations lead to the formulation of the following assumptions:

Assumption 11:
Participants (and especially those in sensitive positions) tend to be more prepared to engage in an “open” debate when they have (at times) the opportunity to contribute to the debate in a non-traceable fashion.

Assumption 12:
Within electronic debates it is much easier to build in facilities for non-traceable communication than in conventional debates.

2.1.7 Participation of decision makers and the influence of the debate

23Active participation in a public debate requires that the participants have confidence that the results will be taken seriously by policy makers. Experiences with conventional (and electronic) public debates suggest that the involvement of decision makers may be important for generating a commitment to use the results of the debate, and also for ensuring that decision makers can apply the debate results. However, it is often practically impossible for decision makers to play a role in a public debate.

24In electronic debates it is – from a logistical point of view – much easier to involve decision makers in one way or another. Decision makers can follow the debate behind the scenes, and contribute if necessary, with or without making use of facilities for non-traceable communication. In the first instance, it is important that the participants know that decision makers are taking part; rather than being “anonymous” the latter may at times be untraceable.

25The above can be summarized in the following two assumptions:

Assumption 13:
Actual involvement of decision makers in a public debate enhances: (a) the motivation of participants to participate actively; (b) the chance that conclusions emerge which decision makers can use; and (c) the commitment on the side of decision makers to use the results.

Assumption 14:
For logistical reasons, it is easier for decision makers to be involved in electronic debates than in conventional debates. This involvement will be even easier if opportunities for non-traceable communication are provided (see assumption 12).

2.2 Conclusion on advantages and disadvantages of electronic forms of public debate

26The foregoing shows that it is plausible that electronic forms of public debate may have several advantages over conventional forms, given a number of normative criteria and conditions. The most significant disadvantage is that it may be relatively difficult to reach agreement on conclusions among participants in an electronic debate. The most fascinating potential added value is that electronic debates may contribute to creating conditions for what Habermas (1981) calls “communicative action” in an “ideal speech situation.” Even if power undoubtedly plays a role in electronic debates (so that a real “ideal speech situation” will never exist, see also Leeuwis 1993: 98) it is significant that the chances for more egalitarian participation seem to be higher in electronic debates. Moreover, the greater openness that is likely to emerge in electronic debates is an important advantage from a Habermasian perspective. Another interesting added value seems to be that electronic debates provide an environment where people in sensitive positions (e.g., administrators and politicians) can communicate safely and easily with citizens. This would indeed imply a reduction in the gap between government and citizens.

III THE ASSUMPTIONS IN PRACTICE: A PRELIMINARY EXPLORATION

27As electronic forms of debate in policy processes are a relatively new phenomenon, few research results are available. Moreover, existing studies are neither designed to make a comparison with conventional debates, nor to test systematically the assumptions generated in this chapter. Nevertheless, it seems possible to draw some provisional conclusions in relation to some assumptions, and give some impressions with respect to others.

28First, it seems possible indeed – provided that debating software is already available – to involve more participants in an electronic debate with comparatively little effort (see assumption 1) (Leeuwis et al., 1997; Van Beelen, 1997; Crasborn, 1997; Hamers, 1997).

29It must be noted, however, that the moderation of debate appears to require a considerable amount of time, so that there may be limitations of scale in which a debate can take place sensibly (Leeuwis et al., 1997; Hamers, 1997). Thus, the significance of this added value must be nuanced. A second limitation seems to be that so far, participants in electronic debates tend to be mainly highly educated and middle-aged males with a job (Leeuwis et al., 1997; Hamers, 1997; Jankowski et al., this volume). Hence, it is misplaced to speak of a “larger variety of participants” (see assumption 2), unless one combines electronic forms of debate with conventional forms. In line with assumption 14, however, we do see that very busy administrators (Leeuwis et al., 1997) and civil servants (Boussen, 1997; Crasborn, 1997) find it possible to participate more actively in electronic forms of public debate than in conventional forms.

30As electronic debates also appear to be dominated by a relatively small number of participants (Leeuwis et al., 1997; Hamers, 1997; Jankowski et al., this volume) there seems little reason to speak of more egalitarian participation due to reduced time pressure (see assumption 4). More generally, it seems incorrect to speak of “reduced time constraints” and “less time pressure” (see assumptions 1 and 3). It emerges that in electronic debates, too, a number of time-related factors have a significant impact on the way in which the debate unfolds (e.g., slowness of the computer system; reading time; response time). Time pressure plays a role as well, as many participants find it difficult to find time to participate in their everyday routine as work and other duties continue as usual (Leeuwis et al., 1997).

31Given the reservations with regard to assumptions 1, 3 and 4, it seems unlikely that a wider variety of relevant arguments and perspectives have been raised in the electronic debates studied (although this issue has not been systematically explored). Nevertheless, government administrators and civil servants point out that they have been confronted with new and surprising insights and arguments which they would not have encountered otherwise (Leeuwis et al., 1997; Boussen, 1997; Crasborn, 1997). It seems that through the direct involvement of administrators and civil servants in electronic debates, a number of organizational “filters” are bypassed; filters that tend to remain intact in cases of conventional debate. Although it is too early to draw conclusions, it may be that such direct involvement of senior government staff will indeed go along with a greater impact from the outcomes of debates on policy development (see assumptions 2 and 4).

32A striking feature of recent electronic debates is that a number of specific opportunities have hardly been used at all. This holds in particular for the indicated potential for a flexible information provision and structuring of the debate, and for the opportunities concerning non-traceable communication. This makes it impossible to make concluding statements with respect to quite a few assumptions (5, 6, 7, 11 and 12). However, participants in some electronic debates indicate the need for a more active and creative moderation and structuring of the debate (Leeuwis et al., 1997). At the same time, observations in other electronic debates show that the active interventions of moderators often meet resistance or are simply ignored. Thus, the potential of various opportunities for structuring the debate may in actual practice be more limited than suggested (see assumption 7).

33With respect to assumptions 11 and 14, the impression remains that it is not so much the government administrators who tend to hold back under conditions of traceable communication, but rather those participants who somehow represent others (e.g., representatives of societal organizations and pressure groups, see Leeuwis et al., 1997). In addition, there are indications that it is tricky for civil servants to participate in traceable electronic debate, as others can be held responsible for their (“semi-black-and-white”) statements. Hence, there indeed seem to be particular categories of actors who might benefit from facilities for non-traceable communication.

34Finally, it emerges (in line with assumptions 9 and 10) that little convergence is achieved in electronic debate. Participants seem eager to put forward their own particular points of view but show little interest in the standpoints of others and are not inclined to adapt their views.

IV FINAL CONCLUSION

35On the basis of insights derived from communication science, it seems plausible that electronic forms of public debate have – under certain conditions – an added value vis-àvis conventional forms of public debate. A provisional exploration of current practice indicates that some of the postulated advantages are indeed realized. This holds, for example, for the larger number of participants and the involvement of government administrators. The latter implies a more direct form of communication between government and citizens. For other suggested added values – such as greater variation among participants and more egalitarian participation – there is little empirical evidence. The most significant disadvantage – the assumed lack of convergence in electronic debates – seems to be confirmed in actual practice. Furthermore, it is striking that several specific opportunities within electronic debates have hardly been used so far. Among these under-utilized opportunities are the most fascinating and challenging aspects of electronic forms of public debate, such as the assumed potential for information provision, structuring and moderation of the debate, and facilities for non-traceable communication. Actual utilization of these opportunities may lead to a greater added value of electronic forms of public debate in the future. However, an overall added value is most likely to occur in debates that are directed solely towards making an inventory of arguments and perspectives. If the goal is that participants reach agreement on a joint statement or a future policy, it is unlikely – given the problems that can be expected in creating convergence – that an electronic debate will suffice. In such cases, conventional forms of public debate seem more appropriate, possibly in combination with an electronic counterpart.

Autor

Lecturer and Senior Researcher at the Department of Communication and Innovation Studies, University of Wageningen, The Netherlands. E-mail: cees.leeuwis@alg.vlk.wau.nl

Salvo indicación contraria, el texto y otros elementos (ilustraciones, archivos adicionales importados) se puede utilizar bajo licencia OpenEdition Books License.

Esta publicación digital es el resultado de un proceso automático de reconocimiento óptico de caracteres.

Comprar

Volumen papel

amazon.fr
Buscar en OpenEdition Search

Se le redirigirá a OpenEdition Search