URL originale : https://books.openedition.org/pressesmines/3707
Strengths and weaknesses of the proposed methodology and areas for future research
Texte intégral
Strengths of the methodology
1I have attempted to present a better system for reviewing regulatory proposals designed to limit access to harmful content on the internet. My purpose is to bring more analytical rigor to the preparation of proposals. The impact assessment (questionnaire, cost-benefit analysis and accompanying steps) will require systematic consideration of:
- what is likely to happen in the absence of regulatory intervention (the “baseline scenario” or “counterfactual”);
- the benefits that the proposals seek to achieve, and how to measure them compared to the baseline (“success metrics”);
- the direct and indirect costs that the regulatory proposals may generate, including those that are difficult or impossible to quantify in monetary terms; and
- alternative approaches, both technical and institutional.
2I explore whether the methodological rigor used for regulation of electronic communications in Europe could be used as a starting point for internet regulation. I also look at the principles used in regulatory impact assessments in the United States – and increasingly in Europe – to see whether they can be applied to internet content regulation. United States “better regulation” principles require that agencies:
“assess both the costs and the benefits of the intended regulation and, recognizing that some costs and benefits are difficult to quantify, propose or adopt a regulation only upon a reasoned determination that the benefits of the intended regulation justify its costs.”1
3As pointed out by the office within the White House in charge of reviewing impact assessments,
“regulatory analysis … provides a formal way of organizing the evidence on the key effects -good and bad - of the various alternatives that should be considered in developing regulations. The motivation is to (1) learn if the benefits of an action are likely to justify the costs or (2) discover which of various possible alternatives would be the most cost-effective.”2
4A more rigorous impact assessment would require the proponents of regulation to identify a clear market failure that cannot be addressed by existing legislation, and to define objective success metrics. This would act as a healthy counterweight to politicians’ normal tendency to create new regulations to deal with new perceived problems emerging from digital markets (Sunstein,1996; Shelanski, 2013).
5The uniform methodology and peer review system I propose would permit knowledge to be shared, and best practices to emerge. Over time, best practices would emerge for each type of internet content problem (copyright infringement, privacy, child pornography, protection of minors) and any new regulatory proposal would be measured against those best practices.
6Finally, the methodology would help distinguish measures adopted in democratic countries like France from measures adopted in totalitarian regimes to censor content. Without a robust benchmark against which to measure proposals, it is difficult to criticize other countries for adopting blocking measures when the country doing the criticizing also applies blocking measures for its own content policies.
Weaknesses of the methodology, and possible responses
7There are a number of reasons why the system proposed here may not work as expected. Some of these reasons have already been mentioned in Chapter 5 in connection with criticisms of better regulation methodology.
Criticism 1: Interference with democratic debate
8The first objection is that the rigors of a well-done impact assessment are incompatible with the sometimes messy trade-offs in the political process. Measures taken to limit access to harmful content involve fundamental rights and societal values that require political debate in the legislature. Debate on these issues cannot be reduced to a formula. The framework for regulating electronic communications, which deals with economic and technical questions of limited political impact, may not be transposable to questions involving fundamental rights and societal values. For measures involving the fight against terrorism or child pornography, the trade-offs involve public fears and moral values that are not always compatible with rational regulatory analysis.
9The response to this criticism is that rigorous analysis of regulatory options does not preclude emotional political debate (Hahn, 2004). On the contrary, a conscientiously-drafted impact assessment can become an important element of the debate, helping avoid costly errors and making political choices more explicit. The impact assessment is merely an input to the decision, not a rule that dictates a particular outcome (Posner, 2000). The impact assessment would not replace a legislative compromise on an emotional issue such as the fight against terrorism, but would provide an objective reference point against which to measure legislative proposals. For example, a national legislature may still decide to grant broad powers to the government to block websites without judicial oversight, but the impact assessment would show that the measure is not in line with best practices. The impact assessment would potentially be used in judicial review of the measure’s constitutionality.
10What’s clear, however, is that political decision makers would have little or no interest in buying in to a system that constrains their own decision making authorities. Why would they? The political process is already sufficiently complex without adding another constraint linked to a cost-benefit analysis. Governments and parliaments are elected based on their political visions and principles, which are generally incompatible with the dry calculations of a cost-benefit analysis. As mentioned by Radaelli and De Francesco (2010), politicians will all support the idea of evidence-based laws and regulations as long as this remains an abstract concept. However, when it comes to actually implementing a rigorous cost-benefit analysis system, politicians will see only a downside. The only exception to this would be where a regulatory impact assessment is mandated for independent regulatory authorities. Here, law makers will see the impact assessment as a useful constraint on an independent regulator’s powers. Law makers sometimes see the delegation of powers to independent regulatory authorities with suspicion since those authorities are not directly accountable to the government, the parliament or to citizens. It is therefore no coincidence that the rigors of market analysis and impact assessments are seen most frequently in the field of telecommunication regulation, where independent regulatory authorities have significant discretion. It is also no coincidence that the OECD sees many countries adopting the principles of better regulation, but failing to actually implement them in practice.
Criticism 2: Conflicts of interest
11The impact assessment methodology described in Chapter 6 will run into conflict of interest problems, as do all regulatory impact assessments. The drafters of the regulatory impact assessment may report to a minister or a president who has already made a political decision to implement a certain measure. The drafters will not want to undermine their boss’s proposal, and so will have a natural tendency to reverse engineer the impact assessment to fit a given outcome. It is not reasonable to expect the government to prepare an impact assessment that would go against a political decision made by the president. A peer review system can help this problem, but it will not cure it entirely.
12The OECD recommends that governments create a separate office to conduct impact assessments (OECD 2012), but even a separate office will not be immune from conflicts of interest if the president or prime minister has publicly supported a particular outcome.
13For this reason, impact assessments of the kind I describe here may work best where the legislature has delegated to an independent regulatory authority the task of finding the optimal solution to a problem such as limiting pollution or access to harmful content on the internet. The regulatory authority’s mission would be to apply a cost-benefit analysis when developing measures to protect citizens against harmful internet content. However, this implies that the government and legislature are comfortable delegating to a regulatory authority the responsibility for making these choices.
Criticism 3: Impact assessments are too expensive and complicated
14Impact assessments are complicated and costly to prepare. If there is anything that this book has shown, it is that a comprehensive cost-benefit analysis would be extraordinarily complex, involving many variables that are difficult or impossible to quantify, such as the impact of a measure on the internet ecosystem or on fundamental rights. Because many of the estimates of costs or benefits rely on subjective factors on which reasonable people can disagree, the results of the cost-benefit analysis could easily be challenged. If the cost-benefit analysis is costly and time-consuming to prepare, and the results are contestable, what good is it?
15Here, too, the European regulatory framework for electronic communications provides some guidance. The framework for electronic communications is also difficult to apply. It is based on principles such as the “three criteria test,” pursuant to which electronic communications markets should not be subject to ex ante regulation unless they show durable barriers to market entry, no signs of evolving toward effective competition, and competition law is inadequate to address market failures. The relevant markets themselves are to be defined using standard competition law tools, and dominance assessed within those markets using competition law methodology. If national regulators were left alone to apply these principles, they would have conducted costly studies, and likely reached different outcomes. The criticism: “Too complicated and too expensive” would apply there, too. To address this problem, the European Commission issued recommendations and guidelines that indicate which relevant markets pass the three-criteria test, and how dominance should be assessed on these markets. National regulators were given an initial instruction manual to guide their decisions. The instruction manual included a number of simplifying assumptions that national regulators could feel safe applying. Regulators could challenge the Commission’s simplifying assumptions if they had strong reasons to do so, but the easiest approach for most regulators was to use the simplifying assumptions contained in the European Commission’s guidelines.
16A similar approach could be used here. The European Commission (or another authority) could provide a list of benefits and costs to be considered in every impact assessment, and issue guidelines one how to measure certain costs. For example, the Commission could indicate that measures involving deep packet inspection (DPI) would normally create “high” or “extremely high” costs for privacy, because of the danger that DPI could be misused to spy on individuals. National authorities would be free to disagree with the Commission’s conclusions, but would have to explain why. The Commission’s guidelines could be developed in cooperation with the European Union Agency for Fundamental Rights.
17Consequently, one useful simplification measure for my methodology would be for a central authority – presumably the one that would conduct peer reviews – to provide initial guidance on how various measures might be scored in the cost-benefit analysis. This would provide a non-binding starting point for the drafters of the regulatory impact assessments. The guidelines may not avoid qualitative trade-offs of the kind: “Is a 300M€ per year increase in net benefit worth a ‘high’ impact on net neutrality?” But if regulators apply a consistent approach to valuation – via scoring, qualitative labels, or other techniques – the overall quality of decisions will increase because it will be possible to compare results across different subject matters and jurisdictions.
18The second lesson from the EU framework for electronic communications is that the first years of impact assessments will be difficult, but that the task then gets easier. In the system proposed in this book, the drafters of impact assessments will be conducting impact assessments on internet content measures for the first time, and there will be few examples available from other countries to rely on. Each impact assessment will be new and complex, even with simplifying assumptions issued by a central authority such as the European Commission. The peer review team will also be learning as they go. The peer reviewers will be trying to develop a consistent approach in assessing proposals, and may make mistakes along the way. However, once the initial round of impact assessments and peer reviews has been completed, things will get easier. The second round will be easier than the first, and the third easier than the second. The peer review panel will share best practices from other countries, and publish criticisms and veto decisions relating to other countries. Over time, it will become clear what regulatory measures are within the realm of acceptable, and what regulatory measures are off limits. A shared blackboard of best practices would begin to emerge. This will lead national regulators to avoid wasting time on measures that clearly have no chance of succeeding, and concentrating on measures, and variants thereof, that have worked in other countries. The development of international best practices and shared knowledge about what measures work, and what measures don’t, are key benefits flowing from the methodology.
Criticism 4: Do not try to quantify the unquantifiable
19Effects on fundamental rights cannot be reduced to a simple formula or scoring mechanism. As we have seen in Chapters 3 and 6, converting effects on fundamental rights into measurable units is fraught with difficulty. This is true, but the problem is not insoluble. A recent research project funded by the European Commission attempts to attribute a score to fundamental rights intrusions showing that a scoring mechanism is not completely crazy. The SURVEILLE project, which involves nine research institutions in Europe, resulted in the creation of a standard methodology against which police surveillance techniques can be assessed (Scheinen and Sorell, 2015). The project is similar to the approach I propose here in that it seeks to develop a rational method for comparing different regulatory measures in light of their effect on fundamental rights. The project is different because it only considers police surveillance measures. Also, the SURVEILLE project does not incorporate into the equation factors such as institutional alternatives and the effects on the internet ecosystem.
20The SURVEILLE project’s deliverables include a scoring methodology that relies heavily on the work of Robert Alexy, described in Chapter 3. The methodology used in the SURVEILLE project first involves attributing a technology assessment usability score from 1 to 10 to the relevant surveillance technology. This assessment score includes the risk of error created by the technology, the level of privacy protection already built into the technology and its cost. The technology assessment score also takes into account the efficiency of the surveillance technology in achieving the desired policy objective (which is to catch criminals and prevent crime). The project then proposes for each type of surveillance technology a fundamental rights intrusion score from 1 to 16 that assesses the degree of intrusion into fundamental rights. These two scores are then compared to provide an initial indication of whether the surveillance technology presents an acceptable benefit compared to the level of intrusion into fundamental rights. This initial assessment would then be evaluated based on a qualitative ethical valuation yielding a green light, a yellow light, or a red light depending on the level of acceptability of the measure. A green light would mean that the measure is acceptable, a yellow light means that additional work is needed in order to make the technology more acceptable for its proposed surveillance purpose, and a red light means that the form of surveillance is unacceptable and must be rejected.
Figure 6: Summary of the European Commission’s SURVEILLE project

Source: Surveille website surveille.eui.eu
21The SURVEILLE methodology has shortcomings (for example, it does not incorporate institutional alternatives), but using a standard methodology, even imperfect, is better than having no standard methodology at all. A methodology (even imperfect) provides an objective measurement tool against which proposals can be assessed. The SURVEILLE project shows that the European Commission is interested in developing evaluation tools for regulatory measures that affect fundamental rights. The project also shows the need to simplify methodologies in order to make them operational. The SURVEILLE project does a good job doing this. The methodology proposed in this book currently is too complex to be easily implemented by government agencies. Additional work is needed to simplify the methodology and make it more user-friendly.
22The SURVEILLE project does not attempt to make any connection between its methodology and the regulatory impact assessments that governments are supposed to conduct before adopting new regulatory measures. The SURVEILLE research team did not appear to consider better regulation methodology in their work, which is unfortunate. As Chapter 5 demonstrated, better regulation methodology applies to all proposed government measures, and police surveillance measures would fall within this scope.
23The SURVEILLE project also did not take into account the institutional options that may accompany police surveillance technology. A surveillance measure that is ordered by police authorities themselves would have a different impact on fundamental rights than a surveillance measure ordered by an independent court. The duration of the surveillance measure, and mechanisms to ensure that the measure is lifted as soon as it is no longer necessary, are also relevant in assessing the level of impact on fundamental rights. Also, the SURVEILLE methodology does not call for a peer review mechanism to ensure that the methodology is applied correctly, and that the effects of conflicts of interest are minimized.
Quantifying the unquantifiable
24Applying a regulatory impact assessment, including a cost-benefit analysis, will lead to an improvement in how regulations on internet content are adopted and applied. However, further work is needed in coming to grips with “quantifying the unquantifiable.” A cost-benefit analysis works best when all costs and benefits can be transformed into monetary equivalents. In the United States, drafters of cost-benefit analyses have developed ways to transform risks to human lives into monetary equivalents, as well as risks (and benefits) to the environment. If the benefit or cost associated with modifying the risk of death can be quantified, there would seem to be no reason why changes in the enjoyment of other fundamental rights cannot also be quantified. To date, research has focused on the monetary value of various privacy rights (Acquisti 2010, Thierer 2013). However, I am not aware of similar research focusing on the monetary value of freedom to access information.
25One approach that has not yet been used to my knowledge would be to take the statistical values of human lives developed in the course of impact assessments in the United States, and attempt to extrapolate how an interference with a fundamental right such as privacy would compare with the statistical value of a human life. Viscusi and Aldy (2003) estimate the value of a statistical life in the United States at $7 million. The European Commission recommends a value of €1 million (OECD, 2009). It may be possible for researchers to approach this question by looking first at fundamental rights that come close to the value of life itself, such as the right not to be tortured, or the right not to be imprisoned. Four years in prison might have a value roughly equivalent to four years of loss of life. Torture might have a value close to that of the loss of an entire life (or perhaps more?). Researchers would then try to work backwards to address other fundamental rights such as freedom of expression and privacy, each of which would presumably have a lower relative value. At the end, it may be possible to convert losses of certain fundamental rights into a number of life years lost, which in turn would allow us to have a rough monetary value for various fundamental right restrictions.
26The monetized value of fundamental rights would be used solely in the cost-benefit analysis to help identify regulatory solutions that are optimal in an economic sense. The outcome of the cost-benefit analysis would still be subject to constraints such as those described in Chapter 6. For example, even if a cost-benefit analysis were to show a given proposal as optimal, the relevant proposal could be rejected if its effect on privacy or freedom of expression was “extremely high.” These constraints would ensure that cost-benefit analyses remain subject to an ethical filter (Nussbaum, 2000). Likewise, the monetized value of fundamental rights would never replace the role of courts, whose job it is to ensure that measures adopted by regulators do not unduly restrict fundamental rights. Courts would naturally not be bound by the statistical value of a fundamental right used in the cost-benefit analysis. The sole utility of the monetized value would be to help analyze regulatory options in a uniform manner.
27Another direction for research would be to draw parallels with the evaluation of environmental costs and benefits. Environmental impact assessments have been conducted for decades, and considerable effort has gone into quantifying environmental costs and benefits. If we consider the internet ecosystem as similar to an environmental ecosystem, the study of environmental impact assessments might help determine how indirect impacts on the internet ecosystem or other hard-to-quantify costs should be measured. Scholars have examined how difficult-to-quantify factors such as the beauty of landscape might be measured in the context of environmental impact assessments (DEFRA, 2007). Some scholars have proposed mathematical tools, including “fuzzy logic,” to help better take account of uncertainty and hard-to-quantify impacts (Shepard, 2005). Some of these methods may be transposable to the evaluation of harms to fundamental rights and/or the internet ecosystem.
Notes de bas de page
1 Executive Order 12866.
2 The White House, Office of Information and Regulatory Affairs (OIRA), Q&As, https://www.whitehouse.gov/omb/OIRA_QsandAs, accessed February 28, 2016.
Le texte seul est utilisable sous licence Licence OpenEdition Books. Les autres éléments (illustrations, fichiers annexes importés) sont « Tous droits réservés », sauf mention contraire.
Smart(er) Internet Regulation Through Cost-Benefit Analysis
Ce livre est diffusé en accès ouvert freemium. L’accès à la lecture en ligne est disponible. L’accès aux versions PDF et ePub est réservé aux bibliothèques l’ayant acquis. Vous pouvez vous connecter à votre bibliothèque à l’adresse suivante : https://freemium.openedition.org/oebooks
Si vous avez des questions, vous pouvez nous écrire à access[at]openedition.org
Smart(er) Internet Regulation Through Cost-Benefit Analysis
Vérifiez si votre bibliothèque a déjà acquis ce livre : authentifiez-vous à OpenEdition Freemium for Books.
Vous pouvez suggérer à votre bibliothèque d’acquérir un ou plusieurs livres publiés sur OpenEdition Books. N’hésitez pas à lui indiquer nos coordonnées : access[at]openedition.org
Vous pouvez également nous indiquer, à l’aide du formulaire suivant, les coordonnées de votre bibliothèque afin que nous la contactions pour lui suggérer l’achat de ce livre. Les champs suivis de (*) sont obligatoires.
Veuillez, s’il vous plaît, remplir tous les champs.
La syntaxe de l’email est incorrecte.
Référence numérique du chapitre
Format
Référence numérique du livre
Format
1 / 3