A methodology for assessing regulatory options
Texte intégral
Bringing it all together
1I will attempt now to bring together the lessons learned from each of the previous chapters in order to propose a methodology for assessing regulatory options to advance content policies on the internet. As discussed in Chapter 2, the range of content policies is diverse: They may target child pornography, racism, anti-Semitism, promotion of terrorism, copyright infringement, illegal gambling, the right to be forgotten or even the promotion of national culture or language. The range of internet intermediaries, and the possible actions they can take, is also varied. Internet access providers, search engines, payment providers, advertising intermediaries, and social media platforms could potentially be called on to help implement a content policy, and the range of actions they could take is potentially vast.
2Chapter 3 explored impacts on fundamental rights. The proportionality test requires a listing of all the fundamental rights potentially affected by each regulatory measure, and an attempt to identify the relative impact of each of the regulatory options on each of those rights. Proportionality then involves evaluating the relative benefit of the proposed measures in terms of promoting certain fundamental rights such as privacy or the right to property, and then choosing the regulatory alternative that gets the job done while causing the least adverse impact on other fundamental rights. In most cases 100% enforcement of a content policy on the internet will not be possible or even desirable, because the cost of approaching 100% enforcement would be disproportionally high in terms of harming other fundamental rights.
3Chapter 4 presented the range of possible institutional and legal frameworks. They include self-regulation, co-regulation, liability and property rules, and full-fledged administrative regulation with a dedicated regulatory authority set up to oversee enforcement. The choice of institutional framework will affect how flexible the regulatory regime is, and how susceptible the regime is to industry capture or regulatory creep.
4Chapter 5 examined the principles of better regulation in the United States and Europe. I identified aspects of better regulation that are particularly important for the internet, focusing on the 2011 OECD recommendations for internet policymaking. Chapter 5 also presented some of the limitations of cost-benefit analyses, and in particular their incompatibility with the realities of the political process.
5In this chapter, I will try to weave these strands into a single methodology that can be used by policymakers to help evaluate regulatory options and fulfill their duties under better regulation principles.
A regulatory impact assessment that incorporates a cost-benefit analysis
6When using the term “regulatory impact assessment,” I mean the entire methodology proposed in this chapter, which includes several steps, including a questionnaire, a cost-benefit analysis, a public consultation and peer review. The cost-benefit analysis is therefore one part of the overall regulatory impact assessment. The two terms, “regulatory impact assessment” and “cost-benefit analysis,” are not synonymous.
Who would use the regulatory impact assessment?
7The regulatory impact assessment would be applied in two possible institutional contexts. In the first, a national government would conduct the assessment before submitting a legislative proposal to parliament. This is currently how impact assessments are conducted in France. The impact assessment is prepared by the government before legislative proposals are presented to parliament. But the impact assessments currently conducted by the French government fall short of the analysis called for in this book. Among other things, the impact assessments do not attempt to measure indirect costs.
8The second institutional context would involve a regulatory authority, who would conduct the impact assessment before adopting a regulatory solution for limiting access to harmful content. This is currently what happens when regulatory authorities in Europe propose measures affecting the electronic communications market. Regulatory authorities must conduct a market analysis and justify the proposed measure based on a methodology imposed by the European Commission.
9Regardless of whether the regulatory impact assessment is conducted by the government or by a regulatory authority, the impact assessment would have to be reviewed by an independent institution in order to ensure that the impact assessment is conducted rigorously. This is the peer review procedure I describe below.
Why would the regulatory impact assessment increase regulatory quality?
10As explained in Chapter 1, the regulatory impact assessment will force the sponsors of regulation to define better the objective they hope to achieve and ways to measure success. The impact assessment will require a better analysis of alternative scenarios, including the alternative of doing nothing. The cost-benefit analysis will require consideration of indirect costs, which are generally ignored today. The process will also foster regulatory improvement by allowing comparisons and learning between different countries and regulatory approaches. Finally, the regulatory impact assessment will permit a clear demarcation between regulatory measures adopted by democratic countries that uphold free expression and net neutrality, and similar measures adopted by less democratic countries.
Elements of the methodology
11To analyze proposed measures limiting access to content on the internet, I propose a five step methodology:
12Step 1: A questionnaire requiring policymakers to identify the relevant variables in the policy equation, i.e.:
- the underlying content policy that needs to be enforced, its relative importance compared to other content policies and fundamental rights, and how “success” in applying the content policy can be measured;
- the range of internet intermediaries and actions that could help enforce the content policy, ranging from the least intrusive to the most intrusive;
- international practices, if any, used to address the problem;
- the institutional alternatives that can be considered, including liability and property laws, self-regulation, co-regulation, and/or administrative regulation;
- the fundamental rights affected by each proposed measure, and how each proposed measure ranks in a proportionality test;
- other indirect costs such as harm to the internet ecosystem.
13Step 2: After completing the questionnaire, policymakers should apply a cost-benefit analysis to the proposals, comparing the alternatives to a baseline scenario. The cost-benefit analysis requires policymakers to:
- build the baseline scenario (or “counterfactual”) consisting of no regulatory intervention, taking into account dynamic factors such as possible evolutions of internet technology and markets, anticipated enforcement of existing laws and self-regulatory policies;
- evaluate the level of benefits flowing from each regulatory proposal, compared to the baseline;
- evaluate the level of direct costs resulting from each proposal compared to the baseline, including direct costs for internet intermediaries, their customers, and taxpayers;
- evaluate the level of indirect costs resulting from each proposal compared to the baseline, including impacts on fundamental rights, on the internet ecosystem and innovation;
- select the one or two proposals that are likely to yield the highest net benefits;
- confront the proposals with the list of constraints, and eliminate any alternatives that do not satisfy the constraints. The constraints would permit certain ethical considerations to be included in the cost-benefit analysis.
14Step 3: Conduct a public consultation based on the initial questionnaire and cost-benefit analysis.
15Step 4: Submit the questionnaire and cost-benefit analysis to institutional peer review, permitting an independent authority to verify that the proposed measure delivers the highest net benefit and takes into account the constraints.
16Step 5: Organize a periodic ex post review mechanism to ensure that regulatory measures are modified or removed as soon as they are no longer necessary.
17Each of these five steps is presented below.
The questionnaire
18The first step in the impact assessment is a qualitative questionnaire. The questionnaire will force policymakers to identify precisely the objective they hope to achieve, and to list all the alternative mechanisms – technical and institutional – available to achieve the objective. The questionnaire will also force policymakers to identify, from a qualitative standpoint, costs associated with each measure, and provide a first guess at their magnitude. The questionnaire will also require policymakers to identify international best practices.
19The questionnaire is designed to put all the variables on the table, before narrowing down the choice of alternatives and applying a full cost-benefit analysis to those alternatives.
20The questionnaire would look something like this:
21Analysis of the underlying content policy that needs to be enforced.
22Question: Please describe the kind of content that is targeted by the proposed measure:
| Content category | Describe the specific content targeted by the proposed policy |
| Malware or other threats to the network or to user data | |
| Spam, phishing | |
| Cookies, tracking software | |
| Right to be forgotten content | |
| Online gambling | |
| Cigarettes, alcohol | |
| Illegal drugs, counterfeit drugs | |
| Other counterfeit articles | |
| Copyright infringement | |
| Defamation, protection of privacy | |
| Racial, religious hatred | |
| Protection of culture, language and cinema | |
| Advertising laws | |
| Protecting minors against violent or pornographic content | |
| Child pornography | |
| Terrorism | |
| Other (please describe) |
23Qualifying the intensity of the harm.
24Questions: Describe the provisions (if any) of the criminal code that apply to this content, including applicable sanctions, if any. Is the content illegal in all circumstances, or only when accessed by certain classes of persons (e.g. minors), or using certain search terms (e.g. the right to be forgotten)?
25Does the content in question create a risk of physical harm for citizens (e.g. sexual exploitation of children, slavery, and terrorism)? If so, please describe the kind of harm, and its extent (e.g. number of cases per year). Where access to the content does not create a risk of physical harm, describe the nature of the harm: harm to a fundamental right, to mental development of minors, to culture? Please describe the extent of the harm: how many persons affected and the range of intensity of the harm for each individual.
26The prevention of these harms, as compared to the baseline scenario, will constitute the benefits flowing from each alternative proposal.
27How to measure the benefits of the content policy.
28Questions: How would the benefits associated with the content policy be measured? If the content policy is successfully applied, how would “success” be measured? Are there proxies that can be used to measure success?
29Developing measurement tools to evaluate success of a content policy is one of the most challenging aspects of the impact assessment. I examine this problem in more detail below.
30International acceptance of the content policy.
31Questions: Is the content in question illegal or restricted everywhere in the European Union? In most countries in the world? Are there international treaties dealing with this form of harmful content?
The range of internet intermediaries and actions to help enforce the content policy
32Questions: List the internet intermediaries that could assist in enforcing the relevant content policy, and the action they could take. In the third column, rate the level of intrusiveness of the relevant measure, i.e. how much would the measure interfere with fundamental rights or the proper functioning of the internet?
| Category of internet intermediaries | Describe internet intermediary (ies) that are potential candidates to enforce the content policy, and the potential actions they could take | Level of intrusiveness of the measure (low, medium or high) |
| Search engines | ||
| Hosting providers | ||
| Social media platforms | ||
| Internet access providers | ||
| Domain name registrars | ||
| Payment service providers | ||
| Advertising service providers | ||
| Application stores | ||
| Content delivery networks | ||
| Internet backbone providers | ||
| End-user applications, browsers, antivirus software, parental control software | ||
| Set-top box or modems | ||
| Device operating systems | ||
| Other (please describe) |
Questions on technical options:
33Least intrusive measures: Among the internet intermediaries on your list, which ones could apply the most “light touch” measures, i.e. the measures that would have little or no impact on the functioning of the internet or on fundamental rights. An example of a “light touch” measure is one that educates internet users on how to protect themselves against the relevant content, by adjusting browser settings, for example.
34Most intrusive measures: Among your list of internet intermediaries and the potential actions they could take, which actions are the most intrusive, i.e. those that would have the greatest effect on the normal functioning of the internet and/or on fundamental rights? Examples of intrusive measures include blocking internet content based on deep packet inspection technology and/or requiring that all internet traffic flow through a central gateway that can detect illegal content. Another example of intrusive measures would be those that have extraterritorial effects.
35Intermediate measures: List actions that could be taken by the internet intermediaries on your list that are in a middle range, i.e. neither the most intrusive nor the least intrusive.
36Territorial enforcement: Among the internet intermediaries on your list, which ones are subject to the jurisdiction of courts and regulatory authorities in your country? If some of the intermediaries potentially considered for regulation are subject to the jurisdiction of your country and other intermediaries of the same category are not, how effective would the proposed measure be? Would the proposed measure create a competitive distortion between intermediaries within your jurisdiction who apply the measure, and intermediaries outside your jurisdiction who do not?
Remedies used in other countries
37Questions: How do other countries deal with this kind of content? What are the advantages and disadvantages of the approaches used in other countries? Are there any international instruments (treaties, recommendations, charters) addressing remedies for this content issue?
38Would any of the measures envisaged have extraterritorial effects? If so, are the measures consistent with the laws of the foreign countries affected? What is the likely response of other countries if the proposed measure has effects in those countries? Katz (2000) refers to these issues as “jurisdictional externalities.”
39How easy is it for the relevant internet intermediaries to limit the effect of the measure to your country? Are the proposed approaches consistent with the OECD recommendations on internet policymaking, and if not, which aspects diverge from the OECD recommendations?
The institutional alternatives, including liability and property rules, self-regulation, co-regulation, and/or full-fledged administrative regulation
Questions on institutional alternatives:
40Criminal law enforcement: How do prosecutors and police currently deal with the problem? What is the current volume of criminal prosecutions and convictions relating to this content policy? How effective is cross-border police cooperation in this field? Will the development of more effective cross-border police cooperation make a difference?
41Administrative regulation: Are there any regulatory agencies or independent regulatory authorities that currently investigate and enforce the relevant content policy? What measures do these regulatory authorities currently take to enforce the content policy?
42Liability and property laws: What role do liability and property laws, and their enforcement before civil courts, play, if any? What existing laws, civil and criminal, are used to address the problem? Are there ways in which the enforcement of existing legal provisions can be improved (OECD, 2009)?
43Unilateral self-regulation: Do internet intermediaries attempt to regulate this problem through their own terms of use and internal procedures? Do the relevant internet intermediaries apply a notice and takedown procedure, and if so how effective is it? What problems, if any, have been reported in connection with these internal procedures? How might they be improved?
44Multilateral self-regulation: Are there any existing codes of conduct or other forms of multilateral self-regulation that address the problem? How effective are those measures?
45Administrative regulation: If a regulatory authority were to become involved in enforcing the relevant content policy, what exactly would the regulatory authority do? How would the role of a regulatory authority differ from that of a judge?
46Taxes: Is the matter one that could be treated through public subsidies and/or tax incentives (e.g. subsidies to promote culture)?
47Territorial level of regulation: What is the best territorial level of regulation: individual country? European Union? International agreement? OECD recommendation?
The fundamental rights affected by each proposed measure
Questions on fundamental rights:
48Describe the impact, positive or negative, that potential actions by internet intermediaries could have on fundamental rights, including the following:
| Category of fundamental right | Describe the persons affected, nature of the effect, and extent (number of persons affected) |
| Freedom of expression, including freedom to access information | |
| Protection of personal data and privacy | |
| Protection of property, including intellectual property | |
| Freedom to conduct a business | |
| Protection of security and/or health | |
| Right to a fair trial, presumption of innocence | |
| Other |
Internet ecosystem
Questions on the internet ecosystem:
49List the other potential harms generated by each proposal, such as negative impacts on:
- Net neutrality and the end-to-end architecture of the internet;
- Innovation;
- Competition.
50For each harm caused by a measure, describe its intensity (low, medium, high, extremely high).
51For each measure, describe whether it respects the principle of technology neutrality, and if not, describe the technological approach that is taken, and the potential risks associated with the absence of neutrality (e.g. technological obsolescence or harm to innovation).
Behavorial economics and “nudges”
Questions on behavioral aspects
52List changes in default settings or user interfaces that might help achieve the desired policy objective.
53List possible actions that could be taken to change user behavior without recourse to internet intermediaries, e.g. advertising campaigns, educational programs.
54Describe possible changes in user behavior that may result from regulatory action and that would be counter-productive, e.g. increased use of VPNs and encryption, use of intermediaries outside the country’s jurisdiction.
55Can taxation be used to influence behavior?
Adaptive and experimental regulation
Questions on adaptive regulation
56For each proposal, the questionnaire should ask whether the effectiveness of the approach has been tested through experimentation. If no experimentation has been conducted, the sponsor of the proposal should explain how effectiveness can be measured through ex post reviews, and whether the approach can be modified or removed as a function of its observed effectiveness. The questionnaire should ask whether the suggested approach is “highly flexible,” “flexible,” or “inflexible.”
A cost-benefit analysis under constraint
57The questionnaire will permit policymakers to eliminate a number of alternatives, either because they are too intrusive, or because the relevant intermediaries are beyond the territorial reach of the regulation. Ideally, the questionnaire will help policymakers narrow down the options to a manageable number. The questionnaire will also help identify the variables that need to be captured in the cost-benefit analysis.
58The next step is the cost-benefit analysis itself. The analysis will require:
- preparation of a baseline scenario or counterfactual;
- an estimate of the net benefits (benefits less costs) generated by each regulatory alternative;
- the application of constraints, which may lead to the elimination of certain proposals.
How to deal with hard-to-quantify benefits and costs?
59As we saw in Chapter 5, transforming certain benefits and costs into measurable units is challenging. Impact assessments dealing with environmental protection or safety measures must deal with this problem on the benefits side. For example, measures designed to protect the environment must deal with questions such as how to measure the social value of a pristine meadow, or a certain kind of wildflower. Health measures must deal with the question of how to measure the value of a human life. As we saw in Chapter 5, Ackerman and Heinzerling (2002) challenge the very idea of “pricing the priceless.” Nevertheless, most cost-benefit analyses involving environmental, health or safety regulations at least attempt to attribute values to the benefits flowing from the regulation. (I emphasize again that the measurement of these values is not intended to suggest that they can be bought and sold, but rather to help choose among alternatives when government resources are limited.)
60Measures designed to limit access to harmful content are even more complicated than these environmental and health examples. Blocking content on the internet is generally aimed at protecting values, such as human dignity, that are hard to quantify. In this respect the exercise is just like an environmental or health regulation. Instead of protecting wildflowers or human life, the policy protects privacy or the right to dignity. However, unlike most environmental or health measures, measures to block internet content will create costs that are equally hard to quantify: limitation to freedom to access information, limitation to privacy rights, harms to the internet ecosystem. Thus we have a problem where hard-to-quantify elements exist both on the benefits and cost sides.
61Before examining the various steps in the cost-benefit analysis, this section describes ways to quantify, and if possible transform into monetary units, these benefits and costs.
Contingent valuation or stated preferences
62One technique for valuing hard-to-measure elements is called contingent valuation (also referred to as the “stated preferences” method) (Viscusi et al., 2005). Under the contingent valuation approach, researchers develop survey questions that attempt to determine individuals’ willingness to pay for certain hard-to-quantify benefits, such as reducing the risk of an oil spill that would spoil the Alaska coastline. The survey questions must be drafted so as to present individuals with concrete choices in hopes of eliciting responses that objectively measure how much individuals would be willing to pay – for example in additional annual taxes – in order to reduce the risk of an oil spill similar to the Exxon Valdez catastrophe. This approach is the same as the “stated preferences” approach described in Chapter 5 in connection with the United States government Circular A4.
Cost effectiveness analysis
63The second technique consists of not trying to convert the benefit into dollar or euro amounts at all, but into some other unit of measurement. For example, a regulation designed to increase the safety of swimming pools might be measured by the number of infant deaths avoided. A regulation designed to protect the environment might be measured by number of hectares of wetlands preserved from development. The unit of measurement (infant deaths avoided, hectares of wetlands) would then be used to compare different regulatory options in order to compare the cost-effectiveness of each option. For example, if one regulatory measure permits an increase in swimming pool safety at a cost of 1,000 euros per avoided infant death, and another proposal increases safety at a cost of 1,000,000 euros per avoided infant death, the first lower-cost proposal would be implemented in priority, all other things being equal. Viscusi (2005) calls this a cost-effectiveness sensitivity analysis. Circular A4 calls this a CEA, a cost effectiveness analysis.
64As we will see below, this technique for measuring benefits can be used for content policies. The persons preparing the impact assessment would have to define units by which to measure the success of different regulatory proposals. For example, a policy designed to protect the so-called “right to be forgotten” might be evaluated based on the time it takes an average person to see the information that is supposed to be hidden on the search engine. A policy designed to fight online copyright infringement might be evaluated based on the number of persons who subscribe to legal streaming services. One or more proxies are used to measure the benefit flowing from a given content policy, and these proxies are not converted into money.
Benefit transfer
65A third approach is to try to convert the benefits or harms into monetary units by comparing them to other benefits or harms for which monetary valuations are available. This is the “benefit transfer” method described in United States government Circular A4 (see, Chapter 5). In the field of fundamental rights, this might be done by comparing the harm to fundamental rights to the statistical value of a human life. Economists have developed monetary values associated with saving a human life, or extending a human life by a given number of years. Restricting a fundamental right such as freedom of expression or privacy diminishes the quality of human life, but in an amount that is presumably less than losing life entirely. Starting from this assumption, it may be possible to develop approaches that attribute values to different levels of intrusions into fundamental rights. For example, a severe restriction of freedom of expression might be deemed the equivalent, in monetary terms, to the loss of one, or two, or ten, years of life. The right not to be tortured may have a value equivalent to, or even greater than, the statistical value of a human life. Saving a child from sexual abuse may also have a value equal to, or greater than, the statistical value of a human life. Preserving an individual’s privacy against government intrusion would have a value less than the statistical value of a human life, although how much less would depend on the level of the intrusion. As an extreme example, if individuals lived in an Orwellian world where they had no fundamental rights at all, some might say that the quality of life has been impaired in an amount that approaches the statistical value of a human life. In other words, a total deprivation of the fundamental human rights might be the equivalent of death. Based on that assumption, it may be possible to work backwards and develop values for different incremental intrusions of fundamental rights. By doing this, it might be possible to convert harms to fundamental rights into dollar or euro figures. I mention this as a possible field of future research in Chapter 7.
66Many would cringe at attributing a monetary value to a fundamental right, arguing that rights cannot be reduced to money. However, the same objections would apply to human life. Yet regulatory impact assessments, at least in the United States, have nevertheless developed a statistical value for a year of human life saved. The sole utility of the valuation is to compare the efficiency of different regulatory alternatives.
Hedonic pricing
67A fourth method used in environmental impact assessments is hedonic pricing. Under this approach, one takes prices of a good for which there exists a market, e.g. the price of a house, and compares the prices in low-pollution areas and high-pollution areas. The difference in price represents, at least in part, the value of living in a pollution-free environment. Similarly, in health and safety regulation, the higher salary given to persons who do dangerous jobs may reflect the market value of the increased risk to health.
68I have not found a way to apply hedonic pricing to benefits and costs relating to blocking internet content. But there may exist approaches I have not thought of.
Qualitative scoring
69The fifth method is to develop a qualitative scoring system. In the field of fundamental rights, Robert Alexy has proposed a scoring system that permits different outcomes to be compared. For example, a measure that yields a fundamental rights score of +4 would be better than a measure that yields a fundamental rights score of -2. Scoring can work when comparing elements with similar unitary values. But when comparing elements whose unitary values differ, scoring can run into an “apples and oranges” comparison problem. For example, courts generally agree that privacy and freedom of expression have equal normative value. A “unit” of freedom of expression is roughly equal to a “unit” of privacy. In that case, it would be theoretically possible to balance units of freedom of expression against units of privacy. A regulatory measure that promotes privacy by +4 but harms freedom of expression by -1 compared to the baseline scenario would yield a net fundamental rights improvement of +3.
70This kind of trade-off is possible, but has limits. First, a fundamental right can never be reduced to zero, even if there is a strong countervailing benefit for another fundamental right. The essence of the right must always be preserved. This is why Alexy’s formula incorporates an exponential progression, so that “severe” restrictions of a fundamental right have an exponentially higher value than “medium” or “high” restrictions. This is also why I propose a step in the cost-benefit analysis that eliminates any proposal that causes a “severe” or “extremely high” negative impact on a fundamental right.
71Second, fundamental rights cannot generally be traded against other values on a one-to-one basis. One unit of harm to privacy would not have the same value as one unit of harm to innovation. Except for freedom of expression and privacy, where there is a rough correspondence in unit values, fundamental rights cannot be traded off against each other on a one-to-one basis.
Qualitative labels
72A last approach consists of attributing qualitative labels for values that are hard to quantify, without trying to add them up through a scoring system. For example, some of the costs associated with a regulatory proposal may be characterized by qualitative labels such as “severe intrusion,” “medium intrusion,” “light intrusion,” “no intrusion.” This technique may permit certain proposals to be eliminated without having to convert the relevant costs into fungible units. For example, in the three proposals below, proposals A and C can be eliminated based solely on a comparison of the qualitative labels:
| Proposal A | Proposal B | Proposal C | |
| Quantifiable benefits | +200 | +250 | +250 |
| Quantifiable costs: | -100 | -100 | -100 |
| Net quantifiable benefits: | +100 | +150 | +150 |
| Non-quantifiable costs: | |||
| Harm to privacy | Light intrusion | Light intrusion | Severe intrusion |
| Harm to freedom of expression | No intrusion | No intrusion | No intrusion |
| Harm to innovation | Light intrusion | Light intrusion | Light intrusion |
73In this example, proposal B emerges as the preferred outcome. Proposals A and C necessarily present lower net benefits than proposal B. It is not necessary to add up the non-quantifiable costs to privacy, freedom of expression and innovate to reach this conclusion.
Conclusion
74In summary, the six methods for measuring hard-to-quantify benefits or costs are:
- Contingent valuation or stated preferences (i.e. using questionnaires);
- Using non-monetary units to measure the level of the benefit or cost (e.g. number of lives saved);
- Benefit transfer method (e.g. finding a correlation between fundamental rights and the statistical value of a human life);
- Hedonic pricing (finding differences in the pricing in some good that reflect the different elements we’re trying to measure);
- Fundamental rights scoring (used by Robert Alexy to compare measures impacting fundamental rights);
- Qualitative labels (i.e. using labels like “extremely high” to denote the level of cost).
Prepare a baseline scenario of no regulatory intervention
75We can now turn to the steps involved in the cost-benefit analysis. The first step is to create a baseline scenario, or counterfactual. As noted in Chapter 5, preparation of a baseline scenario is one of the most important elements of a regulatory impact assessment, and also one of the most widely misunderstood. The baseline scenario is not the status quo, but an attempt to project how the status quo will evolve in the absence of regulatory intervention.
Anticipate technological and market changes
76The baseline scenario must try to anticipate technological or market developments that may intervene in the following years, as well as increased use of existing laws and self-regulatory initiatives. For example, a baseline scenario involving a proposed regulatory framework to deal with online copyright infringement should take account of:
- future use of civil lawsuits, including notice and takedown, and requests for injunctive relief against internet intermediaries;
- future market changes in how music and films are consumed online;
- cross-border criminal actions;
- any existing self-regulatory regimes and their likely development over time.
77In the fast-moving internet ecosystem, market or technological developments can make regulation obsolete. It is important that the baseline scenario attempt to characterize these developments and their dynamic effect on the problem. The baseline scenario should attempt to anticipate whether the identified problem is likely to get better or worse over time based on different assumptions, such as increased resources devoted to law enforcement, or increased adherence to voluntary codes of conduct. A sensitivity analysis should be done to measure the effect of changes of assumptions on the outcome.
Defining “success”
78If possible, the baseline scenario should identify the level of intensity of the problem at different time periods, for example 12, 24 and 36 months, should be quantified. Measuring the intensity of the problem will depend on the nature of the problem and the definition of “success” in achieving the relevant content policy.
79Defining how success is measured is one of the most challenging aspects of the exercise. I discuss below ways of breaking down benefits into different categories, analyzing each category of benefits in terms of their importance to society, and measuring how the level of benefits varies over time. It may be necessary to measure benefits through proxies. For example, success in fighting online copyright infringement might be measured through various proxies, such as:
- an increase in the number of prosecutions or fines for copyright infringement;
- an increase in revenues for legal online music and film distribution platforms;
- a decrease in detected illegal downloads;
- an increase in box office or live concert sales.
80Success in fighting online child pornography might be measured through:
- an increase in the number of arrests and confiscations of images;
- a decrease in the number of visits by internet users to prohibited sites (if that can be measured);
- the amount of time and effort required for an average person to obtain access to a prohibited site (if that can be measured);
- the ease with which one can locate a child pornography site using various search engines.
81Success in promoting French film production might be measured through:
- an increase in the number of French films produced annually;
- an increase in aggregate production budgets;
- an increase in the number of French films viewed on digital distribution platforms;
- an increase in the number of French films viewed in cinemas;
- an increase in the number of French films receiving international awards.
82Whatever the content policy is, a crucial step is to define a way to measure its achievement. And the first place this comes into play in the cost-benefit analysis is in the baseline scenario, i.e. what is likely to happen in the absence of regulatory intervention? The exercise will reveal the sometimes tenuous link between the objective being pursued and the corrective measure being proposed.
83The problem of measuring success also raises the important question of whether a regulatory action creating costs that we know exist, such as an impact on fundamental rights and direct costs on internet intermediaries, should be implemented when there is no way to know for sure if the measure is effective in achieving the relevant content policy.
84To provide an example, some policymakers in France are considering measures that would obligate certain internet intermediaries to provide a preference for French content in recommendation algorithms. The objective is to promote French culture, language and content production. But how exactly would the impact on French culture, language and content production be measured to determine whether the regulatory measure is effective in achieving the desired objective? The baseline scenario will require the drafters of the cost-benefit analysis to attempt to project what the relevant success parameter would look like in the absence of regulatory intervention. This baseline result would then be compared with the result flowing from various regulatory alternatives. Would a policy be considered successful if every recommendation contains at least one French-produced picture? Does it matter in this context if users actually click on the recommendation that is proposed? Does it matter what the search terms were?
Measuring benefits compared to the baseline scenario
85After building the baseline scenario, the drafters of the cost-benefit analysis must examine each of the regulatory alternatives and attempt to measure the benefits that those alternatives generate compared to the baseline.
Choice of non-monetary units to measure benefits
86If the drafters of the cost-benefit analysis choose to measure benefits using an alternative non-monetary unit such as number of lives saved, or number of French films produced, the same metrics should be used when measuring the likely achievement of the content policy in the various alternative scenarios. The nature of the success metrics will naturally affect the intensity of the total benefits flowing from the measure. A success metric based on projected increases in revenues for online music platforms would have a different absolute value in terms of benefits for society than a success metric based on number of lives saved from decreased sales of counterfeit drugs. Recognizing the difference in the absolute value of the success metrics is important to be able to compare the overall benefits to the overall costs, particularly in situations where benefits will require trade-offs against fundamental rights. To illustrate the point, 1000 saved lives will justify a higher level of costs, including interferences with fundamental rights, than 1000 saved record sales. This is why using monetary units to measure benefits is always better, whenever possible.
Breaking benefits into different categories
87Certain content policies will yield multiple related benefits. To measure benefits, the drafters of the cost-benefit analysis should unpack the different categories of benefits and measure them separately. For example, for measures to block access to images of child pornography, the set of benefits may include:
- a reduction in the number of children subject to sexual abuse;
- a reduction in the number of children or young adults who are psychologically harmed by being exposed to child-abuse images on the internet;
- increased used of the internet flowing from the fact that the internet is a safer place; and
- the symbolic value of showing that society does not tolerate serious violations of human rights on the internet.
88In this example, there are four separate benefits with different qualitative characteristics. Some of the benefits may correspond to saved lives, some of the benefits may correspond to increased usage of internet, and some of the benefits may relate to the affirmation of important societal values.
89The drafters of the cost-benefit analysis should determine which of the categories of benefits are quantifiable, and which are not. Saving a child’s life may have a quantifiable benefit, if the number of saved children can be estimated. The saved lives may also be translated into monetary values, using the statistical values of human lives published in connection with United States health and safety regulations.
90Affirming the symbolic value of not tolerating human rights violations on the internet may be impossible to quantify in monetary terms, whereas the benefit flowing from increased usage of the internet might be quantifiable, albeit hard to measure in practice.
91The purpose of this exercise is to highlight the likely impact of the regulatory measure on each category of benefit. For a measure designed to block access to child pornography, the different categories of benefits might look like this:
| Category of benefit | Quantifiable? Value of the benefit to society |
| Reduction in number of molested children | Quantifiable: estimate number of saved children compared to baseline. Monetary units: perhaps possible to convert to monetary units using US statistical values. Value of each unit of benefit (life of a child): extremely high. |
| Reduction in young or vulnerable individuals exposed to shocking images | Quantifiable: estimate reduction in number of persons exposed to images compared to baseline. Monetary units: difficult to convert to monetary values.Value of each unit of benefit: high. |
| Increased use of internet due to making the internet safer | Quantifiable and possible to convert to monetary values. |
| Affirming society’s refusal to tolerate serious human rights violations on the internet | Non-quantifiable: high symbolic value. |
92Where benefits cannot be translated into monetary values, they should be given qualitative labels or scores, using one of the methods examined above.
The causal link between the regulatory measure and the benefit
93Once the benefits have been broken down into categories, the drafters of the cost-benefit analysis should then determine the strength of the causal link between the proposed regulatory measure and the achievement of the relevant category of benefit. For example, the linkage between the blocking of access to child porn images and the reduction in the number of abused children may be difficult to establish. The link would be indirect at best. The most direct way to reduce the number of abused children is to arrest child molesters. Fewer criminals would result in fewer victims. A regulatory measure limiting access to child porn images could potentially reduce demand for child porn images and therefore potentially lead to a reduction in supply, and therefore a reduction in the number of sexually abused children. But this is not the only possible outcome. An enforcement measure that achieves 100% blocking might make the task of finding and arresting child molesters more difficult for law enforcement authorities because the criminals would use more hidden ways to exchange images. This could paradoxically increase the number of victims. Thus the causal link between the hypothetical “perfect” blocking measure and the category of benefits related to saved children is likely to be indirect, and may (depending on the views of law enforcement authorities) even be counter-productive.
94The link between the enforcement measure and the creation of a safer internet may be more direct. There, too, however, it would still be necessary to show that the safer internet in fact increases internet usage.
95The most direct causal link would relate to achieving the symbolic benefit of showing zero tolerance for serious violations of human right on the internet. A new regulatory measure would in most cases have a direct causal link to achieving this benefit.
| Category of benefit | Value of the benefit to society | Causation link with the blocking measure |
| Reduction in number of molested children | Quantifiable | Weak |
| Reduction in young or vulnerable individuals exposed to shocking images | Quantifiable | Strong |
| Increased use of internet due to making the internet safer | Quantifiable | Medium |
| Affirming society’s refusal to tolerate serious human rights violations on the internet | Non-quantifiable | Strong |
Practical measurement of success
96As a last step, the drafters of the cost-benefit analysis should try to determine the practical measurability of “success” for each category of benefits. What would one count in order to determine if the benefit has increased, decreased or remained the same, and is the counting practical? In our example, the number of sexually abused children in a year is probably easy for regulators to count based on police records. For the second category, it would be hard to measure directly the number of persons exposed to shocking child-porn images. Indirect measurement may be possible through questionnaires, or by developing a test to determine the time it takes an average internet user to find shocking images. Presumably the more time it takes to find child-porn images, the less likely it is that young or vulnerable people will find them by accident. There may be an indirect correlation between the time it takes to find a child-porn image and the number of persons exposed in a given year.
97The third category would be difficult to measure without experiments between two populations, one in which a child-porn blocking measure has been imposed by regulation, and another where no such regulation has been applied. One would then measure the difference in usage levels of the internet.
| Category of benefit | Value of the benefit to society | Causation link with the blocking measure | How to measure success |
| Reduction in number of molested children | Quantifiable | Weak | Number of children sexually abused during a year. |
| Reduction in young or vulnerable individuals exposed to shocking images | Quantifiable | Strong | Time it takes an average uninitiated adult or adolescent to find child-porn images on the internet. |
| Increased use of internet due to making the internet safer | Quantifiable | Medium | Usage statistics, but requires experiment with control group. |
| Affirming society’s refusal to tolerate serious human rights violations on the internet | Non-quantifiable | Strong | Surveys to measure citizens’ awareness of regulatory measure. |
98The last category, which consists of affirming an important normative value in society, might be measured by surveys asking people if they are aware of the regulatory enforcement measures designed to block access to child porn images on the internet and the sanctions that accompany any effort to circumvent those measures. Success would be measured through the level of public awareness of the measure.
Identifying the maximum possible benefits
99When developing success metrics for the regulatory alternatives, the drafters of the cost-benefit analysis should try to define first what “total success” would look like for each category of benefits, as compared to the baseline scenario. The difference between total success and the baseline scenario would represent the maximum benefits that the enforcement policy can possibly achieve. The total success scenario should correspond to 100% enforcement of the content policy on the internet, i.e. 100% blocking of the harmful content, regardless of costs.
100For example, for a measure designed to fight child pornography, the 100% enforcement scenario might correspond to the use of deep packet inspection and image recognition technology to block anything that looks like a child without clothes. This technology would probably block all child porn images, but would also block images from medical textbooks and artwork. The over-blocking would create significant costs. But the benefits would correspond to the maximum achievable, i.e. the outer limit of what is possible.
101This level of 100% enforcement, and the corresponding maximum potential level of benefits, will almost never be the optimal level, but will help define the maximum range of available benefits. The baseline scenario represents the lower limit of the range, and the 100% enforcement represents the upper limit of the range. The optimal level will be somewhere in between. The optimal level will correspond to the amount of enforcement at which the marginal cost of an additional unit of enforcement is equal to the marginal benefit gained by the extra unit of enforcement. Put another way, the level of enforcement should be situated at a point where the sum of the costs of harms related to the content policy and the costs of prevention is minimized.
102In most cases, the marginal cost of enforcement increases as the level of enforcement approaches 100%. Achieving 99% enforcement may be hugely more costly than achieving 95% enforcement. The marginal benefit generally remains constant, or may even decrease. Thus optimal enforcement is usually less than 100%.
103By first looking at an extreme enforcement scenario, it will be easier to apply a sensitivity analysis to see what benefits less-than-total enforcement would deliver. For example, the drafters of the cost-benefit analysis may conclude that 100% blocking of child porn images is not likely to reduce the number of sexually abused children, or that the connection between blocking and the reduction in the number of victims is too tenuous to support projections. By contrast, total blocking may reduce by “x ” the number of vulnerable people who, in a given year, accidentally find child porn images and are influenced by them. “x ” is the maximum benefits in this category that could possibly be achieved. The sensitivity analysis would seek to determine by how much “x ” is reduced for various regulatory options that consist of blocking of less than 100% of all images.
104For the category of benefits associated with affirming society’s refusal to tolerate exploitation of children on the internet, the total enforcement scenario might yield a public awareness factor that is 100% higher than the baseline scenario. The drafters of the cost-benefit analysis may find, however, that an enforcement scenario of 85% (as opposed to 100%) also yields a 100% increase in awareness compared to the baseline scenario. The sensitivity analysis would show that a lower level of enforcement still yields a high level of benefits for this category.
For each regulatory proposal, estimate the level of benefits between the maximum and minimum
105Once benefits have been broken down into categories, and the minimum (corresponding to the baseline scenario) and maximum (corresponding to 100% enforcement) for each category have been identified, the drafters of the cost-benefit analysis should determine where benefits lie for each of the regulatory proposals. Drafters should use the same methodology as they applied to the baseline scenario and to the 100% scenario. For each technical option (search engine, DNS blocking, self-regulatory initiatives), the drafters would have to determine if the option yields benefits that are close to the baseline, close to the 100% scenario, or halfway in between. It may be possible to position each option according to its level of achievement of the maximum. For example, a DNS blocking measure may attain 100% of the maximum benefit for the category of affirming society’s refusal to tolerate child porn images, and 85% of the maximum for category corresponding to preventing young or vulnerable people from being exposed to shocking images.
The direct costs resulting from each proposal
106After analysing benefits, the next step in the cost-benefit analysis involves examining the direct costs generated by each proposal compared to the baseline.
107Calculating the level of direct cost is relatively straightforward. This is what most regulators already do when they prepare a cost-benefit analysis. If the proposal involves setting up a new regulatory authority, the cost-benefit analysis will include a calculation of the setup costs and annual operating expenses of the new authority. The impact assessment then generally attempts to calculate the level of compliance costs for businesses. If the proposal involves new reporting requirements for businesses, the cost-benefit analysis will attempt to calculate the number of businesses affected and the annual cost per business of completing the new paperwork.
108Direct costs would also include capital expenditures and operating expenses for internet intermediaries that would not have been incurred but for the new regulatory measure. An example of this kind of expense is the cost of installing and operating filtering equipment at various points in an ISP’s network. Another example is the annual cost of processing “right to be forgotten” requests by a search engine, including the costs of hiring new staff to evaluate delisting requests. After the Costeja judgment of the European Court of Justice, operators of search engine created online processes to handle delisting requests made by European individuals. The search engine operators had to hire new employees to handle the hundreds of thousands of delisting requests filed using the online tool. Search engine operators also had to deal with new legal claims before courts and data protection authorities, giving rise to legal costs. The one-time costs of setting up the process and the annual costs of administering the process would be included in the category of direct costs created as a result of the regulatory measure.
109Direct costs can generally be estimated in monetary terms. In the case of delisting requests under the European “right to be forgotten,” it should be fairly easy to calculate the amount of the direct costs associated with each delisting request. It would suffice to calculate the sum of (i) the additional costs incurred by regulatory authorities and courts to deal with delisting requests and (ii) the total costs incurred by operators of search engines, and then divide these aggregate costs by the number of delisting requests. This would yield the direct cost associated with enforcing the new right recognized by the European Court of Justice.
The indirect costs resulting from each proposal
110Direct costs are only part of the story. Most actions taken by internet intermediaries to enforce content policies create indirect costs, and these are much harder to evaluate. They are generally ignored today in impact assessments. Some of these costs may be difficult or impossible to quantify in monetary terms, in which case one of the methods examined earlier in this chapter.
111If we use the example of measures to fight online copyright infringement, the following indirect costs may be generated:
Harms to innovation and competition
112The imposition of obligations on technical intermediaries might discourage market entry, or encourage market exit by smaller players, thereby decreasing competition and innovation (Haber, 2010; Breyer, 1982). Technical measures might be proportionally less costly for large players to implement than for smaller players or new market entrants, thereby reinforcing the market power of large existing players. If these costs for smaller intermediaries are not reimbursed, they could create barriers to entry, thereby making markets less contestable and competitive. Technical intermediaries might also increase their prices to reflect the cost of the technical measure, which could cause certain internet users to reduce their activity (Lichtman, 2005). Imposing obligations on technical intermediaries may also damage innovation and growth by altering the open and neutral architecture of the internet. This might decrease innovation, competition and growth, not to mention affect fundamental rights.
113In addition to the cost burden associated with implementing a given technical measure, its imposition may also skew competition because the measures reflect a particular technology or approach promoted by one actor that disadvantages its competitors. To minimize this potential cost, regulators should endeavor to impose technologically neutral remedies, performance standards as opposed to technologically specific design standards (Breyer, 1982). Regulators should make sure that measures are subject to public consultation and stakeholder comments, so as to identify any hidden competitive distortions (Hancher, Larouche and Lavrijssen, 2003).
Harms to fundamental rights
114A measure designed to limit access to harmful content will in most cases affect freedom of expression and privacy. Those indirect costs are hard to quantify, so one of the techniques mentioned in earlier in this chapter will have to be used. Freedom of expression and freedom to access information do not justify unrestricted access to all content. Nevertheless, most technical measures will be over-broad, blocking some content that is legitimate. Any overspill will be considered as an interference with freedom to access information.
115While freedom of expression and freedom to access information are probably the most important rights affected by technical measures, the right to privacy will also likely be affected. Measures intended to limit access to harmful content will often have a potential adverse effect on privacy. For example, a system that tracks online behavior to detect copyright infringement carries inherent risks because the system, or the data it collects, can potentially be misused. Like the collection of geolocation information, the collection of IP addresses for purposes of limiting copyright infringement must be surrounded by safeguards to avoid possible abuse. A system that does not permit the identification of individual downloaders will create lower privacy costs than a system that permits such identification.
Dividing indirect costs into categories
116For each category of indirect cost, the drafters of the impact assessment should determine whether the costs are quantifiable, and if so whether quantification is possible in monetary terms. For those costs that are not quantifiable in monetary terms, the drafters of the impact assessment should use one of the techniques described earlier in this chapter to measure the intensity of the relevant cost. Scoring systems, such as the one proposed by Robert Alexy for fundamental rights, might be used. Using qualitative labels such as “extremely high,” “high,” “medium,” and “low,” may still permit certain proposals to be eliminated without it being necessary to add up the qualitative scores in a column in order to arrive at an aggregate score of non-quantifiable costs. A proposal may be eliminated because we know that its net benefits are necessarily lower than those of another competing proposal. For example, as between two proposals with equal benefits and equal quantifiable costs, the proposal that creates a “high” level of harm to freedom of expression and to innovation would necessarily yield lower net benefits than the proposal that creates a “medium” level of harm to freedom of expression and to innovation.
117Another way to eliminate certain proposals based on the qualitative labels is to impose a constraint in the cost-benefit analysis, such as: “any proposal with a harm to freedom of expression exceeding the ‘medium’ level shall be eliminated.” Constraints will be discussed further below.
118To illustrate how non-quantifiable costs are treated, let us examine three proposals designed to reduce online copyright infringement. Proposal A involves deep packet inspection, proposal B involves DNS blocking, and proposal C involves graduated response.
119In this example, proposal A consists of deep packet inspection permitting the ISP and right holders to determine with a high degree of accuracy the content that should be blocked because of a violation of copyright. The impact on freedom to access legitimate information will be low because there will be few, if any, false positives. However people may reduce their activity because they feel they’re being watched. Reduction of activity would indirectly reduce freedom to access information.
| Proposal A deep packet inspection | Proposal B DNS blocking | Proposal C graduated response | |
| Quantifiable benefits per annum compared to the baseline scenario | 500M€ | 500M€ | 400M€ |
| Direct costs per annum compared to the baseline scenario (quantifiable) | 400M€ | 100M€ | 300M€ |
| Net quantifiable benefits | 100M€ | 400M€ | 100M€ |
| Indirect costs per annum compared to the baseline scenario | |||
| Harms to freedom to access information (non-quantifiable) | Low | Medium | Low |
| Privacy harms (non-quantifiable) | Extremely High | Low | Medium to high |
| Harm to net neutrality and the proper functioning of the internet (non-quantifiable) | High | High | None |
120The negative impact on privacy will be extremely high because deep packet inspection can easily be used to spy on individual opinions. Some countries use this technology to do precisely that. The negative impact on the internet’s end-to-end architecture will be high because network equipment will be used not just to route packets but to apply content policies. This would be viewed by net neutrality advocates as a dangerous disrespect for the layered character of internet transmissions. Net neutrality dictates that except for measures needed to assure good network management, routing and transmission equipment should not become involved in the content of packets. Content policies should be implemented whenever possible at the edges of the network, using software on terminal equipment.
121Proposal B consists of blocking access to certain websites through DNS blocking. DNS blocking consists of substituting a wrong IP address for the right one in the central directory that permits websites to be identified and found. The potential harm to freedom to access information is medium because there may be over-blocking. DNS blocking can block perfectly legal content in addition to the targeted illegal content. The negative impact on privacy is low because the technology is less capable of tracking individual behavior. Unlike deep packet inspection, DNS blocking does not keep track of individual attempts to access the content. The negative impact on internet architecture and net neutrality is high because DNS blocking amounts to inserting false addresses in the local version of the DNS registry, thereby tricking routing equipment to deliver packets to the wrong destination.
122Proposal C consists of issuing administrative fines against persons who have been detected as illegally downloading copyrighted works. No blocking occurs, but individuals may reduce their activity because they are afraid of being watched, leading to some restriction on freedom to access information. This yields a low harm to freedom of expression. The negative effect on privacy is medium to high, because the mechanism tracks individual usage, and results in network operators and public authorities translating IP addresses into names of individual subscribers. Mechanisms like this could potentially be misused to spy on individuals, even though most graduated response mechanisms include strict controls over the use of individual IP addresses.
123Proposal C respects net neutrality because the system does not interfere with routing functions on the internet. The internet continues to function as it should. There is no harm to the internet architecture. The IP addresses of potential infringers are identified by agents of rightholders who introduce themselves into peer-to-peer networks and make note of users who offer copyright-protected content. Outside agents watch what is happening, but do not interfere with the functioning of the internet. The benefits of graduated response are lower than for DNS blocking and deep packet inspection because graduated response only affects peer-to-peer file exchanges.1
How to rank proposals
124Going through this exercise already permits us to conclude that proposal C is superior to proposal A. Both have a positive quantifiable net benefit of 100M€, but proposal A scores less well than proposal C with regard to harms to privacy and to net neutrality. However, we are not able to determine whether proposal B is superior to C, or vice versa. Proposal B has a much higher net quantifiable benefit (400M€ versus 100M€) than C, but B scores worse than proposal C on freedom of expression and net neutrality, while better than proposal C on privacy. As between proposals B and C, the trade-off is therefore whether the 300M€ difference in quantifiable net benefits, plus the better protection of privacy under proposal B more than offsets the marginal harm to freedom of expression and net neutrality caused under proposal B compared to proposal C.
125If we assume that the harms to freedom of expression and privacy are roughly equivalent (which is not an unreasonable assumption given European case law), we are left with a more explicit trade-off: is the DNS blocking system’s harm to net neutrality worth more than 300M€ per year?
126We see in this simplified example that even where certain costs are not quantifiable, the process of comparing costs and benefits can nevertheless permit some proposals to be eliminated, and for the remaining proposals, the process can make trade-offs more explicit.
Applying additional constraints
127The list of benefits and costs in the cost-benefit analysis may not capture the full range of parameters identified in the questionnaire. Where this is the case, the additional parameters can be applied to the outcome of the cost-benefit assessment in order to help rank the proposals. For example, if two or three proposals emerge from the cost-benefit analysis as requiring trade-offs based on non-quantifiable costs or benefits, additional rules may help eliminate certain proposals, or give added weight to others.
128Fundamental rights: Where qualitative labelling is used to characterize costs to fundamental rights, a rule could be created to eliminate any proposal that contains an “extremely high” harm to any fundamental right. Similarly, a proposal that contains more than one “high” interference with a fundamental right could be eliminated. This constraint would permit the application of an ethical filter after the initial cost-benefit analysis, such as that proposed by Nussbaum (2002).
129International legitimacy: If international effects have not been incorporated into the indirect costs of the cost-benefit analysis, an additional rule could be created to give added weight to a proposal that has been used in other democratic countries, versus a proposal that has not been used in another democratic country. Similarly, added weight could be given to a proposal that is consistent with the OECD recommendations on internet policymaking.
130Technology neutrality: If technology neutrality has not been incorporated into the indirect costs of the cost-benefit analysis, an additional rule could be created to give added weight to a proposal that is technologically neutral versus one that is not.
131Adaptive regulation: If “flexibility” has not been incorporated into the benefits or costs of the cost-benefit analysis, a rule could be created to give added weight to a proposal that is flexible, and can be easily modified or withdrawn, as compared to a proposal for which the modification costs are high.
Conclusion on how to rank proposals
132Cost-benefit analyses will fall into one of four categories:
133Category 1: All benefits and costs can be converted into monetary amounts. This case is the most straightforward, but will be unusual for measures designed to limit access to harmful content.
134Category 2: Benefits and costs will both include a combination of quantifiable (in monetary terms) and non-quantifiable elements. This will be the most frequent scenario, to which we will return below.
135Category 3: Benefits are quantifiable in monetary terms, and costs consist of both quantifiable (in monetary terms) and non-quantifiable (in monetary terms) elements.
136Category 4: the mirror image of category 3: All costs are quantifiable in monetary terms, and benefits consist of both quantifiable (in monetary terms) and non-quantifiable (in monetary terms) elements.
137The simplified example above (proposals A, B and C to fight copyright infringement) falls into category 3: All benefits are quantifiable in monetary terms. This example helped show how the use of qualitative labels on the costs side can still help eliminate certain proposals, and make choices more explicit.
138Environmental, or health and safety measures often fall into category 4. Benefits include difficult-to-quantify elements such as saving a species of frog or wildflower, whereas costs are generally quantifiable in a monetary sense. To deal with category 4, United States regulators advise using a “cost effectiveness analysis” (CEA), which consists of calculating the monetary cost of achieving a given unit of benefit, the unit being a non-monetary measurement such as a year of life saved, or the preservation of a hectare of wilderness. The proposal that delivers a unit of benefit at the lowest monetary cost would be preferred over other competing proposals.
139Let us examine category 2, which will be the most frequent for regulatory measures designed to restrict access to internet content. Non-quantifiable elements will appear on both the benefits and cost sides, which makes the ranking process complicated. Here is a way it could be done.
140Step 1: calculate net monetary benefits. As with our category 3 example examined above, a useful first step is to calculate the net quantifiable monetary benefits corresponding to each proposal.
141Step 2: check if any proposals can be eliminated based on the constraints mentioned above. For example, a proposal with an “extremely high” impact on privacy or freedom of expression would be eliminated.
142Step 3: check that the success parameters have been carefully defined and applied on the benefits side of each proposal. Eliminate false success parameters. The important point is to break benefits into different categories, determine ways to measure success compared to the baseline scenario for each category (“success metrics”), and then estimate how the success metrics will vary for each proposal compared to the baseline, taking into account the uncertainties of causation. As illustrated in the child pornography example, causation may be so tenuous for certain categories of benefits that the relevant categories should be eliminated entirely from the equation.
143It may be possible to convert the relevant the success metric into a score. The European Commission’s SURVEILLE project on police surveillance measures (examined in Chapter 7) attempts to do this by attributing a score for effectiveness of the surveillance measure in fighting crime. However, where there are different categories of benefits – as will often be the case for measures limiting access to harmful content – attributing a single aggregate score for multiple benefit categories can be problematic.
144Step 4: for difficult-to-quantify costs, choose between using a scoring mechanism such as Alexy’s, or using qualitative labels such as “high,” or “extremely high.” Because European courts have held that the right to privacy and the right to freedom of expression have equal normative value, a scoring mechanism relating to these two rights would work. A score of +1 for privacy could offset a score of -1 for freedom of expression. As Alexy’s system recognizes, severe restrictions on either of these rights are not permitted, so the set-off rule only works for light or moderate restrictions. One simplifying measure therefore would be to use scoring for privacy and freedom of expression rights, both on the costs and on the benefits side.
145However, even this is tricky. Let us take the example of measures designed to protect the right to be forgotten. The benefits would consist in protecting privacy, and costs would involve limitation to freedom to access information. Theoretically, these could be set-off. Various technical measures used to achieve the right to be forgotten would carry different levels of benefits for privacy, and different levels of impacts on freedom of expression. Scoring these different levels is not straightforward, in large part because the right to be forgotten benefit is not intended to be total. Its objective is to make access to certain content less likely, but not impossible. Attributing a score to the benefit corresponding to each technical measure could be difficult.
146Example: comparing two right to be forgotten measures:
147Proposal A: Delisting on local domain search results covers 98% of search requests in the territory being covered.
148Proposal B: Delisting on all domains for users within the territory using IP restrict covers close to 100% of search requests in the territory.
149Proposal A results in a very large difference compared to the baseline (from 0% to 98% of coverage). Proposal B results in a small incremental benefit (+2%) compared to proposal A. It is unclear how Alexy’s system would score the small incremental increase in benefit between proposals A and B.
150Considerable work is needed before relying on a scoring mechanism, particularly where incremental differences in effects on fundamental rights are small.
Conclusion on cost-benefit analysis
151It is difficult to conduct a cost-benefit analysis with hard-to-quantify benefits and costs. I have touched on methods that can be used. The ideal solution is to transform all costs and benefits into monetary values. Standardized values have been created relating to the value of a human life. Once a standard value has been fixed for the statistical value of one year of life saved, this statistical value can be used to measure the benefits associated with regulatory proposals to limit air pollution, for example. If a monetary value can be associated with a year of human life, it is not absurd to think that a monetary value could be attributed to the protection of privacy or freedom of expression.
152Other methods consist of choosing non-monetary units to count benefits or costs, such as the number of films produced annually, or the time it takes for an average person to find pornographic images on the internet. Scoring or qualitative labels are also possible. Before the cost-benefit analysis I describe can become operational, it would be necessary to refine and possibly standardize these approaches. The methodology would lose much of its value if regulators used different approaches.
Public consultation
153The questionnaire and cost-benefit analysis should then be subject to public consultation. If we refer again to regulatory proposals in the European system for regulating electronic communications, public consultation is a critical part of the process. The regulator prepares its justifications for the regulatory measure based on the market analysis and the requirements of the European Framework Directive 2002/21/EC. The analysis and proposal are then submitted to public consultation. The comments received from the public consultation are analyzed by the regulator. The regulator must show in its final proposal, which is sent to the European Commission’s Article 7 Task Force, that it has taken due account of the various comments received as a result of the public consultation. The peer review conducted by the European Commission will verify whether the comments of stakeholders have been taken into account.
154In the system I propose here, the questionnaire and cost-benefit analysis would be published for public consultation before being sent to peer review. After receiving public comments, the drafters of the impact assessment would summarize the main inputs from the consultation process, and revise the conclusions of the cost-benefit analysis and questionnaire accordingly.
155Public consultation contributes to transparency, which enhances the legitimacy of the proposed action and the likelihood of compliance by affected stakeholders (Hancher, Larouche and Lavrijssen, 2003). Public consultation is part of better regulation (OECD, 2012).
Institutional peer review
156The literature on better regulation shows that regulatory impact assessments are of limited utility if they are not reviewed by an independent institution (OECD, 2012; Conseil d’Etat, 2016). The initiators of regulatory proposals generally act in response to political needs to take action with regard to a perceived market or regulatory failure. The proponents of regulation will often have a conflict of interest because their superiors have made a political decision that a given regulatory solution is needed. The drafters of the impact assessment will tend to reverse engineer the impact assessment in order to support the proposal that meets the political needs of their management. The persons completing the impact assessment will often have no interest in examining indirect costs of the measure, or the risk of regulatory error. Those costs and risks will materialize only much later, well beyond the relevant political horizon. The drafters’ priority will be to complete the impact assessment as quickly as possible and for the document to contain enough evidence to support the proposal that political decision-makers want.
157This is why independent review is critical. As noted above, the European Commission’s “Article 7” reviews in the electronic communications sector provide a useful example. The peer review is conducted by individuals from the European Commission who are independent from the institutions proposing the regulatory measure. The peer review team is supervised by officials at the European Commission whose only objective is to ensure that the principles set forth in the European Directives are applied properly. Their goal is good regulation rather than satisfying political demands for action. Proponents of regulatory measures must plead their case before the peer review panel, in some cases asking for exceptions from the strict rules imposed by the directives.
158The Article 7 Task Force also has the advantage of reviewing similar regulatory proposals from throughout Europe. The panel gains accumulated knowledge and perspective that may not be available to regulators in a single member state.
159In Europe, the best peer review panel for regulatory proposals relating to internet content would be the European Commission, because of its relative independence from political pressure at the member state level. However, this would require the adoption of a European directive or regulation giving the European Commission power in the field of internet regulation. As is the case for peer review in the electronic communications sector, the European Commission could potentially have the right to open phase II proceedings where the rationale given by member states for their regulatory proposals is insufficient. In some cases, the European Commission would have the right to veto measures if they diverge too much from regulatory best practices. A peer review system could also be established at the national level (Conseil d’Etat, 2016).
160A peer review system is currently used in the United States for any federal regulatory proposal. Each agency must submit its cost-benefit analysis to the OIRA (see Chapter 5, supra). The OIRA scrutinizes the cost-benefit analysis to ensure that it complies with the methodology imposed under Circular A4. OIRA can request revisions to the cost-benefit analysis, or even veto the measure. This power explains why the director of OIRA is referred to as the United States’s “regulatory czar.”12
161Currently, OIRA does not review proposed legislation that is to be submitted to Congress, or proposed regulation from independent regulatory authorities such as the FCC. Thus most regulatory or legislative proposals involving internet regulation escape OIRA’s scrutiny. United States law would have to be modified if OIRA were to have power to review regulatory proposals involving internet content. However, OIRA seems to be a good example of effective peer review.
162After receiving comments from the peer review institution, the drafters of the impact assessment would modify the proposal to take the peer review comments into account. Once this is done, the proposal, including the questionnaire and cost-benefit analysis, would be submitted to the relevant decision-making authority for final adoption. If the measure is legislative, the relevant decision-making authority would be the legislature. If the measure is regulatory, the relevant decision-making authority would be regulatory authority or government agency.
Periodic review of the measure
163The last element of the methodology is to ensure that regulatory measures are reviewed ex post to ensure that they are still fit for purpose. Measures that are not delivering the expected level of benefits, or that are creating unanticipated costs, should be modified or eliminated. In the field of electronic communications, regulators must conduct periodic market analyses to identify technological and market trends that have occurred during the last 36 months, and evaluate the impact of those trends on competition in the market. Where a particular regulatory measure is no longer necessary to ensure effective competition, the measure must be removed. The European Commission’s Article 7 Task Force panel for electronic communications will examine the market analysis and ensure that the regulatory authority has considered the option of removing regulatory provisions where they are no longer necessary.
164Given the fast rate of change of internet technology and markets, the risk of regulatory error is high (Shelanski, 2013). The impact assessment and peer review are designed to reduce the risk of error upfront. Periodic ex post review is designed to detect and correct errors that become visible later. As examined in Chapter 5, “adaptive regulation” (Whitt, 2009) requires that regulatory measures be tested regularly to determine whether they are still fit for purpose. Ex post review is advanced by the Conseil d’État (2016) as a key element of good law making.
Conclusion
165The entire system is summarized in the following table:
| Step 1: | Questionnaire to define:objectives of the proposal;available alternatives (technical, institutional);benefits, costs (description);international benchmarks. |
| Step 2: | Cost benefit analysis (CBA)–1: baseline scenariocreate baseline scenario/counterfactual;define benefits and success metrics under baseline. |
| Step 3: | CBA-2: measure benefits under proposals measure quantifiable benefits under various proposals compared to baseline;use qualitative measurements for non-quantifiable benefits. |
| Step 4: | CBA-3: measure costs under proposals direct costs: measure quantifiable direct costs under various proposals compared to baseline;indirect costs: measure quantifiable indirect costs compared to baseline;non-quantifiable costs: use qualitative measurements, Alexy scoring, or qualitative comparisons. |
| Step 5: | CBA-4: Additional constraintsApply additional rules to CBA:any proposal with “extremely high” cost score for any fundamental right would be eliminated;any proposal with more than one “high” cost score for fundamental rights or net neutrality would be eliminated;any proposal with low international acceptance would be downgraded. |
| Step 6: | Public consultationQuestionnaire and CBA are published for public comment. |
| Step 7: | Questionnaire and CBA are modified as a result of public consultation Impact assessment summarizes public comments and modifies conclusions where necessary. |
| Step 8: | Institutional peer reviewImpact assessment (questionnaire and CBA) are submitted to independent institution (e.g. European Commission) for review and quality control. |
| Step 9 | Proposal is modifiedIf necessary, the proposal is modified to take into account comments of peer review. |
| Step 10: | Proposal is adoptedIf legislative proposal, text of proposal and impact assessment are presented to legislature;If regulatory proposal, text is adopted directly by government or regulatory authority. |
| Step 11: | Perodic review mechanisms to:measure benefits (success metrics);measure costs;modify or remove regulation where necessary. |
Notes de bas de page
Le texte seul est utilisable sous licence Licence OpenEdition Books. Les autres éléments (illustrations, fichiers annexes importés) sont « Tous droits réservés », sauf mention contraire.
Nouvelles énergies pour la ville du futur
Eva Boxenbaum, Brice Laurent et Annalivia Lacoste (dir.)
2013
Recharger les véhicules électriques et hybrides
Matthieu Glachant, Marie Laure Thibault et Laurent Faucheux
2013
« Moi je lui donne 5/5 »
Paradoxes de la critique amateur en ligne
Dominique Pasquier, Valérie Beaudouin et Tomas Legon
2014
Les technologies numériques de santé
Examen prospectif et critique
Valérie Fernandez, Laurent Gille et Thomas Houy
2015
Le phénomène « pro-ana »
Troubles alimentaires et réseaux sociaux
Antonio A. Casilli et Paola Tubaro
2016
Diversifier le recrutement public
Le cas des magistrats
Florence Audier, Maya Bacache-Beauvallet et Éric Mathias
2016
Smart(er) Internet Regulation Through Cost-Benefit Analysis
Measuring harms to privacy, freedom of expression, and the internet ecosystem
Winston J. Maxwell
2017
