Desktop versionMobile version

Proceedings of the fourth Resilience Engineering Symposium

 | 
Erik Hollnagel
, 
Éric Rigaud
, 
Denis Besnard

New Balanced Scorecard leading indicators to monitor performance variability in OHS management systems

Frédéric Juglaret, Jean-Marc Rallo, R. Textoris, Franck Guarnieri and Emmanuel Garbolino

Abstract

While it has been established for many years that the management of Occupational Safety and Health (OHS) is carried out by means of management systems, the question of how to measure the performance and control of these systems is still current. Uncertainties related to their operation and the difficulty of gathering information about their level of performance makes control of performance variability a challenge. This article addresses this problem. It is in three parts. The first part outlines the general context. It describes traditional health and safety indicators, the links between processes in management systems and establishes the requirements of resilience engineering. An advanced indicators model is proposed. The second part describes the Balanced Scorecard tool. An Occupational Health and Safety (OHS) Balanced Scorecard model is built using the control of regulatory compliance sub-process. The final part presents some specific examples of compliance control indicators, which are the results of an experiment carried out in a French aerospace company.

Author's note

Note portant sur l’auteur1

Note portant sur l’auteur2

Note portant sur l’auteur3

Note portant sur l’auteur4

Note portant sur l’auteur5

Full text

1 Background

1This first section discusses the concept of the safety indicator. First the general concept is defined. Then it is demonstrated that traditional safety indicators are in many ways lacking when it comes to proactive safety management. We then show the sub-processes of Safety Management Systems (SMS) may, to a certain extent be able to fill some of the gaps identified. Finally, we propose a model of advanced indicators which aims to reduce the performance variability of SMS and promote the implementation of resilience engineering.

1.1 Safety indicators

2All management systems in all domains use indicators. An indicator aims to facilitate the assessment of a situation or the state of a system by an actor (or group of actors) within a management system. The information provided by indicators facilitates decision-making and implementation of appropriate actions in situations that deviate from the norm. Indicators therefore interact with three components: strategic targets, actors, and actions (Lorino, 2003). The appropriateness and the quality of indicators are measured according to their strategic relevance, their cognitive efficiency and their operational relevance.

3The literature deals extensively with the differences between leading and lagging indicators. Typically, it demonstrates the lack of consensus on a naming convention. We propose in this article to adopt a simple definition that incorporates the principal features of the most widely used designations. An indicator is described as leading or lagging according to its place in the management system and its operational goal. Lagging indicators indicate results, and leading indicators either act as control (implementation) or intermediate indicators. It should be noted that depending on the angle of analysis, and its place in the management system, an intermediate indicator may be used to evaluate the results of activities that underlie the overall management system.

1.2 The limitations of traditional safety indicators

4Traditional measures of performance and monitoring of OHS management are based on result indicators that measure past failures. They reflect a dysfunction in the system that is the cause of work-related accidents or illness. The analysis of accidents and incidents facilitates the establishment of feedback mechanisms. This knowledge helps to avoid similar adverse events recurring in the future. However, this learning from experience can be limited by the small number of case studies. Depending on the maturity of the management system and its performance levels, the number of work-related accidents or illnesses may be insufficient to provide enough material for an effective assessment. When the absence or lack of safety is measured in this way, it does not help at all to anticipate future events that have never happened before (Cambon, 2007). Moreover, the description of events can be different, sometimes even within the same industrial group. Usually expressed as an index or ratio, these indicators have a very low predictive capacity. Therefore they do not facilitate action and decision-making. When used alone, these OHS management results do not meet the requirements of resilience engineering (namely reaction, learning, monitoring and anticipation). It is therefore necessary to obtain and identify ‘advanced’ sources of information from the OHS management system to fill these gaps and move towards a more proactive OHS management.

1.3 OHS safety management systems: sub-processes

5OHS Safety Management Systems are composed of several interacting processes. Depending on the system used, the number and names of these sub-processes can vary, but generally they all assume the same logic of continuous improvement. We will show that most of these SMS sub-processes support the implementation of resilience engineering (Table 1).

6The Management review sub-process facilitates organizational learning and the anticipation of unexpected situations. It adapts OHS goals and policy to evolution and changes in the environment of the management system (social pressure, competition, etc.) and to the corresponding safety results. Training, preparation for emergency situations, skills development and the definition of employees’ roles and responsibilities can improve their responsiveness when faced with a threat. Medical monitoring and audit and control supervisory sub-processes help to monitor these threats. Regulatory compliance management can be used as a knowledge base, enabling companies to implement various protection and prevention measures. Risk analysis is used to round out the list of measures to be implemented and provides a more exact and responsive analysis of actual work situations. Both of these activities reduce risk through prevention measures used to isolate the threat, and protective barriers which limit the adverse consequences of an event. Risk analysis and regulatory compliance management can therefore help to better prepare an organization when faced with adverse events (responding) and to better monitor these threats (monitoring). Risk analysis, complemented by accident analysis can promote learning from experience in order that events that have already occurred do not recur in the future. The risks models established must be updated to improve anticipation of future threats (knowing what to expect).

Table 1. Main safety management sub-processes (Cambon, 2007) and their contribution to the requirements of resilience engineering.

Table 1. Main safety management sub-processes (Cambon, 2007) and their contribution to the requirements of resilience engineering.

1.4 A model of advanced indicators

7For resilience engineering, it is relevant to evaluate the findings and the extent to which each of these sub-processes can contribute. Improved monitoring of activities related to the sub-processes of a management system can also improve control of variability in overall system performance. Advanced indicators, when implemented, can help to better monitor and assess the level of operation.

8The aim of the model is to improve implementation and monitoring of results from sub-processes (Figure 1). The advanced indicators system also improves control of the variability in overall system performance and better anticipation of health and safety performance.

2 The OHS Balanced Scorecard Model

9First, the Balanced Scorecard tool is described. Then we propose a model of the system of second-level leading indicators on the sub-process of the management of regulatory compliance, integrated in the form of an OHS Balanced Scorecard. Another Balanced Scorecard model based on the management of professional risk sub-process has also been constructed (Juglaret et al., 2011).

Figure 1. Leading indicators on OHS sub-processes integrated into an OHS management model (adapted from Hollnagel, 2006).

2.1 Scorecard overview

10The scorecard tool is designed to facilitate decision support. The quality of a scorecard comes primarily from its ability to inform the various actors in the management system of the presence of deviant situations, and to facilitate decision-making in order to implement the necessary corrective actions. The information provided should be updated at intervals appropriate to the particular management system in question. Depending on the nature of the information required, indicators can be represented digitally, graphically, etc. Finally, the quality of a scorecard rests upon both its ability to alert managers to the occurrence of a deviant or unwanted event (in order to reduce, or ideally, remove it), and on its ability to anticipate the consequences of adverse situations.

11The Balanced Scorecard concept (Kaplan & Norton, 1996) was originally intended to be used by companies in their strategic management and implementation activities. It is interesting to now see how the application of this tool can be transferred from the domain of management systems to that of health and safety. The value of the approach lies in the ability to take into account factors other than just the end result. The Balanced Scorecard tool has the ability to integrate advanced performance indicators at different levels. In this context, construction of an OHS Balanced Scorecard is relevant to the proactive management of health and safety.

2.2 Advanced indicators for regulatory compliance

12The objective of the Regulatory Compliance Management process is to comply with all applicable regulatory requirements. It can be seen as a system composed of sub-processes which operate according to a rationale of cyclic functioning and continuous improvement. This Plan-Do-Check-Act (PDCA) procedure involves the following: planning of compliance actions (Plan), achieving compliance with un-met regulatory obligations (Do), identification of discrepancies with the regulations (Check), and regulatory monitoring and adjustment of the applicable scope (Act/Adjust). Using these different processes, a model of second level advanced indicators can be established (Figure 2).

Figure 2. Regulatory compliance management model.

13The Rate of compliance indicator is based on completed assessments. The Progress of audits management indicator improves the implementation of these assessment activities and clarification of the results. A second management indicator Actions scheduled makes it possible to better anticipate and prepare compliance activities.

3 Method and Results

14This section describes the actual implementation of an OHS BSC and presents the results. After outlining the experimental context, some examples of indicators for compliance control are presented and discussed.

3.1 Experimental context

15The company concerned is a global player in the field of aeronautics. The activities of this company are very varied and they have multiple production sites. The risks associated with these production activities are many and varied: chemical hazards, handling, mechanical, fire and explosion, electrical, work environment, etc.

3.2 Regulatory compliance indicators

16The experiment shows that the number of applicable regulatory obligations increased significantly from 2009-2011 (Figure 3). The increase is from 6,373 requirements assessed as applicable in the first quarter of 2009 to over 9,000 in 2011. Despite this increase in the applicable scope, the organization has improved its level of control of regulatory compliance, from 89 % to 97 %. The classification of regulations into categories provides a new perspective on the distribution of regulatory requirements (Figure 4) and shows that legal requirements related to chemicals have a significant impact in this study.

Figure 3. Regulatory compliance evolution

Figure 4. Regulatory compliance distribution according to category of hazard

17The evaluated regulatory requirements were all associated with prevention measures (Figure 5) which correspond to management principles (individual protection, staff training, operational control, etc.). In this case study, regulatory requirements for personal protective equipment are not all met (89 %) which leads to the conclusion that the provision of personal protective equipment would significantly reduce risks.

Figure 5. Level of regulatory compliance according to prevention measure

4 Conclusion

18This experiment in the aviation industry shows that a system of advanced indicators based on a sub-process of an OHS management system offers interesting perspectives for the implementation of resilience safety engineering. Traditional OHS indicators, although essential to validate the overall OHS strategy and the implementation of organizational learning, are by themselves insufficient to meet the requirements of proactive safety management. The construction and use of leading indicators integrated into an OHS Balanced Scorecard improve the control of performance variability in safety and encourage proactive safety management. To further improve this level of control and proactive management, it would be interesting to identify and construct leading indicators based on other OHS management processes (e.g. operational control, documentation, communication).

Bibliography

References

Cambon J. (2007). Vers une nouvelle méthodologie de mesure de la performance des systèmes de management de la sante-sécurité au travail. Doctoral Thesis. Mines ParisTech.

Hollnagel E., Leveson N., and Woods D. (2006). Resilience Engineering: Concepts and Precepts.

Juglaret F., Guarnieri F., Garbolino E., Rallo J-M, Textoris R., (2011). The Contribution of Balanced Scorecards to the Management of Occupational Health and Safety, ESREL 2011.

Kaplan R., and Norton D. (1996). The Balanced Scorecard -Measures that Drive Performance, Harvard Business School.

Lorino P. (2003). Méthodes et pratiques de la performance, 3ème édition, Editions d'organisation.

Notes

1 Mines ParisTech / Preventeo, Sophia Antipolis, France. frederic.juglaret@preventeo.com

2 Preventeo, Le Cannet, France. Jean-Marc.Rallo@preventeo.com

3 L’Oreal, Aulnay sous bois, France . rTextoris@fr.loreal.com

4 Mines ParisTech, Sophia Antipolis, France. Franck.Guarnieri@mines-paristech.fr

5 Mines ParisTech, Sophia Antipolis, France. Emmanuel.garbolino@mines-paristech.fr

List of illustrations

Title Table 1. Main safety management sub-processes (Cambon, 2007) and their contribution to the requirements of resilience engineering.
URL http://books.openedition.org/pressesmines/docannexe/image/1015/img-1.jpg
File image/jpeg, 212k
Caption Figure 1. Leading indicators on OHS sub-processes integrated into an OHS management model (adapted from Hollnagel, 2006).
URL http://books.openedition.org/pressesmines/docannexe/image/1015/img-2.jpg
File image/jpeg, 188k
Caption Figure 2. Regulatory compliance management model.
URL http://books.openedition.org/pressesmines/docannexe/image/1015/img-3.jpg
File image/jpeg, 240k
Caption Figure 3. Regulatory compliance evolution
URL http://books.openedition.org/pressesmines/docannexe/image/1015/img-4.jpg
File image/jpeg, 252k
Caption Figure 4. Regulatory compliance distribution according to category of hazard
URL http://books.openedition.org/pressesmines/docannexe/image/1015/img-5.jpg
File image/jpeg, 208k
Caption Figure 5. Level of regulatory compliance according to prevention measure
URL http://books.openedition.org/pressesmines/docannexe/image/1015/img-6.jpg
File image/jpeg, 295k

The text and other elements (illustrations, imported files) may be used under OpenEdition Books License, unless otherwise stated.

Search OpenEdition Search

You will be redirected to OpenEdition Search