1 Introduction

1The increasing interest in resilience engineering (RE) has demanded frameworks for identifying and analyzing sources of resilience (SRs) and their opposite, sources of brittleness (SBs). However, some assessments of resilience have not used any explicit analytical framework (Morel et al., 2009), while others, although they have adopted one, limit themselves to investigating resilience in pre-determined areas of an organization, such as the safety management system (Saurin and Carim Junior, 2011). This study introduces a framework for identifying and analyzing SRs and SBs, which does not restrict boundaries when searching for such sources. The application of the framework is illustrated by a case study of an air taxi carrier in Brazil.

2 The Proposed Framework

2.1 Identifying SRs and SBs

2An SR is any element of the socio-technical system that contributes to the existence of either the proactive or reactive dimensions of resilience. As to SBs, they are any element that is detrimental to resilience. The identification of both the SBs and SRs should start with a description of the socio-technical system, comprising its four sub­systems: technical, social, work system design and external environment. Since defining the boundaries of a system depends on the objectives of the investigation, two questions (namely, what are the objectives? What are the boundaries?) should be answered before starting data collection. The description should give visibility to the gap between the practices laid down and those that are really used, and emphasize the context that explains why informal practices are used. Both the research strategy and the data collection techniques should take a qualitative approach, since this makes it easier (in comparison with quantitative approaches) both to understand the context and to access multiple perspectives, which are key requirements when investigating complex systems.

3Other data should be collected to assess the extent to which RE principles are used. Thus, a protocol should be designed that expands each RE principle upon less abstract characteristics and, for each of them, defines sources of data that could allow them to be assessed. The protocol should be domain specific, using technical language and sources of data that ought to be expected in that context. Although the framework does not assume that any single element of the system should be emphasized, it should be recognized that the definition of the data sources might bias the study. Thus, it is suggested that the protocol be designed after describing the socio-technical system. Thus, areas of particular interest for investigating SBs and SRs may well have already been identified. Based on the data collected in the previous stages, the sources might be identified. The SBs should be identified and stated before the SRs, since the latter are often an adaptive response to the former. Due to both the large volume of data that might have been generated and to the multiple viewpoints from which the data might be interpreted, at least two researchers should codify the sources based on the raw data.

2.2 Analysis of the SBs and SRs

4The sources should be analyzed and classified based on a set of categories. Similarly to what was suggested with regard to identifying the sources, at least two researchers should be involved in such classification. As to the SBs, the categories proposed are:

1. the opposite SR: SBs frequently trigger adaptive responses, at individual or team or organizational level. So, the lack of an opposite SR is undesirable;

2. the risk associated with the SB: the assessment of risks supports the prioritization of the sources, since this is based on assessing the probability of their occurrence and their severity. In order to estimate these parameters, the assumption should be made that they are independent of the adaptive responses;

3. the origins of the sources: on the one hand, it is necessary to assess whether the sources arise either from the formal or informal work organization. SBs arising from the formal work organization are typically associated with failures in documented management routines. Informal SBs are more subtle and they are not clearly connected to any management routine. Both informal and formal sources are equally undesirable, since many formal ones point out the ineffectiveness of the work organization laid down, while many informal SBs point out to interactions that are difficult to control. On the other hand, an assessment should be made of whether the source comes from the internal processes or from the external environment. The existence of external origins is worse, since the organization has a lesser degree of control over them.

5As to SRs, the categories proposed are presented below:

1. the opposite SB: the lack of an opposite SB means both that the SR is not an adaptive response and that it has not had any detrimental systemic impact. However, since the assessment shows the organizational status in a given period, it should not be assumed that the SR will continue to be of this nature in the future;

2. the effectiveness of the SR: this takes into account the shortcomings and new hazards introduced by the source. Three levels of performance are proposed, referred to as high, moderate, and low effectiveness;

3. the origins of the source, adopting the same categories that were applied to assess the origins of the SBs. The highest possible share of the SRs should arise from the formal work organization, since this indicates the availability of resources both for their maintenance and improvement. The existence of informal SRs normally indicates that the adaptive capacity has been used to fill gaps in badly designed systems, rather than being used to improve good designs. As to the other category for assessing the origin, it is better that as few SRs as possible come from the external environment.

3. Research Method

6In order to test the framework, the objective defined for this study was to identify and analyze SBs and SRs in an air taxi carrier in Brazil. Thus, the external environment encompasses everything that, while impacting on the company´s operations, is out of its control (e.g., regulators and ATC). The field study took four months, involving eight visits by a researcher to the company’s facilities – each visit lasted about three hours. The study was preceded by a meeting, in which the research team introduced the objectives and data collection procedures to the directors and staff.

7In line with the framework, the study started by describing the socio-technical system. With reference to the external environment, a review of the literature review on the legal environment shared by all airlines in Brazil was conducted. Thus, the minimum legal technical and managerial requirements that should be complied with by an air taxi carrier were identified. To describe other core elements of the external environment (e.g., political ones) is difficult for airlines, since they are exposed to the environment of all the places in which they operate. As to the social subsystem, workers were characterized according to their marital status, age, job, remuneration schemes, length of employment in the company, academic background and previous professional experiences. As to the technical subsystem, emphasis was placed on the description of the facilities and aircraft. The work organization sub-system was characterized according to: (a) the hierarchical levels; (b) the flight, maintenance and administrative processes; (c) the activities carried out by each employee. Identifying the differences between work laid down and real work was based on multiple sources of data, such as documents (e.g., incident reports), observations (e.g., during real flights) and interviews.

8The next stage of the study emphasized the assessment of RE principles. The four cornerstones of resilience proposed by Hollnagel et al. (2008) were used as a basis. Thus, a data collection protocol was designed, based on breaking down each principle into less abstract characteristics. The sources of data were defined, to a large extent, based on the data collected while characterizing the socio-technical system. That stage of the research pointed out issues that were likely to be associated with SRs and SBs (e.g., procedures) as well as available sources of data that were linked to these issues (e.g., SOPs). The protocol includes 32 recommendations for observations, 60 questions to be used in interviews and 15 recommendations of documents to be consulted. Some of the employees interviewed when the socio-technical system was characterized were interviewed again while the RE principles were being assessed. All interviews were tape-recorded and analyzed based on techniques of content analysis. The researcher who collected the raw data is a commercial pilot and flight instructor. The list of SBs and SRs established by this researcher was refined after discussion with another researcher.

4. Results

4.1 Main characteristics of the socio-technical system

9This company had been operating for forty years and its facilities are located within a hangar, which shelters the aircraft and both the administrative and operational staff. The fleet consists of three airplanes, each of them with a capacity for nine passengers. The workforce is, on average, forty years old. It comprises fifteen people; seven of them are pilots. Four pilots also have administrative jobs, a requirement set out in regulations, such as being responsible for flight safety. This multifunctional characteristic is allowed by regulations and, to a great extent, is a result of the seasonality of the demand for flights. A number of differences between work laid down and real work were detected. For instance, the assignment of pilots to flights often differs from what was planned. This usually happens in order to ensure that all pilots have a similar number of monthly flight hours. This number has a strong impact on pilots’ remuneration, since it has a variable portion, which is dependent on the number of hours flown. The high turnover rate of pilots is another characteristic of this company. The typical length of a pilot´s stay in the company is from six months to one year. According to the interviews, pilots are usually interested in working for this company as a means to accumulate hours of flying-time and, therefore, increasing the possibility of their being hired by major commercial airlines.

4.2 Sources of brittleness

10One out of the thirteen SBs identified in this company (Table 1) was chosen to illustrate the application of the framework. It concerns both the obsolescence and low reliability of the airplanes´ navigation instruments, which is a particularly serious drawback when pilots do not have any visual references. In such situations, the crew divides its attention span between the instruments and a GPS device, which was added to the control panel in order to partially compensate for the unreliable instruments. However, the pilots reported that, sometimes, they forget to check the GPS and, as a result, the flight paths differ from what was planned. This SB is known to all staff and top management. However, according to the director, there are no plans to substitute the instruments, due to the costs involved – this will happen only when new airplanes are acquired. Thus, the origin of this SB was classified as internal and formal, since the processes for equipment acquisition and improvement have not been brought into effect. Although none of the interviewees were able to pinpoint when this SB began, all of them reported that these problems existed in the three airplanes, there being slight variations in terms of intensity. Thus, this SB was regarded as of high probability and of high severity.

Table 1. An extract of how the SBs were categorized

4.3 Sources of resilience

11Ten SRs were identified in this company. The one chosen to illustrate the application of the framework is the SR opposite the SB presented in the previous item (Table 2). Of course, the use of the GPS has a number of limitations as an adaptive response, such as: the control panel was not designed to hold a GPS, and therefore its location was improvised in a place that does allow perfect visibility; as the GPS does not present all the information displayed on the navigation instruments, the pilots still need to check them. Attenuating the overload generated by monitoring two devices is the fact that the fairly low speed of the aircraft reduces the time pressure on decision-making. As to the origin, this SR was regarded as internal and formal, as all staff had reached a consensus that the GPS could be used.

Table 2. An extract of how the SRs were categorized

5. Discussions and conclusions

5.1 Overall assessment of the company investigated

12Table 3 summarizes the results of the assessment. In this table, some sources were identified by the initial “S”, which refers to those that are associated with both resilience and brittleness. The arrows in the lower right corner represent the SBs that do not have a corresponding SR. These SBs exert a pressure which is detrimental to all the other sources, as they push them to the zone of high risk and low effectiveness. The different sizes of the arrows show the relative intensity of the risks. Thus, SB11 has a lower risk than SB12 and SB13. Some pieces of evidence indicated that safety had substantially deteriorated in the company investigated, such as the high percentage of SRs with low effectiveness (60 %) and the fact that there was no SR without a corresponding SB.

Table 3. Summary of the assessment

13In line with these conclusions, there was a serious accident towards the end of the field study. A key event in this accident was the decision made by the pilot (he was also the general-director), during a stop-over, not to refuel the aircraft. According to the data obtained, this decision was made because of the high fuel prices in that location, which underscores the manifestation of a SB referred to as production pressures from top management. As a result of that decision, in the next leg of the flight, one of the engines stopped due to a lack of fuel. At this moment, the lack of SOPs (another SB) made it impossible to check what the rules to be applied in this situation should be. Also, in the final minutes of the flight, there was a discussion between the pilot and the co-pilot, who were arguing about the proper emergency procedures to be adopted. Apparently, the pilot was not aware of the procedures, since, due to difficulties in his inter-personal relationships with his colleagues (another SB), he did not attend the informal meetings in which crew members exchanged information on procedures (a SR). It is also worth noting that this company went bankrupt a few months after this accident.

14Another important result was that none of the SRs were classified as highly effective. This means that the adjustments of performance can also be brittle and, even worse than that, they can introduce new hazards. Although performance adjustment is a core characteristic of resilience (Hollnagel et al., 2008), adjustments do not take place in a vacuum and, due to the explosion of interactions that happen when anything changes in a complex system, it is not reasonable to expect that adjustments will lead only to beneficial interactions. From a broader perspective, this insight indicates that resilience is not intrinsically positive and that, as illustrated by the case study results, knowing the nature of resilience is critical in order to assess how good it is. The objective should be both to bring to the surface and to control the factors that underlie adjustments to brittleness. In this respect, it is worth stressing the importance of identifying whether the sources arise internally or externally. With regard to the external origins (e.g., the lack of slots in the main airport used by this company), the tolerability of brittle adjustments might be higher. Also, most SBs had a formal origin (61.5 %) and most SRs had an informal one (80 %). This is consistent with previous studies that stress the informal behavior of employees at the sharp end as being essential to ensuring badly designed systems are kept operational.

5.2 Assessment of the framework

15The framework brought to the surface informal practices that had been incorporated into the routine and that, because they helped to keep the system working, employees regarded either as being intrinsically good or accepted with resignation. The proposed SBs and SRs analytical categories were shown to support the understanding of the impacts of these practices. It also points out which sources (i.e., the high risk SBs and the low effectiveness SRs) could be prioritized for monitoring.

16Some drawbacks of the framework were highlighted during the field study. For example, the identification of the sources as discrete entities creates the need to anticipate the nature of their interactions. A practical measure to deal with this issue could be to give as much visibility as possible to each individual source, either by formal or informal means. The interactions seem to be more detrimental when several SBs and ineffective SRs occur at the same time. For example, in the company investigated, simultaneity might mean undertaking a flight using the GPS as an improvised back up, with the autopilot out of service, while bound for an airport with poor infrastructure, and for which the decision to take-off was made by pilots who were not aware of the tacit rules that allowed unfavorable meteorological conditions to be anticipated. Although each of these SBs does not normally interact with the others, the system dynamics might trigger undesired and unforeseen interactions, thus changing loosely-coupled interactions into tightly-coupled ones. Moreover, the small size of the company investigated tends to favor such couplings being tight.

17The definition of the level of abstraction of the sources has also emerged from the field studies as a relevant issue concerning the limitations of the framework. Although the word “source” has been used, there was no tracking of the deeper organizational origins of either the SBs or the SRs. In this study, the abstraction level of the sources is mostly the one that emerged from employees’ reports and, therefore, it is at this level that the sources are likely to be realized as a daily routine – thus, this is the level at which they could be monitored. Due to the dynamic nature of the sources, it is more appropriate to say that for the time being they are in a given classification, rather than saying that they belong to it. Another possible perspective is that the stated abstraction level represents the status of each source during the assessment period. If the assessment was repeated in the future, it is likely that many sources would be realized differently and at another level of abstraction. The framework is also limited in terms of completeness, as the sources identified should be regarded as a portion of all existing SBs and SRs.