1 Introduction

1A command and control (C2) system is an essential cognitive aid that military operating forces can use to manage their resources. U.S. Navy expeditionary forces are the smaller operational units that are deployed in combination with ships, aircraft and submarines and must be highly adaptive to meet mission requirements that vary widely in type, scale, and location. C2 systems are essential to expeditionary forces’ ability to adapt to uncertain, complex, and rapidly evolving missions and work settings. A C2 system can serve as a source of resilience (Hollnagel, Woods and Leveson, 2006) for these forces by providing a common operating picture that is available to each level in the command hierarchy.

2Our project is the first to use cognitive systems engineering (CSE) (Woods and Roth, 1988) to develop a human-centered C2 information system for the Navy. We have done this by developing system requirements and the C2 information display based on data collected in observations and interviews, as well as documenting the process in a guidebook for use by senior managers to mid-level engineers (Dominguez, et al, accepted). In this paper, we describe some of the sources of brittleness for deployed expeditionary forces, the role of cognitive aiding to overcome them, the process our research team used to develop and evaluate the C2 system protyotype for expeditionary forces, and the guide we produced for program managers to integrate human-centered design into Navy small scale systems acquisition.

2 Naval Expeditionary combat command

3Created five years ago, the Naval Expeditionary Combat Command (NECC) was formed in response to the need for force protection. Any portion of NECC’s eleven different commands can be assembled into what is termed a “bolt-on” force and deployed to support Navy and Marine elements in missions that range from Iraq combat operations to Haiti earthquake relief humanitarian assistance. Missions that these commands perform necessarily involve complex operations and are conducted in uncertain settings. Our research has studied two of the NECC commands with the highest operations tempo and greatest need for real-time coordination: the Maritime Expeditionary Security Force (MESF) that protects Naval and civilian vessels coastal waters and the Riverine Force that maintains the security of inland waterways.

4As Figure 1 shows, patrol leaders, mid-level supervisors, and senior commanders act in concert with allied and coalition forces. At the lowest level of the command structure, crews aboard boats conduct operations along coasts and harbors (MESF) as well as rivers (Riverine Force). At the next senior level, a tactical watch staff manages boat operations and coordinates their employment. Terrain effects on radio signals may impede radio communications between the tactical operations center (TOC) and Riverine boats, and a mobile TOC may be assigned to relay communications. At the level above the watch staff, the senior watch staff oversees the tactical watch and handles roles such as planning, communications support, logistics and coordination with the next senior command echelon.

5The work that these expeditionary forces perform at each level includes the all of the macrocognitive (Crandall, Klein and Hoffman, 2006) functions:

• Naturalistic decision making—Reliance on experience to identify plausible courses of action that can be evaluated through mental simulation

• Sensemaking and situation assessment—Diagnosis of how things came to be the way they are and anticipation of how it will develop in the future

• Planning—Modifying action in order to change a current state into a desired state

• Replanning/adaptation—Changing or replacing a plan that is already in progress

• Problem detection—Recognition of anomalies in time to avoid or deflect consequences

• Coordination—How team members sequence actions to perform a task. The complexity of macrocognitive activities and the work domain requires cognitive aiding to support individual and team performance.

2 Research Design

6Development of a valid C2 system to support cognitive functions requires understanding the actual work operators perform and analyses to distill data into decision requirements and display requirements that can guide display concept development and evaluation. The research team followed a five phase process to perform human-centered design in the context of Navy small scale systems acquisition.

2.1 Preparation and Framing

7The research team read a range of materials related to NECC planning and operations including Concept of Operations (ConOps), system and operational views, and manuals, then conducted five in-depth interviews with subject matter experts (SMEs) who are familiar with both NECC requirements and operations. Based on results from this phase, we developed a guide for interviews to be conducted during the next phase.

2.2 Data Collection/Knowledge elicitation

8The team conducted 43 Cognitive Task Analysis interviews in with officers and enlisted personnel from MESF (San Diego, CA and Norfolk, VA) and Riverine Forces (Norfolk, VA) who had been deployed to either Kuwait, Iraq, United Arab Emirates, or to Haiti. Interviews sought to learn about cognitive work performed at each level of the hierarchy in Figure 1. Team members also observed Exercise TRIDENT WARRIOR in San Diego to learn about current and newly-developed systems.

2.3 Analysis and Representation

9Team members performed a thematic survey of each interview to decompose them according to cognitive work required to perform missions, then detect patterns across the data and identified decision requirements, as well as requirements for mission performance that could be used to evaluate the prototype once it was developed.

2.4 Concept Development

10The team created hard copy images of display concepts for exploratory usability assessment (Nemeth, 2004) that would be conducted with NECC SMEs to verify initial findings and display concepts. After the assessment session, the team created an interactive software display using the Raytheon JEC3 software development platform.

2.5 Evaluation

11Studies such as this require some way to demonstrate how the use of such a system contributes to resilience. Our research included a Cognitive Wall Walk to assess initial prototype concepts with SMEs from both MESF and Riverine Forces. Team members led the experts through scenarios that had been developed from CTA interview data and presented screen shots of the C2 display for their comments. We also used evaluation criteria from the CTA data to conduct prototype field evaluations with officers and crew members at MESF and Riverine commands. Results of both Cognitive Wall Walk and field evaluations were used to refine C2 display concepts prior to delivery.

12Additional means are available to evaluate such displays. For example, cognitive indicators (Brown, Kosnick, Cox 2007) can be used to determine how well an aid such as a command and control information system supports operator work. Awareness of essential information in the work setting can also be tested in laboratory simulations. For example, a lab experiment can be used to introduce information and determine the time it takes for operators to become aware of it and for others in the command hierarchy to make adaptive changes such as assigning and reassigning boats.

3 Findings

3.1 Sources of brittleness

13The Navy’s current acquisition system accounts for task-level activities such as contact tracking, but not for the macrocognitive activities described in Section 2. Decision support relies on understanding these more complex individual and team behaviors. The lack of human-centered design in the systems acquisition process can itself be considered a source of brittleness.

14Our research into NECC needs has shown that various portions of the NECC command structure include circumstances, that might be considered sources of brittleness which a common C2 operating picture can benefit. Some of the barriers that stand in the way of performing necessary cognitive work include: communications that can be impeded by terrain, disruptions to information sharing, suboptimal cognitive workload, tools and equipment that are a poor match for mission performance, organizational issues such as lack of standardization, and training issues such as the need to cross-train individuals in more than one job.

15The following section briefly describes how a real-time picture of operations that each node in the command hierarchy can see could overcome communications disruptions.

3.2 C2 System Requirements

16Information requirements our research revealed key cognitive functions that need the kind of support a C2 system can provide:

• Understand the Operational Picture

• Develop, Maintain, Repair Common ground across the C2 chain

• Share Information

• Detect and Assess Potential Threats

• Anticipate and Plan

• Manage Communications

• Detect and Solve Problems

17Table 1 shows the link from data through analyses to solution concept using the first information requirement—the need to understand the operational picture—as an example.

Table 1: Cognitive Demand to Display Solution Content

18Understanding the operational picture is individual activity to build and maintain an accurate sense of the situation at hand. It is a sensemaking activity that involves tracking, filtering, and integrating information from multiple sources, and constantly updating one’s understanding.

19Operators at each C2 node need an accurate, real-time operational picture that integrates information from both patrol boat and maritime operations center (MOC) staff watch perspectives. Without it, patrols can be distracted from the tactical situation. Multiple relays required to transmit information by voice from node to node can omit or distort information. Threats may go undetected or be detected too late and the force’s ability to engage a threat may be hampered if there is insufficient time to react.

20Information needed for an accurate, integrated, real-time contact picture includes the status and capability of assets such as boats, potential threats, and schedules and movement of other craft. These elements can be developed into display prototypes and evaluated using criteria fron the data collection and analysis phases. Evaluation results will indicate how well the sources of brittleness have been addressed.

21The data collection phase also revealed other requirements beyond information needs that would also support cognitive demands and overcome gaps and barriers. These included technology/architecture, training, and organizational/operational requirements.

4 Discussion

4.1 Design for an Envisioned World

22As a new system, the NECC problem is in a work context that is being substantially changed by the introduction of new technology. This is designing for an “envisioned world” (Hoffman, et al, 2010). Designing technology to fit the cognitive demands in a work setting that has yet to be created presents a number of challenges for a research, design, and development project such as ours. Envisioned world studies explore how people will operate in their world and how to support the way the world is expected to work. Analyses of operators and their cognitive work that they perform in current circumstances can be used to develop hypotheses about ways to improve performance. The hypotheses can then be embodied in design prototypes and used to discover additional support requirements. New technologies, then, can serve as hypotheses about the effects of interventions on macrocognitive work patterns (Woods and Dekker, 2001).

4.2 The Ability to Adapt

23Results of this project are intended to contribute to Navy adaptive capacity in two ways. The first and most evident way is to enable each level in deployed expeditionary force command hierarchy (Figure 1) to share a common view of operations. Shared awareness of, and ability to reliably communicate about, the capabilities of one’s own force and threats enables the expeditionary force to adjust and continue operations in the face of unexpected challenges.

24The second and less evident benefit is to integrate human-centered design into Navy systems acquisition. By accounting for human performance of actual work, future systems that are acquired are more likely to reflect and support macrocognitive functions that we described in Section 2.

5 Conclusion

25Our approach demonstrates how the Navy can incorporate human-centered research and design in the acquisition of smaller scale systems. The result is expected to contribute to resilience in the Navy’s expeditionary forces, as well as in the Navy’s system acquisition process.