Desktop versionMobile version

The DARPA Model for Transformative Technologies

 | 
William Boone Bonvillian
, 
Patrick Windham
, 
Richard Van Atta

Part I. Perspectives on DARPA

9. Some Questions about the DARPA Model

Patrick Windham

Full text

  • 1 The questions listed here about apparent paradoxes in the DARPA model were first raised in the fall (...)

1Often observers of DARPA ask basic questions about how the agency operates and the role it plays within the U.S. Department of Defense. This chapter provides brief answers to some of these questions.1

2Is decision-making at DARPA “top-down” or “bottom-up”? DARPA is a mix of the two, but mostly “bottom-up”. The agency director and deputy director do identify broad technical areas that they and others in the Defense Department think are important, but program managers, in consultation with the broader technical community, propose and then run specific R & D programs. In the “systems offices” at DARPA, office directors and the agency director talk with DOD officials and identify what they believe are significant long-term technological challenges and opportunities for U.S. national security. But again, the program managers propose and then run the actual R & D programs.

3How can DARPA respond to Defense Department needs but still have great autonomy? DARPA asks both senior defense officials and the broad technical community what challenges and opportunities they see in the decades ahead. However, DARPA’s job is to think about and create long-term technologies. It is not responsible for developing, maintaining, and improving current military systems; other parts of DOD perform those duties. In this way, DARPA has the freedom and funding to identify and create new, long-term technologies.

4However, in time of war, senior DOD officials may ask the agency to help solve some difficult and immediate technical problems. For example, during the wars in Afghanistan and Iraq DARPA worked with other DOD agencies on the problem of detecting roadside bombs ( “improvised explosive devices”) and also helped to improve communications in those war zones.

  • 2 Bonvillian, W. B., and Van Atta, R. (2011). “ARPA-E and DARPA: Applying the DARPA Model to Energy I (...)

5How can DARPA make long-term progress with new technologies when the agency’s programs are only three to five years long? William B. Bonvillian and Richard Van Atta point out that DARPA has “multigenerational programs”: if the results of an initial program are promising, then there can be follow-on work. But if the initial program fails or points in a different direction, then the program is terminated or redirected. The use of three- to five-year projects allows great flexibility.2

6Why does DARPA sometimes fund several different research projects within a single program? While some programs will fund a single large R & D project, such as the development of a prototype military system, other programs fund multiple research projects performed by different research teams. There are at least two reasons for multiple awards within a single program.

7First, when trying to develop a new basic technology the agency often funds multiple teams with different technical approaches, to see which approaches are most promising. This is a “portfolio policy,” in which the agency funds multiple ideas and then learns which work and which do not. Moreover, funding different teams with different ideas also allows the research teams to learn from each other, further advancing the overall technology. For this reason, a program manager may organize periodic meetings of a program’s various R & D performers and ask these researchers to share information and learn from each other.

8Second, in some cases the development of a new technology or capability requires several complementary parts. For example, the desired technology might need several hardware components plus associated software. In these cases, a program might fund several R & D teams, with each of them responsible for an important part of the overall effort. If the technology proves promising, then that program or a follow-on program might fund work on additional steps that go beyond the initial R & D work, such as the integration of components or applications or the demonstration of the new technology’s applications.

9How do DARPA programs maintain continuity and success when program managers change every few years? New program managers have responsibility for existing programs and then make their own judgments about whether and how to continue them.

10How can an agency build political support when it will not generate significant new technologies until many years from now? In the U.S., this can be a problem for several reasons: political leaders often want relatively quick results, applicants who do not get grants can complain to Congress, and other agencies or parts of your own department may see your agency as a rival. DARPA succeeds because it has an important defense role, it has a record of successes, and it does not threaten the budgets of other R & D agencies.

11The new Advanced Research Projects Agency—Energy (ARPA-E) has thus far built important political support. It has done so by investing in a range of areas that people care about, by having credible processes, by soliciting views from everyone, by being transparent, by helping even losing applicants with valuable advice, and by working hard to convince other parts of the Department of Energy (DOE) that it is a good partner, not a rival.

12How can a DARPA-type agency or program avoid rigid internal bureaucratic processes? Van Atta and others emphasize the importance of a very “lean” management structure. At DARPA a program manager needs approval from only two levels to get a new program: his/her office director (and deputy) and the agency director (and deputy). In addition, DARPA does not have a separate evaluation or audit office; evaluation is a constant process of judging which programs and R & D projects within those programs are succeeding or not. Program managers are not required to spend a great deal of time reporting to an audit unit.

13Related, how can an agency demonstrate accountability (and create the political credibility that it needs to survive politically) but still be relatively free of outside bureaucratic processes such as committees, layers of approval, audits, and so forth? Senior government groups oversee (supervise) all U.S. government agencies, including DARPA. For DARPA, these groups include senior DOD officials, the DOD Office of the Inspector General, the President’s Office of Management and Budget (OMB), DOD and Congressional audit agencies, and of course the U.S. Congress.

14However, DARPA has successfully argued that it does good work, that it follows all government rules, that it is has good internal evaluation processes, and that it needs autonomy and freedom from bureaucratic processes in order to do its job well. These arguments have largely succeeded, and neither senior DOD officials nor Congressional committees try to manage the details of the agency’s work. In the long run, DARPA’s successes and lack of scandals help it convince Congress and senior administration officials that it is doing a good job and does not need intensive bureaucratic supervision.

15Specifically, how can an agency have credible rigorous evaluation of projects without highly bureaucratic and time-consuming reviews? This is a very important question, because evaluation is important not only to the effectiveness of the agency but also to its political credibility, since any agency that does not carry out proper evaluation and maintain high quality will eventually lose political support.

16DARPA’s process for evaluating programs and R & D projects within those programs differs from other U.S. Government science and technology agencies. Some other agencies use formal evaluation groups, which examine projects and provide useful information on the quality of research and how to improve operations. The former Advanced Technology Program/Technology Innovation Program at the U.S. Department of Commerce had a highly respected evaluation unit. Other agencies, such as the National Science Foundation (NSF) and the National Institutes of Health (NIH), maintain quality through both rigorous competition among applicants and the use of peer review as part of their overall merit review processes. At these agencies, the review processes include examinations of whether those applying for new grants have done good work in the past and therefore are likely to do good work in the future.

  • 3 Dugan, R. E., and Gabriel, K. J. (2013). “‘Special Forces’ Innovation: How DARPA Attacks Problems”, (...)

17 DARPA is different. It expects a great deal of its R & D performers, but it also expects that these high-risk R & D projects will not always work as originally planned. Things will “go wrong”. Some DARPA directors therefore will not judge projects in terms of the original and sometimes unrealistic milestones.3 Other directors have required that program managers get formal approval to change milestones and metrics. In both cases, however, the problems that arise provide important information that contributes to learning and adaptation. Surprise and change are normal. DARPA program managers therefore help R & D performers learn from problems and adjust research projects.

18In this world, evaluation is a constant process, done by program managers and their office directors. Based on this ongoing process of learning, DARPA program managers try to help their R & D performers and discuss changes in projects. However—and this is very important— if a specific R & D project, or even an entire program of projects, fails to produce results, then DARPA will stop this work and move money into other, more promising areas. In addition, every year the agency formally reviews all of its programs. In addition to working with R & D performers, DARPA officials routinely talk with senior Defense Department civilian officials, with leaders of the military services, and sometimes also with leaders of DOD laboratories and research agencies to get their views on the usefulness and quality of the agency’s programs. Here, too, the agency engages in continuous process of communication and evaluation.

19This process of “continuous evaluation” works for several reasons: DARPA program managers are technical experts who can both help R & D performers and judge whether the performers are making acceptable technical progress or not; office directors and the agency’s directors and deputy directors are themselves technical experts who can judge results and are willing to terminate unproductive programs; the agency emphasizes the importance of learning; and both senior Defense Department officials and members of the U.S. Congress see that DARPA does high-quality work.

20 How does DARPA survey and analyze needs, technological trends, and future developments? Does DARPA use think tanks or consultants? DARPA has two major processes for gathering information.

21First, program managers talk extensively with scientists and engineers in their fields, understanding technology challenges and opportunities. For example, program managers will talk with university scientists, corporate researchers, and experts in government laboratories to understand technology trends and possible future developments.

22Second, program managers and DARPA leaders talk extensively with military officers and leading experts to understand what long-term needs the Defense Department might have and what types of technical solutions might help. These conversations take several forms: informal conversations with military officers assigned to DARPA, frequent conversations between DARPA leaders and the top civilian officials in the Defense Department, meetings every three months or so between DARPA leaders and senior military officials, “study groups” that meet regularly over several months to discuss a topic, interactions with the Defense Science Board (DSB) and other high-level advisory groups, and, in some cases, formal studies conducted by outside analysts and think tanks, such as the Institute for Defense Analyses (IDA).

23One important issue for DARPA is whether or not other DOD agencies or think tanks have already done a good job of analyzing needs, trends, and opportunities in particular areas of technology or national security. If other agencies conduct useful analyses of, for example, space technologies and needs, then DARPA can use that information. But in some areas no one else has considered which types of new technologies might solve long-term challenges. In these cases, DARPA needs to organize its own meetings with military officers and others and conduct its own analyses.

24How does DARPA recruit program managers? And how specific or broad is the subject area that DARPA presents when recruiting program managers? Because program managers usually serve for only a few years, DARPA’s office directors and the agency director and deputy director spend much of their time recruiting new program managers. In some cases, departing program managers will recommend people to replace them. In other cases, office directors and the agency director and deputy director will ask colleagues in the technical community for recommendations. DARPA officials will look for candidates who are technically strong, have a good vision of where technology might go in the future, and have strong leadership skills.

25These informal recruitment processes work well because the office directors and agency heads are themselves technically-trained individuals who know the R & D community well and can effectively judge the technical qualifications of potential program managers. They also understand what leadership skills are needed.

26Recruiting prospective employees can sometimes be difficult, for both professional and personal reasons. University professors usually are not required to give up their current jobs, since they can take “leaves of absence”, but they may worry about leaving graduate students or interrupting their own research. Company people face other concerns. DARPA usually requires that company employees leave their jobs before being eligible to join the agency, which is difficult even if they know that they will probably get a good job when they return to the corporate world. In addition, government salaries in the United States are much lower than corporate salaries.

  • 4 Van Atta, R. (2013). Innovation and the DARPA Model in a World of Globalized Technology. Presentati (...)

27In turn, both university and corporate people may also have personal concerns about joining DARPA. People with school-age children and whose spouses have careers may be reluctant to move to the Washington, DC, area for several years. Some people decide to come to DARPA while their families stay at home, leading to weekly commutes, but not every prospective employee wants to go through that constant travel. On the other hand, older people who are semi-retired and whose children are grown may find it easier to accept a DARPA position. Van Atta provides additional important insights into why individuals may or may not accept a position at DARPA.4

28A new program manager will work in a specific subject area. He or she will propose and then run new programs in that area, and sometimes may also run existing programs created by earlier program managers. The programs can be quite complex, often involving work that brings together researchers from multiple disciplines. For example, a program that seeks ways to improve how an injured person can use her brain to control artificial arms will involve physicians, neuroscientists, robotics experts, and others. Meanwhile, an effective program manager will need to understand enough about all of these disciplines to design a sensible research program and identify and select competent researchers. In this way, the work of a DARPA program manager can be both specific (focused on specific questions or challenges) and broad (that is, multi-disciplinary).

29When DARPA recruits a university professor, does DARPA allow that professor to continue his/her university research and teaching? Usually professors will temporarily stop their academic research and teaching while serving as DARPA program managers. They need to focus on their DARPA responsibilities—not on their previous activities. Of course, this situation can cause difficulties. A professor may have on-going research projects and a number of graduate students working on their PhD projects. Usually, professors coming to DARPA will ask other professors to handle these responsibilities. But in some cases, they may continue working with existing graduate students, advising them from Washington and also reading drafts of their PhD theses. If a DARPA program manager is also a medical doctor, that person sometimes will be allowed to continue working part-time in a hospital or academic medical center.

30How does DARPA decide about R & D themes, and what do program managers decide about R & D themes? Four points are important.

31First, as mentioned earlier in this book, DARPA has two general activities: (1) maintaining strong leadership in basic technologies and (2) creating and demonstrating new equipment or processes that could help the Defense Department in the future. So, DARPA’s technology offices pay attention to promising new technologies and often also pay particular attention to the long-term challenges facing the military services.

  • 5 Van Atta, R. (2008). “Fifty Years of Innovation and Discovery”, in DARPA, 50 Years of Bridging the (...)

32Second, within these overall subject areas DARPA’s main criterion for selecting R & D themes and programs at any given time is to ask whether the agency can make a significant difference. That is, both program managers and senior agency managers look for game-changing technologies that can contribute to U.S. national security. Selecting themes and programs in any given year is a matter, therefore, of looking at both technological opportunities, and the specific long-term challenges facing the Defense Department at that time. Sometimes, senior Defense Department officials or even the President will direct DARPA to work on specific topics. For example, when ARPA first began work in 1958 it focused on three key presidential priorities: space, missile defense, and nuclear-test detection.5 Usually, however, DARPA officials will talk with senior Defense Department officials about long-term challenges, talk with the technical community about new technical opportunities, and then decide itself which projects offer the most potential.

33Third, DARPA’s priorities do change over time. For example, computer networking was a major priority from the 1960s through the 1980s, and this pioneering work led to the Internet. Subsequently, other agencies and the commercial sector took the lead in building the Internet, and DARPA switched to other opportunities and challenges. It still does work in computing and communications, but now it concentrates on new problems and opportunities, such as cybersecurity and big data. Another example is that, for many years, DARPA did little work in biology; DARPA was a physics and engineering agency. But the combination of bioterror threats, severe brain and other injuries to U.S. soldiers, and exciting new scientific and technical opportunities, has led DARPA to make biology, medicine, and synthetic biology major priorities.

34Fourth, the fact that DARPA has no internal laboratories and instead funds temporary three- to five-year programs gives the agency the flexibility it needs to change themes and programs as new challenges and opportunities arise.

Bibliography

References

Barber Associates, R. (1975). The Advanced Research Projects Agency, 1958–74. Report prepared for the Advanced Projects Research Agency. Springfield, VA: Defense Technical Information Center.

Bonvillian, W. B. (2013). Evolution of U.S. Government Innovation Organization: From the Pipeline Model, to the Connected Model, to the Problem of Political Design. Presentation at the National Graduate Institute for Policy Studies (GRIPS) GRIPS Innovation, Science, and Technology Seminar, Tokyo, April.

Bonvillian, W.B. (2009). “The Connected Science Model for Innovation—The DARPA Model”, in 21st Century Innovation Systems for the U.S. and Japan, ed. S. Nagaoka, M. Kondo, K. Flamm, and C. Wessner. Washington, DC: National Academies Press. 206–37, https://doi.org/10.17226/12194, http://books.nap.edu/openbook.php?record_id=12194&page=206 (Chapter 4 in this volume).

Bonvillian, W. B., and Van Atta, R. (2012). ARPA-E and DARPA: Applying the DARPA Model to Energy Innovation. Presentation at the Information Technology and Innovation Foundation, Washington, DC, February, https://www.itif.org/files/2012-darpa-arpae-bonvillian-vanatta.pdf

Bonvillian, W. B., and Van Atta, R. (2011). “ARPA-E and DARPA: Applying the DARPA Model to Energy Innovation”, The Journal of Technology Transfer, 36: 469–513, https://doi.org/10.1007/s10961-011-9223-x (Chapter 13 in this volume).

Chesbrough, H. (2003). Open Innovation: The New Imperative for Creating and Profiting from Technology. Boston, MA: Harvard Business School Press.

Christensen, C. M. (1997). The Innovator’s Dilemma: When New Technologies Cause Great Firms to Fail. Boston, MA: Harvard Business School Press.

DARPA. (2005). DARPA—Bridging the Gap, Powered by Ideas. Arlington, VA: Defense Advanced Research Projects Agency, http://www.dtic.mil/cgi-bin/GetTRDoc?Location=U2&doc=GetTRDoc.pdf&AD=ADA433949

Dugan, R. E., and Gabriel, K. J. (2013). “‘Special Forces’ Innovation: How DARPA Attacks Problems”, Harvard Business Review 91/10: 74–84.

Heilmeier, G. (1992). “Some Reflections on Innovation and Invention”, Founders Award Lecture, National Academy of Engineering, Washington, DC.

National Research Council. (2013). 21st Century Manufacturing: The Role of the Manufacturing Extension Partnership Program. Washington, DC: The National Academies Press, https://doi.org/10.17226/18448, https://www.nap.edu/catalog/18448/21st-century-manufacturing-the-role-of-the-manufacturing-extension-partnership

National Research Council. (2012). Rising to the Challenge: U.S. Innovation Policy in the Global Economy. Washington, DC: The National Academies Press, https://doi.org/10.17226/13386, https://www.nap.edu/catalog/13386/rising-to-the-challenge-us-innovation-policy-for-the-global

Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics. (2001). “Other Transactions” (OT) Guide for Prototype Projects. Washington, DC: Department of Defense, www.acq.osd.mil/dpap/docs/otguide.doc

Shinohara, K. (2014), “High-Risk & High-Impact Program in Japan: ImPACT”, in Weekly Wire News from East Asia and Pacific, National Science Foundation Tokyo Regional Office, July 4, 2014.

Singer, P. L. (2014). Federally Supported Innovations: 22 Examples of Major Technology Advances That Stem from Federal Research Support. Washington, DC: Information Technology and Innovation Foundation, http://www2.itif.org/2014-federally-supported-innovations.pdf

Van Atta, R. (2013). Innovation and the DARPA Model in a World of Globalized Technology. Presentation at the National Institute of Science and Technology Policy and the Center for Research and Development Strategy, Tokyo, July

Van Atta, R. (2008). “Fifty Years of Innovation and Discovery”, in DARPA, 50 Years of Bridging the Gap, ed. C. Oldham, A.E. Lopez, R. Carpenter, I. Kalhikina, and M. J. Tully. Arlington, VA: DARPA. 20–29, https://issuu.com/faircountmedia/docs/darpa50 (Chapter 2 in this volume).

Notes

1 The questions listed here about apparent paradoxes in the DARPA model were first raised in the fall of 2013 by Hiroyuki Hatada, then Chief Representative of the Washington, DC, office of Japan’s New Energy and Industrial Technology Development Organization (NEDO). The editors are grateful to him for raising these questions and helping us to frame this discussion.

2 Bonvillian, W. B., and Van Atta, R. (2011). “ARPA-E and DARPA: Applying the DARPA Model to Energy Innovation”, The Journal of Technology Transfer 36: 469–513, at 473–74, https://doi.org/10.1007/s10961-011-9223-x (Chapter 13 in this volume).

3 Dugan, R. E., and Gabriel, K. J. (2013). “‘Special Forces’ Innovation: How DARPA Attacks Problems”, Harvard Business Review 91/10: 74–84.

4 Van Atta, R. (2013). Innovation and the DARPA Model in a World of Globalized Technology. Presentation at the National Institute of Science and Technology Policy and the Center for Research and Development Strategy, Tokyo, July.

5 Van Atta, R. (2008). “Fifty Years of Innovation and Discovery”, in DARPA, 50 Years of Bridging the Gap, ed. C. Oldham, A. E. Lopez, R. Carpenter, I. Kalhikina, and M. J. Tully. Arlington, VA: DARPA. 20–29, https://issuu.com/faircountmedia/docs/darpa50 (Chapter 2 in this volume).

Author

Lecturer in the Public Policy Program at Stanford University and a Partner with Technology Policy International, a consulting firm. In the past he has taught at the University of California’s Washington, DC, center and the University of Maryland. From 1984 until 1997 he served as a Senior Professional Staff Member for the Subcommittee on Science, Technology, and Space of the United States Senate’s Committee on Commerce, Science, and Transportation. He helped Senators oversee and draft legislation for several major civilian science and technology agencies and focused particularly on issues of science, technology, and U.S. industrial competitiveness. He has served on five committees and roundtables of the U.S. National Academies. Mr. Windham received a BA from Stanford University and a Master of Public Policy degree from the University of California at Berkeley.

CC-BY-NC-ND-4.0

The text only may be used under licence CC BY-NC-ND 4.0. All other elements (illustrations, imported files) are “All rights reserved”, unless otherwise stated.

Search OpenEdition Search

You will be redirected to OpenEdition Search