Program Value: What Can We Learn From Major Defense Programs?

Regular Paper Exploring the Concept of Value Creation in Program Planning and Systems Engineering Processes Peerasit Patanakul1 and Aaron Shenhar2 * 1 School of Technology Management, Stevens Institute of Technology, Hoboken, NJ 07030 Rutgers Business School, 180 University Avenue, Room 311, Newark, NJ 07102 2 VALUE CREATION IN PROGRAM PLANNING AND SE PROCESSES Received 16 July 2007; Revised 13 March 2008; Accepted 10 June 2009, after one or more revisions Published online 20 October 2009 in Wiley Online Library (wileyonlinelibrary.com) DOI 10.1002/sys.20155 ABSTRACT Almost every large system is initiated with an intention of building something, which has new value to its stakeholders. Yet, turning this simple statement into reality is often problematic since the concept of value is not always explicitly defined or structually incorporated in the formal frameworks of program planning and systems engineering. Quite often, beyond traditional financial measures or technical and operational requirements, there is little agreement of what program value really means, and there are limited theoretical models that could help conceptualize and focus its essence. In this exploratory study, we investigated the components of program value in several major defense programs that were nominated for Aviation Week’s Program Excellence Award. Among other things, these programs had to declare what value their product created. Our findings resulted in an initial framework, which identifies program value according to three major groups: value to the customer, value to the performing organization, and value to the team. The specific value components identified in our data suggest that program value could be formally and explicitly integrated into program planning and systems engineering processes. It can also lead to further research and new applications that will be focused on value creation, rather than just meeting performance and technical specifications. © 2009 Wiley Periodicals, Inc. Syst Eng 13: 340–352, 2010 Key words: defense programs; program value; value management; program management; systems engineering; strategic program management 1. INTRODUCTION almost all projects or programs are initiated with the intention of creating something that has new value for stakeholders. The value could be tangible such as financial gain or better operations, or intangible such as the creation of new knowledge. However, in a rather paradoxical way, beyond financial measures, there is limited agreement of what does value really mean—how to articulate it, measure it, or manage the process of its creation. This gap exists both at the conceptual level as well as the practical level. On one hand, very little has been written in the research literature on how to define and study value that is created by a program, and there are no accepted The defense and aerospace industry has traditionally been the home of large programs, which help companies translate their strategies into growth and profitability and serve the longterm needs of their customers. It goes without saying that *Author to whom all correspondence should be addressed (e-mail: [email protected]; [email protected]). Systems Engineering Vol 13, No. 4, 2010 © 2009 Wiley Periodicals, Inc. 340 VALUE CREATION IN PROGRAM PLANNING AND SE PROCESSES theoretical frameworks that could guide such research in the future. On the other hand, in spite of broad statements in acquisition guidelines, it seems that program stakeholders do not always provide an explicit statement of the value created by their programs and managers have difficulty defining it to their teams. While programs are typically well defined in terms of expected performance and requirements and even in terms of long-term financial goals, they sometimes overlook the obvious and do not articulate what is the value that stakeholders should expect from such programs. If the expected value is not clearly defined at the outset, it will be more difficult to manage programs toward achieving this value. These questions motivated the current study. Our hypothesis was that the program management and systems engineering disciplines might benefit from a better understanding of the concepts of value both theoretically and empirically. If programs will learn how to include a formal concept of value in their program definition and systems engineering processes, the awareness of teams toward achieving that value will increase and overall end results will better serve the needs of customers, teams, and performing organizations. Thus the objective of this exploratory research was to study the concept of program value. The fundamental research questions were: “What is the value that is created by programs? How do programs articulate such value, and how could such articulations become a standard part of every program’s planning phase?” To achieve this objective, we examined the submissions of several major defense and aerospace programs that were nominated for Aviation Week’s Program Excellence Award [e.g. Aviation Week, 2009]. The award guidelines require programs to define the value they have created, beyond the classical measures of meeting time, budget, and requirements goals, and beyond the common financial objectives to the performing organization. We examined the submissions of 18 programs; searching for consistency and patterns that may guide further and deeper research and eventually help articulate a value component in program definition and systems engineering processes. Our findings identified some lessons, which could lead to a more extensive study in the future. We hope that this process will eventually help create better guidelines on how to explicitly define value ahead of time and how to focus large programs on value creation during the program’s lifecycle. Such guidelines are relevant both for program planning and for systems engineering processes. Note, however, that this study was conducted to investigate the value of aerospace and defense programs of for-profit organizations. Additional research must be done to investigate the value of programs in other industries or notfor-profit organizations. 2. BACKGROUND Typically, the concept and definition of value has been dispersed and dependent on the perspectives of individuals and their disciplines [Ahmed and Yannou, 2003]. Yet, much of the literature of systems engineering has been devoted to the systems engineering process. The process has typically focused on defining the right system requirements and developing the means to achieve them [Sage and Rouse, 2009; Bahill 341 and Henderson, 2004]. As for system value, Blanchard [2004] suggested that a system should be measured in terms of its total value to the consumer, using both technical and economic factors. On the technical side, the value of the system depends on its operating characteristics, broken down into performance and effectiveness. The economic factors focus on benefit (revenues) vs. cost (lifecycle cost). Ridder and Vrijhoef [2007] proposed a dynamic systems engineering approach for adding value. They suggested that, in a construction environment, the system value can be defined in terms of the psychological value, functional value, and technical value. The system cost, on the other hand, is broken down to development costs, operational costs, and maintenance costs. The purpose of this approach is to increase the system benefit (the difference between value and cost) through the dynamic control of the aspect systems, subsystems, and elements. In Space Systems, Nilchiani and Hastings [2006] proposed a six-element framework for measuring the value of flexibility. They claimed that this framework will allow the decisionmakers to measure the value of flexible systems design in its different dimensions through which flexibility in engineering systems can be mapped. Rouse and Boff [2004] and Bodner and Rouse [2007] discussed value in the R&D-related issue. Rouse and Boff [2004] proposed 10 principles for characterizing, assessing, and managing value of value-centered R&D organizations. Later on, Bodner and Rouse [2007] used organizational simulation to investigate the R&D value creation. They focused their study on the issues of (1) how does one attach a value to a particular proposed R&D activity and (2) how does one allocate funding over the various stages comprising an R&D value stream. While these studies did not define directly the program value, their findings were used to guide and enrich our research. In a broader context it would be helpful to look at the project management literature. However, here too, project management research has rarely focused on project value. Some studies [e.g., Kwak and Ibbs, 2000; Ibbs and Reginato, 2002] proposed a model to calculate the financial value (return on investment) of project management. Also, Thiry [1997; 2004a] attempted to link the classical concepts of value management to project management. Many studies can be found on project selection and portfolio management. Most of these studies suggest that projects should be selected based on the benefits they are expected to contribute to the organization’s strategic direction [Pennypacker and Dye, 2002]. Those benefits can be captured in terms of financial benefits, organizational benefits, competitive benefits, etc. Numerous project selection techniques have been proposed [Cabral-Cardoso and Payne, 1996], where resources, operation, and technology limitations of the organization must be considered. Overall, these studies suggested that management should maintain a balanced mix of projects in the organization’s project portfolio [Cooper et al., 1998]. Value creation is clearly connected to an organizational strategy and long-term goals. Thus, several works have indicated that projects and programs should help companies transform their business strategy into action [Bowen et al., 1994; Cleland, 1998]. This realization has focused researchers’ attention on the strategic aspects of project management, studying, for example, the concepts of project Systems Engineering DOI 10.1002/sys 342 PATANAKUL AND SHENHAR strategy, strategy implementation by projects, and alignment of project management with business strategy (Artto et al., 2004; Morris and Jamieson, 2005; Shenhar et al., 2005; Milosevic and Srivannaboon. 2006]. Another relevant area of study deals with the question of project and program success. Traditionally, projects were judged based on time-cost-performance dimensions. However, similar to corporate balanced scorecard studies, some project management researchers have suggested using a compound stakeholder approach to evaluate project success. This approach proposes multiple success dimensions such as project efficiency, benefits to customers, benefits to the performing organization, and preparation for the future [Decotiis and Dyer, 1979; Pinto and Slevin, 1988; Freeman and Beale, 1992; Shenhar et al., 2001]. Clearly, project success dimensions and project expected value are closely related. Once we explicitly define the concepts of project value, they can be used as a basis for assessing and measuring project success. Finally, no discussion of program value is complete without looking at Value Engineering. The Society of American Value Engineers (SAVE) and the European Governing Board of the value management training and certification system (EGB) suggest that “value” can be defined as a ratio between the satisfaction of an explicit or implicit need and the resources invested to achieve it. Value, therefore, can be seen as typical ratios between quality & cost, function & cost, worth & cost, performance & resources, satisfaction of needs & use of resources, and benefit & investment [Thiry, 2004b]. SAVE and EGB also define Value Management (VM) as the combined application of value methodologies and other methodologies at the organizational level in order to improve organization effectiveness. VM includes, for example, the use of processes, tools, and techniques derived from the work of Lawrence Miles [1961] and specific value methodologies such as value analysis (VA) and value engineering (VE). 3. RESEARCH DESCRIPTION The main objective of this research was to investigate the expected value created by programs beyond typical financial measures. Specifically, we looked at the value of major defense programs in for-profit organizations. We also investigated the value of programs of different types, categorized based on different stages of program lifecycles, namely, development, production, or sustainment programs. The complexity of the programs we studied was at the “system level,” according to Shenhar and Dvir [2007]. This enabled us to look for system program values both in planning and systems engineering processes. None of the programs we studied were at the assembly or array (systems of systems) level. Data sources: In 2004 Aviation Week and Space Technology Magazine initiated an Annual Program Excellence Award for the aerospace and defense industry. The goal of the Award program was to create higher standards of excellence in the industry and share lessons learned across companies for better future performance. Each year, major aerospace and defense contractors are encouraged to submit their leading programs Systems Engineering DOI 10.1002/sys as candidates for the Award. About 20 programs are then nominated. Based on a common framework, every nominee submits a report detailing the management of their program. The submissions are then evaluated by an independent team of judges, who score the submissions on different criteria. The evaluation is based on four major criteria areas, divided into several subcriteria. The areas and their relative weights are: Value Creation (10%), Organizational Processes (30%), Addressing Complexity (30%), and Metrics (30%). The information in the reports submitted for the Award is unclassified and it relates only to the managerial aspects of the programs, and not to any propriety, business, or technical data; thus enabling crosslearning by sharing the data across the industry. In addition, nominees are asked to describe any best practices they developed internally, which can be shared with the industry during Aviation Week’s A&D annual conference. Sample: Out of 19 programs that were nominated for the award in 2006, we studied 18 programs from 6 for-profit organizations (see Table I). The only program that was dropped was from a nonprofit organization. Studying programs from different organizations helped ensure the validity of the findings across organizations. Content analysis: Using the nomination reports, we conducted content analysis [Bauer and Gaskell, 2000] to identify what was declared as program value. To extract the most information from each report, we performed content coding into major and secondary groups. Based on such coding, we were able to group the value of programs into three categories—value to the customers, value to the performing organization, and value to the team. We then performed cross-case analysis to finalize the categories of the value elements in each category. Pattern analysis and discriminant analysis: The programs we studied were in different phases in their lifecycle—development, production, and sustainment. This distinction had an impact on the dynamics of program management, especially when the programs were very large and extended over long durations. The typical main activities of development programs involved the creation of a totally new product or system through development, testing, and validating. In production programs, most development was already completed and major activities involved manufacturing work and improvements of production lines. In a program in the sustainment phase, the system was already operational and the main activities of the team were to maintain the system’s operation capabilities and to keep improving them. The only development activities that were performed at this phase related to system enhancement, upgrades, and modernization. Having these data points, we investigated the different value statements of programs in different phases. To do so, we employed pattern analysis based on frequency count to observe any emerging pattern. We also performed some statistical analyses, namely, tests of equality of group means and discriminant analysis. This was done to observe any statistical significant difference among program value of different types. VALUE CREATION IN PROGRAM PLANNING AND SE PROCESSES 343 Table I. Sample Programs Studied 4. RESULTS AND DISCUSSION 4.1. Toward a Definition of Program Value As the literature review shows, although several definitions of value were suggested, there is very little agreement of what program value really means. In this study, we initially defined program value as: The explicit and implicit benefits to the program stakeholders generated from the program’s outcome versus the tangible and intangible resources invested to achieve those benefits. This definition is a formal expression of: “Are you getting value for your money?” or “Are you getting your money’s worth?” Both conceptually and practically, different individuals or organizations may or may not perceive the same value of a program. These perceptions are dependent on the benefits each stakeholder expects to gain and the resources they invest into the program. This definition is similar to the general definition of value, suggested by the Society of American Value Engineers (SAVE) and the European Governing Board of the value management training and certification system (EGB). However, it is interesting to note that this definition is broader than the Department of Defense’s definition of “the capability provided by a new product or service, and the schedule and financial terms allocated to generate that capability.” While both definitions are similar, we believe that value should be seen as more than just “creating capability,” and that benefit can be in a wider range of options—safety, security, or quality of life, to name just a few. 4.2. How Did Program Managers Articulate Their Program Value? Even though we defined the value of programs as mentioned above and used it to frame our research, based on content Systems Engineering DOI 10.1002/sys 344 PATANAKUL AND SHENHAR analysis, we found that the concept of program value has not been fully articulated as such by the nominating programs across the board. When asked to describe the program value, the majority perceived the value in terms of benefits gained from the program outcomes (first half of the definition). Others described value in general terms such as better outcomes or improvements. This observation leads to several arguments. On one hand, it suggests that the definition of program value has not been clearly defined to managers in the program, and it is the reason why some discussed only “the explicit and implicit benefits to the program stakeholders generated from the program.” On the other hand, it also possibly shows that these practitioners focused on the more tangible outcomes of their programs such as meeting requirements. The argument is that value to the user is already articulated in user requirement documents. In addition, many would argue that national defense is extremely costly and benefits such as protecting freedom or saving human lives are hard to quantify in terms of dollar value. This point clearly needs more research. Yet, how important is it for program managers to fully understand the value of their program? Perhaps if one just sticks to initially well-defined requirements, the value will consequentially be created. However, it is well known that “things that don’t get measured don’t get done.” Conceptually, one could argue that the higher the managers’ understanding of their program value, the more it will help them effectively lead their program to achieve it. Furthermore, since systems engineering plays a critical role in any major defense program, it makes sense to ask: How do systems engineers deal with the definition and expected creation of value? From a systems engineering perspective, if the program value is not clearly defined, it may be difficult for a systems engineer to identify the total system value and to distribute its components across the system and/or its development phases. One may also argue that, although the program value was not always well articulated, those programs were among the most successful. Yet, it is difficult to determine the degree of success without the articulation of value, and it is unknown whether program management and systems engineering were executed effectively. Finally, we may also ask: How does the lack of value definition impact the performance of less successful programs? How many programs in progress today are Table II. Program Value to the Customers * Number of programs explicitly or inexplicitly states this benefit. Systems Engineering DOI 10.1002/sys struggling because the program value concept is still not fully understood and not formally incorporated in the planning and systems engineering processes of large programs? While still exploratory, this research suggests that there are still a few open questions, which need a better understanding regarding the role of value in major programs and their systems engineering processes. However, at this point we were able to categorized program value into three groups: value to the customers, value to the performing organization, and value to the team. The following sections discuss each group in more detail. 4.2.1. Value to the Customers The programs we studied were mostly government-contracted programs, having US Army, US Navy, US Air Force, and FAA as their main customers. While not stated explicitly, we found from the content analysis that the value to the customers as reported by these programs can be grouped into consistent subgroups with either near- or long-term benefits (see Table II). The most common value to the customer of these programs was the near-term value of operational excellence (15 out of 18 cases). The program managers’ perception was that with operational excellence, the customers would have an ability to maintain a leadership role in their operation arena. For example, from the GPS program we learned that “developing new capabilities of GPS IIR-M is the first step toward major advancement in the GPS which will assure continue US preeminence as a world leader in Global Navigation Systems.” And for the F-22 program, the program manager reported “the F-22 Raptor is the US Air Force’s air dominance fighter for the next 40 years.” It is important to note, however, that due to its high cost, there is an on-going debate between the Air Force and the DoD about how many planes would finally be produced. (It may be that the Air Force may only be able to order about 120 aircraft, instead of the requested 380, or initial Air Force estimates of 740). This example demonstrates clearly how the resources portion of the value definition put into question the overall benefits from a program. In addition, from the end users’ (e.g., war fighter) standpoint, program managers in our study perceived that having operational excellence means having the ability to effectively, safely, and securely perform day-to-day operations by having VALUE CREATION IN PROGRAM PLANNING AND SE PROCESSES a system with superior performance that will perform better in combat operations and will eventually lead to better mission success. The program manager of GCSS reported, “GCSS will revolutionize the way the US Army performs its logistics operations. It allows commanders on the ground to make tactical decisions using real-time, reliable data.… [T]he program is an integral part of the US Army’s transformation plan to become leaner and more agile, able to respond to situations with significantly decreased lead times.” ARH program reported that “[t]he ARH-70A provides the war fighter an improved net-ready, see first, understand first, act first, and finish decisively capability over the venerable OH-58D [existing aircraft in use].” Another important value is modernization, which may in turn lead as well to operational excellence. The program manager of F21 stated, “It has transformed maneuver C2 operations from using analog voice radio traffic and paper maps with grease pencil markings to using modern digital, computerized, moving map displays and convenient shortform digital messaging.” Finally, customers may also see the long-term benefits of the program as a platform for future effort (e.g., F21 and ERAM). In some of the programs we studied (RSLP, ICBM, and F21); the scope of the programs was providing systems engineering and integration or mission assurance, verification, and validation. Overall, it seems that knowing and explicitly defining the value to the customer could benefit program management and systems engineering in many ways. First, it will help managers and customers align their mutual expectations beyond achieving the formal requirements and specifications. If such alignment does not exist, programs may deliver a system that meets its performance goals but does not satisfy the updated need of the customer. Such situations may happen when dynamic changes in markets or operational arenas have shifted beyond what was originally articulated in system requirements. Second, a clear understanding of those expectations can help program managers develop a program strategy for leading programs to better satisfy their customers. Once again, a clear articulation of value can help both sides to see potential benefits along the way that were not recognized during the program inception and lead to improvements that otherwise would not have been included. Third, the explicit value of the program to the customer should be articulated in the systems engineering process. It will help a systems engineer better understand the customer’s needs and the total system value even before system requirements are written. Taking an earlier viewpoint and including formal value statements in the systems engineering work may enhance the chances of later success. After the needs and the total system value are clearly identified, the later steps in systems engineering process can then be executed more effectively. Finally, most of the programs we studied were DoD-contracted programs and are guided by formal processes and documentation under the Joint Capabilities Integration and Development System (JCIDS) (e.g., CJCSM 3170.01C, updated in May 2007). Most of these guidelines relate to such processes as needs and capabilities analysis, solutions and 345 performance analysis, and capabilities development and production. While the DoD guidelines are mostly process-oriented, they indeed require that analyses will relate to issues such as objectives, scope, scenarios, desired capabilities, functions, and doctrines. These requirements are later translated to system parameters and technical specifications, which form the basis for acquisition decisions and vendor selection. It is most likely that a clear focus on program value would contribute to higher success in addressing the JCIDS. 4.2.2. Value to the Performing Organization Besides value to the customer, programs also generate value to the performing organizations. From an organizational perspective, the value of programs is frequently expressed first in terms of financial measures. There are several financial models available to calculate returns on project/program investment [Loch and Kavadias, 2002; Meredith and Mantel, 2005]. However, it was encouraging to see that, in addition to financial values in terms of revenue and growth, programs indicated that they created other types of value to the performing organization. Once again, these kinds of values appeared to be consistent across programs and related to issues such as market opportunity, strategic positioning, organizational learning, name and brand recognition, and relationship development (see Table III). Even though revenue and growth was recognized as important, market opportunity was mentioned in more cases to be an important value created by programs. Fifteen out of 18 program managers perceived that their program helps the company restore and expand market share in existing markets, exploit opportunities in new market segments, and leverage them to future business opportunities. For example, the program manager of ARH mentioned, “the ARH contract will likely be one of the last manned rotorcraft purchased by the U.S. Army. Winning this contract brings a major former customer, the US Army, back to Bell and ensures another 30+ year relationship.… [T]he ARH program represents the potential sales of up to 1,000 airframes when considering U.S. government purchases and foreign military sales.” For ICBM, the program will lead the expansion to adjacent markets via new client projects. The program manager of GCSS wrote that his program helps the company “increase market share in the logistic arena.” Strategic positioning was also noted as an important value. Eleven out of 18 program managers reported that their program supports organizational business strategy and will put the company in a unique strategic position. For example, “STAR is the world’s only ISR platform in use today which provides real time GMTI data as well as providing real time battle management. Thus, the company is uniquely positioned to exploit this advantage.” The program manager of ICBM wrote, “We are viewed as leaders in supplier quality management and large scale integration.” For the GPS program, “As the developer and supplier of 21 GPS IIR satellites, the Lockheed Martin/ITT team became the premier Navigation satellite provider beginning the mid 1990’s.” Six out of 18 program managers reported organizational learning as an important value to the organization. Organizational learning leads to the improvement of process, people, and products from program learning and the opportunity for Systems Engineering DOI 10.1002/sys 346 PATANAKUL AND SHENHAR Table III. Program Value to the Performing Organization * Number of programs explicitly or inexplicitly states this benefit. knowledge sharing within and across the organizations. “Execution of the IIR-M program has positioned the Lockheed Martin/ITT Team to lead the way for developing even more exciting advancements in GPS capabilities,” as reported by the GPS program manager. Similarly, “Methodologies from ATLAS that are demonstrated to be ‘best practices’ are passed on to both the enterprise and corporate levels and support the core competency and strategic objective of critical knowledge sharing across product lines.” The program manager of ARH reported that the program helps “develop and strengthen several key core competencies such as complex avionics integration.” Name and brand recognition was also mentioned among the values generated by programs. By doing a great job, the program manager of RSLP noted that the company would be recognized in the community, which will enhance the company’s strategic position. “429 helps restore the Bell Helicopter brand name for customer satisfaction after Bell’s less than meaningful participation in the market with 427,” as articulated by the 429 program manager. Finally, we also found that the program can bring value in terms of relationship development with the customers, or enhancing future business opportunity. This finding is not surprising. Clearly, programs could create more value to the organization than just financial. Yet, it was encouraging to see how well some program managers were able to recognize this value, especially strategic positioning. Notably, in previous studies, conducted at the project level, managers had difficulty articulating their project’s contribution to the strategic aspects of the organization [e.g., Morris and Jamieson, 2005]. Such difference may relate in part to the positions and seniority of program managers in our study. The majority of program managers were either Vice Presidents of their business units or senior program managers, Systems Engineering DOI 10.1002/sys leading large and expensive government contracts. During the long process of winning a contract, managers must clearly understand the value and the position that these programs must create for their mother organizations. Conceptually, understanding the value that programs bring to an organization will help programs to be better aligned with the strategic direction of the organization. Furthermore, as the specific value components in Table III show, programs can focus on more than just adding to the strategic alignment. Issues such as recognition, learning, and relationships can also be identified upfront as excepted values and become a focus attention for the team. In sum, if the program manager and the team members understand the impact of their program on the strategic direction of the organization, there is a high possibility that all program management decisions will be made with the organizational benefits in mind. From the organization’s perspective, the program value listed above can also be used as part of the criteria for program prioritization and program success assessment [Cooper and Edget, 1998]. 4.2.3. Value to the Team In terms of program value to the team, we found that besides incentives, program value is perceived in terms of challenge and excitement, career prospect, pride, name and recognition, networking, and user benefits. Even though it was not defined explicitly, our content analysis revealed that value to the customer and value to the performing organization were mentioned very often in different contexts throughout the submission reports. However, value to the team was mentioned less frequently or in some cases was not mentioned at all. Table IV shows the distribution of the value to the team among the sample programs. As reported by program managers, many of the teams perceived that working in these programs is valuable since it VALUE CREATION IN PROGRAM PLANNING AND SE PROCESSES 347 Table IV. Program Value to the Team * Number of programs explicitly or inexplicitly states this benefit. requires “pushing the envelope,” which felt exciting and challenging. For example “DSP’s evolving technology insertion to provide new capability and its expanded mission are a source of continuing inspiration and challenge”. The IFIS program manager noted, “The nature of this product is such that the engineering team always has new and interesting features to work.” Career prospects were also perceived by teams as valuable, particularly when working in the same program for a long time: “Many employees have stayed on DSP for decades. In a few cases, two generations of families have worked on the program.” And some teams felt proud to work on programs where they could easily identify with their program’s mission. “People want to work on a successful program, particularly a program whose mission is so clearly associated with saving the lives of our war fighters in harm’s way,” noted F21 program manager. Or as the DSP program manager wrote, “…[T]heir pride in being part of an extraordinarily successful effort that our war fighters live to tell us makes a huge difference to our nation’s defense.” And in the ARH program, “[T]he team knows they are involved in a very unique program that has great consequences for our nation and an immediate need for our soldiers in harm’s way.” Finally, program managers also mentioned networking, as well as working with people who are professional and knowledgeable help them learn and develop new skills. It seems that understanding the value to the team helps program managers create the right working environment, which may help their teams excel. The explicit and clearly articulated value can be used as a source of motivation and encouragement, helping teams to be more effective. Incidentally, as we have seen, cash incentives were not a major driver of value for the team. In summary, program teams see value in their pride of being part of a significant and challenging program, career opportunity, networking, and recognition gained from working in the program. As we have seen from the former discussion, explicit value creation may become an integral part of program management, and program managers should learn to manage their programs to achieve this value. Similarly, system engineers should focus on value to the customer in order to understand the customers’ needs and emphasize it during requirement analysis and development. In the next section, we will discuss program value at different stages in program life cycle. 4.3. Program Value and Program Life Cycle Phase As mentioned, the programs we studied involved different types in terms of their lifecycle phase—six development programs, four production programs, and seven sustainment programs. We performed frequency analysis (of the specific values that were mentioned for each program type) to investigate the impact of program types on the perceived program value (see Table V). Since one program (USSGW) was already completed at the time of reporting; it was removed from this analysis. Based on program categorization and frequency count, we found that programs in different phases have a similar pattern of value to the customer (e.g., operational excellence—83% for development, 100% for production, and 71% for sustainment; modernization—33% for development, 25% for production, and 29% for sustainment). Not surprisingly, value in terms of platform for future effort was not found in all production programs. Success enhancement is the value that was only found in sustainment programs. Multiple discriminant analysis also indicated that there is no significant difference among the value to the customer of programs in different Systems Engineering DOI 10.1002/sys 348 PATANAKUL AND SHENHAR Table V. Perceived Program Value and Program Type * The percentage of programs, which mentioned this value out of the total number of programs in this phase phases, whereas some difference was noted in value to the organization and to the team as we discuss below (see Table VI, Tests of Equality of Group Means, with significance at 0.1 level or better). A similar pattern was found among three different types of programs in terms of value to the performing organization, (Tables V and VI). One significant note is that, even though name and brand recognition is a value that was noted in most of the development programs (67%), it has not been found as a common value of programs in other types (0% for production and 14% for sustainment). The result indicates a statistically significant difference among three types of programs based on name and brand recognition (Table VI). In addition, when performing stepwise discriminant analysis, name and brand recognition was the first variable entering the discriminant function, indicating that it can be used as a predicting Table VI. Tests of Equality of Group Means (Discriminant Analysis) Systems Engineering DOI 10.1002/sys VALUE CREATION IN PROGRAM PLANNING AND SE PROCESSES variable for categorizing programs of different types. In sum, the results from our statistical analyses indicate that programs in different phases may have different perceptions in terms of value to the performing organization. One value that is different is name and brand recognition. The reason is that new development programs create more public image and excitement than long-range production or sustainment programs that have been around for years. As for the value to the team, the results show no significant difference for all kinds of value except networking and pride (Tables V and VI). Networking was only found among sustainment programs. Once again, this is because sustainment programs have been in service for decades (e.g., DSP, ICBM, and ATLAS exist more than 35, 37, and 50 years, respectively). Working for years in such programs, team members have had the opportunity to accumulate and share knowledge and experiences with many colleagues within and outside of the program. It is surprising, however, that networking was not found as a perceived value in the other types of programs. In terms of pride, the results show 50% in development and production programs and 14% in sustainment programs. And as mentioned, such a difference is a result of the degree of the product newness and excitement the program creates. Overall, when analyzing different program types, we found that all programs created value of three kinds (to the customers, to the performing organization, and to the team) both in the near term and in the long term. Yet, specific values differed along the lifecycle stage of programs. This analysis validates our earlier claim that program value assessment metrics should be developed with specific considerations to program phase. 5. LIMITATIONS This study is not free of limitations. First, on one hand, we must note that the data were not independently or randomly collected. Most likely, program managers made every effort to win the Award. Thus, difficulties or problems were typically not reported in these submissions, and we may not know what really happened. On the other hand, the self-reporting may have forced managers to do their best in thinking and articulating the expected values from their programs, thus contributing to the richness of the data. To remedy this limitation, research in the future should focus on fewer programs but conduct in-depth case studies that may expose deeper shortcomings in studied programs. Second, while several parties were involved in these programs, the study was based on the information provided by program managers who are responsible for executing programs in the aerospace and defense industry. We may expect that other stakeholders will have additional perspectives. For example, customers (who fund these programs) may perceive additional values such as value to society, public image, etc. Interviews with customers and other stakeholders will enrich the data collection process in future research. Third, the small sample of cases (18 programs in six organizations) may suggest that program value identified may be unique to those companies and their industries. Perhaps in other industries, other program values may have different 349 weights than the ones noted in the defense and aerospace industry. Similarly, programs in not-for-profit organizations may indicate other forms of value than those found in our study. More research is clearly needed to refine and expand of the concepts of value in major programs both in for-profit and not-for profit organizations. 6. IMPLICATIONS Even at its preliminary stage, in addition to an explicit updated definition of value, this study confirmed that programs create different forms of value for different stakeholders—value to the customer, value to the performing organization, and value to the team. It also suggests that there is a rich ground for further and deeper conceptual and empirical investigation, as well as to more detailed research questions. For example, is there value that is specific to a unique customer? Does the industry impact the type of value created? Do programs in not-for-profit organizations create different values from those of for profit organizations? Another important aspect is the interdependence among different kinds of values, that is, does one type of value affect the others? Since this study identifies each form of value independently, future study should be conducted to investigate the interrelationship among different forms of value and the way to address those interrelationships in program management. The results of this study are important for practitioners for several reasons. First, the definition of program value helps create a common understanding when it comes to the discussion about value during program initiation and execution. After the expected value is determined, programs should be managed toward achieving it. A program manager should consider creating a program strategy, which balances all forms of value, and all program activities should be linked to the program expected value. A program manager must first deal with the expected value to the customer to help ensure that all program deliverables will meet or exceed the customers’ expectation. Understanding the value to the performing organization helps aligning program management with the business strategy of the organization. And understanding the value to the team helps the program manager create a working environment that supports greater team effectiveness. In addition to program management, the value concept can play a significant role in the activities of systems engineering. Understanding program value should impact the systems engineer’s job from the outset. Together with dealing with system requirements, the systems engineer should focus on the value proposition, which will have an impact on the rest of the SE activities. Value creation should be included in future systems engineering guidelines and frameworks. Once a problem has been defined and potential feasibility established, systems engineers should become aware of the required value forms and articulate those in their formal planning process. While understanding all forms of value is useful, the systems engineer should pay the highest attention to the value to the customer. If this value is clearly defined, the systems engineer can then translate it into better requirements and Systems Engineering DOI 10.1002/sys 350 PATANAKUL AND SHENHAR Figure 1. Program value and the systems engineering process in the life cycle. Adapted from Blanchard [2004]. technical specifications. At every step of the process the systems engineer should test its impact on enhancing the expected value from the system. Identifying and articulating the expected value from a program should become part of the systems engineering process. Figure 1 describes this process as a modification of the traditional process. 7. CONCLUSION Given the limited information about program value in previous literature, this preliminary study was initiated to identify how major defense programs look at the concept of program value. Using data from aerospace and defense companies’ submissions to Aviation Week’s Program Excellence Award, the study looked for consistent patterns of different kinds of value that were described by program directors. Program value was defined as “the explicit and implicit benefits to the program stakeholders generated from the program versus the tangible and intangible resources invested to Systems Engineering DOI 10.1002/sys achieve those benefits.” As seen, the value created by programs can be divided into three different types for different stakeholders—value to the customer, value to the performing organization, and value to the team. Furthermore, value of programs in different phases of their lifecycle, namely, development, production, or sustainment, is perceived differently by program directors. As we concluded, to be more effective, both program management and systems engineering need to formally incorporate the concept of value into their formal processes. And as suggested, the findings from this study can be used to create guidelines, especially when they are applied to different industries, environments, and type of the organizations. ACKNOWLEDGMENT We are grateful to Aviation Week and Space Technology Magazine for initiating the Program Excellence Award Program in the aerospace industry and for providing the data for VALUE CREATION IN PROGRAM PLANNING AND SE PROCESSES this study. In particular to Carole Hedden for her continued support and encouragement of our research efforts. REFERENCES W.B. Ahmed and B. Yannou, Polysemy of values or conflict of interests: A multi-disciplinary analysis, Int J Value-Based Management 16(2) (2003), 153–179. K.A. Artto, P.H. Dietrich, and M.I. Nurminen, Strategy implementation by projects, PMI Res Conf, London, UK, PMI, 2004, 103–122. Aviation Week, Aviation Week Sixth Annual Program Excellence Award (February 24, 2009) http://www.aviationweek.com/ aw/generic/story_channel.jsp?channel=busav&id=news/ AWARD022409.xml. A.T. Bahill and S.J. Henderson, Requirements development, verification, and validation exhibited in famous failure, Syst Eng 8(1) (2004), 1–14. M.W. Bauer and G. Gaskell (Editors), Qualitative researching with text, image, and sound. A practical handbook, Sage, London, 2000. B.S. Blanchard, System engineering management, Wiley, Hoboken, NJ, 2004. D.A. Bodner and W.B. Rouse, Undertanding R&D value creation with organizational simulation, Syst Eng 10(1) (2007), 64–82. H.K. Bowen, K.B. Clark, C.A. Holloway, and S.C. Wheelwright, Development projects: The engine of renewal, Harvard Bus Rev 72(5) (1994), 110–120. C. Cabral-Cardoso and R.L. Payne, Instrumental and supportive use of formal selection methods in R&D project selection, IEEE Trans Eng Management 43(4) (1996), 402–410. D.I. Cleland, Strategic project management, The Project Management Institute Project Management Handbook, Jossey-Bass, San Francisco, CA, 1998, pp. 27–54. R.G. Cooper, S.J. Edgett, and E.J. Kleinschmidt, Best practices for managing R&D portfolios, Res Technol Management 41(4) (1998), 20–33. T.A. Decotiis and L. Dyer, Defining and measuring project performance, Res Management 16 (1979), 17–22. M. Freeman and P. Beale, Measuring project success, Project Management J 1 (1992), 8–17. W. Ibbs and J. Reginato, Quantifying the value of project management, Project Management Institute, Newtown Square, PA, 2002. Y.H. Kwak and C.W. Ibbs, Calculating project management’s return on investment, Project Management J 31(2) (2000), 38–47. 351 C.H. Loch and S. Kavadias, Dynamic portfolio selection of NPD programs using marginal returns, Management Sci 48(10) (2002), 1227–1241. J.R. Meredith and S.J. Mantel, Project management: A managerial approach, Wiley, Hoboken, NJ, 2005. L.D. Miles, Techniques of value analysis and engineering, McGrawHill, New York, 1961. D.Z. Milosevic and S. Srivannaboon, A theoretical framework for aligning project management with business strategy, Project Management J 37(3) (2006), 98–112. P.W.G. Morris and A. Jamieson, Moving from corporate strategy to project strategy, Project Management J 36(4) (2005), 5–19. R. Nilchiani and D.E. Hastings, Measuring the value of flexibility in space systems: A six-element framework, Syst Eng 10(1) (2006), 26–44. J.S. Pennypacker and L.D. Dye, “Project portfolio management and managing multiple projects: Two sides of the same coin?“ Managing Multiple Projects, J.S. Pennypacker and L.D. Dye. Marcel Dekker, New York, 2006, pp. 1–10. J.K. Pinto and D.S. Slevin, Project success: Definitions and measurement techniques, Project Management J 19(3) (1988), 67–73. H.de Ridder and R. Vrijhoef, Dynamic system engineering for adding value in the built environment, Insight 10(1) (2007), 28–34. W.B. Rouse and K.R. Boff, Value-centered R&D organizations: Ten principles for characterizing, assessing, and managing value, Syst Eng 7(2) (2004), 167–185. A.P. Sage and W.B. Rouse, Handbook of systems engineering and management, 2nd edition, Wiley, Hoboken, NJ, 2009. A.J. Shenhar and D. Dvir, Reinventing project management: The diamond approach to successful growth and innovation, Harvard Business School Press, Boston, 2007. A.J. Shenhar, D. Dvir, W. Guth, T. Lechler, P. Patanakul, M. Poli, and J. Stefanovic, Project strategy: The missing link, Annu Meet Acad Management: A New Vision of Management in the 21st Century, Honolulu, HI, 2005. A.J. Shenhar, D. Dvir, O. Levy, and A. Maltz, Project success: A multidimentional strategic concept, Long Range Plan 34 (2001), 699–725. M. Thiry, Value management practice, Project Management Institute, Newtown Square, PA, 1997. M. Thiry, “Program management: A strategic decision management process,” The Wiley guide to managing projects, P. W.G. Morris and J.K. Pinto (Editors), Wiley, Hoboken, NJ, 2004a, pp. 257– 287. M. Thiry, “Value management,” The Wiley guide to managing projects, P.W.G. Morris and J.K. Pinto (Editors), Wiley, Hoboken, NJ, 2004b, pp. 876–902. Peerasit Patanakul holds a B.E. in Chemical Engineering from Chulalongkorn University (Thailand), an M.S. in Engineering Management, and a Ph.D. in Systems Science/Engineering Management from Portland State University (Portland, OR). Currently, he is an Assistant Professor at the School of Technology Management, Stevens Institute of Technology (Hoboken, NJ). Peerasit’s current research interest includes multiple project management, strategic project management, and value-focused project management. His works have been published in IEEE Transactions on Engineering Management, Journal of High Technology Management Research, International Journal of Project Management, and Engineering Management Journal. Peerasit is a recipient of the 2007 Best Paper Award from IEEE Engineering Management Society. Systems Engineering DOI 10.1002/sys 352 PATANAKUL AND SHENHAR Aaron J. Shenhar, a world leader in project management, is currently a Professor of Project and Program Management at Rutgers Business School. Until 2008 he was Institute Professor of Management and the founder of the project management program at Stevens Institute of Technology. Previously he was at various positions at the Universities of Minnesota and Tel-Aviv. He holds five academic degrees in engineering and management from Stanford University and the Technion in Israel. He was the first recipient of the Project Management Institute Research Achievement Award, and the IEEE Engineering Manager of the Year Award. Prior to his academic career, Dr. Shenhar accumulated 18 years of technical and management experience as an executive at Rafael, a leading organization in the defense industry in Israel. In his research he is focused on project management, innovation management, and the leadership of professional workers in technology-based organization. He is co-author of the book, Reinventing Project Management: The Diamond Approach to Successful Growth and Innovation (Harvard Business School Press, Boston, 2007). The book was selected among the best five top business books of 2007. Systems Engineering DOI 10.1002/sys View publication stats