Process

lmprovement
and the
Lorporate
Balance Sheet
RAYMOND Drohr , Ray theon

I) Since it started a for the success of a new software-devel-

opment process is the accurate evalua- Raytheon, a diversified, internation-
pm-improvement tion of how effective it is in reducing the al, technology-based company, is one of
zizitiative in 1988, bottoni-line cost of getting the job the 100 largest corporations in the US.
done.’ In this article, I describe The Equipment Division is one of eight
RaytbemkE+qnnmt Raytheon’s approach to measuring how divisions and 11 major operating sub-
changes in its software process resulted sidiaries within Raytheon, with annual
Dizirim bas hpoued in reduced development costs. sales in electronic systems that comprise
At Raytheon, we have been able to about 13 percent of the corporation’s
its bottomline, zizcvased convince top-level management that $9.1 billion annual sales.
software-process improvements withln Most of these electronic systems are
pdzutivity, and its Equipment Division have yielded software-driven and real-time, and most
+ a $7.70 return on every dollar in- of the software is developed by the divi-
cbanged the cqorate vested; sion’s Software Systems Laboratory,
+ a two-fold increase in productivity; which employs about 400 software engi-
culture.Mwb oftbe and neers.

saving camefi-om
+ an evolution from level 1 ( h t i a l ) There are seven &rectorates, or busi-
to level 2 (Repeatable) to level 3 (De- ness areas, w i b the Equipment Division,
fined) process maturity, as measured by each one servicing a different application
reducing rework. the Software Engineering Institute’s area and a unique set of customers. Thls
Capability hlaturity Model. creates challenges for the SSL: Usu-

28 0 7 U 0 7 4 5 9 / 9 3 / 0700/’W28 SO? CG C IEEE JULY 1993

ally, it is the application and sometimes the velopment and management process in analysis (first on pilot projects to confirm
customer that drives the selection of target accordance with the principles of W. Ed- the findmgs) and how to diffuse the new
processors for new systems. In turn,h s wards Demin$ andJosephJuran? Basically, methods among practitioners (technology
constrains the selection of their phlosophy teaches transition).

I
a development environ- that real proceSS-improve- Improvement is continuous; thus,
ment and tool set. RAYTHEON’S ment must follow a se- completion of the h r d phase signals a be-
T h e software-devel- quence of steps, starting ginning of the first.
opment process that was PARAD’‘’ with making the process
in place in early 1988 was PUTS PROJECTS visible, then repeatable, FINANCING IMPROVEMENT
not adequate to deal with
h s diversity. As a result, AT THE CENTER. a n ~ ~ ~ ~ ~ ; a t Ineattempting s , to apply h s process-im-
many SSL brojects were Raytheon’s process-im- provement paradigm to the development
delivered late and over m-ovement Daradigm is process over a broad spectrum of projects,
Y

.- That year, the SSL performed an based on a three-phase cycle of stabiliza- we found that the number, inherent com-
budget.
assessment of its own software-develop- tion, control, and change, which applies plexity, and predominantly intellectual
ment process using the SEI’s capability- the principles of Deming and Juran. nature of the labor-intensive steps in the
assessment questionnaire. The SEI’s five- In the procesr-stabilimtim phase, the process required that we proceed with
level maturity model and questionnaire emphasis is on distilling the elements of caution.
(described in the article onpp. 18-27) pro- the process actually being used (acheving
vide a simple analytical tool for bench- visibility) and progressively institutional- Funding sources. In the first place, it is
marking an organization’s status. In 1988, izing it across all projects (providmg re- expensive and time-consuming to iterate
the SSL rated itself at level 1 (In~tial). peatability). even once through the three-phase im-
In the procesr-control phase, the empha- provement cycle. Financing stabilization,
OBJECTIVES AND APPROACH sis shifts to instrumenting projects to control, and change with short-term proj-
gather significant data (measurement) and ect funds is likely to be unsuccessful.
T h e results of the self-assessment analyzing the data to understand how to Discretionary funding (overhead, in-
prompted the Equipment Division to ini- control the process. dependent research and development,
tiate, in mid 1988,a process-improvement In theprocesr-change phase, the empha- and reinvested profit) is the obvious solu-
program within the SSL. The program, sis is on determining how to adjust the tion, but tl-us approach requires two im-
known as the Software Engineering Ini- process as a result of measurement and portant ingredients if management is to
tiative, was strongly influenced by the
SEI’s process-assessment project.*
The Initiative has since been the focal I 1
point for improving the division’s soft-
ware-engineering process and a channel
for institutionalizing knowledge of soft-
ware-engineering methods and technol-
ogy as well as division policy, practices,
and procedures.
The Initiative’s objectives, goals, and
actions are determined by a Steering
Committee, whch includes the SSL line
managers responsible for software devel-
opment. The major activities withm the
initiative are directed by four worlung
groups: Policy and Procedures, Training,
Tools and Methods, and Process Database
(metria). These were the areas our self-as-
sessment had identified as needing im-
provement. In the box on pp. 30-3 1, I de-
scribe these entities and some of the
specific tasks they perform.
The Initiative’s fundamental objective Figure 1. Raythem’sthree-phase proresj-improvementparadigm applies the principlesof Demingand3uran,
is the continuous improvement of the de- andpkzcesprojectsat thefocalpoint.

IEEE SOFTWARE 29
 Perhapsbecausewebasedit uniqueCharafteristcs). One of ress in their own areaand aid well as development practices.
on the results of a self-asses- itsmostimportanttasksisto communication amonginter- Dismbutfdtoalldivision
ment, the organization of the prepare a n a n n d plan and actingpmjects.TheInitiative personnelinvolvedin anyway
Initiative has stood the test of present it to the division gen- Manager (which is my tide) insoftH.aredwelopment;tfiis
time. Since we started in 1988, eral manager. monitDrsthe day-to-day activi- documentis updated periodi-
the scope of the activities un- This group adjusts priorities ties, funding allocations,and cauy,as our standard process
derclken byeachentitywithin and reallocdtesbudgets, when the statusof individual asks. WOlYtS.
theInitiativehas broadened neasary. The amount of fund- ThegrOUphaSalSodefined
somewhat, but we h v e not had ing allocated to ea& of the bur Pakydi+fOdm%rn a set of detailedprodures for
to expgnd beyond a steering workinggroupsvarieseach working group is charged with use in keyprcmss areasand a
committeeand fourworlung y”,as priorities shift.In 1992, draftinga new kh-Engi- setof +lines that include
groups. the Initiative’s $9Sl,OOO budget neering Policy fbr the division recommended support soft-
was allocated as follows: Policy and keeping it ament. ware, fw example. These de-
ShingCa&tmThisen- and Procedures, 18percent; BYAuslst1~,~group tailed pllmxhlm cover things
tity, &ch meets at least once a T W , 23 percent;Tmls and hadddidapdicythatwasap- like unit testing, costing and
month, provides direction and Methods, 30 percenq and Pro- p m v e d b y b d r v i s i a p l ~ scheduling,design and code in-
oversight It reviewsthe status cess Database (metria), 29 per- m a r r a g e s a n d d t o a l l spections, andtimingandsiz-
and plans of each working cent SSLand dlremmentanagers, in- ing.The p u p continues to ex-
group,not only in the context The compositionof the ciudmghowthepolicywtobe pand the series.
of long-termimprovementin SteeringCommittee is very im- tr;msitiolledto** Policy and pnredures is
the division’ssranrtPdproces portant It is chaired by the By the third quarter of alsochargedwithmakingsure
(dehed by the SEI as a coher- SSL manager, who is the driv- 1990, it had put in place a rea- ongoingprojects are in compli-
ent collection of management ing force b d the Initia&e sonablyhigh level,yet practical, ance withthe Equipment
and software-engineering pro- and the communication chan- description of what is involved Division’s standard process.
cesses), but also in the context nel to top-level manage men^ in developing softwarein the This includesreviewing the de-
of the short-term+dpro- Other members ax the chairs Equipment Division. TI& pro- velopment plansfor all new
cessusedbyprojerrsandpro- and cochairs of the bur work- prietary document covers all as- proposalsand Programs
pasals (a version of the m d a r d inggroups, who contribute pens of the lifecyde and in- performingconlpliancech&
process tailored to a project’s first-hand knowledgeof prog- cludes managementissues as at project milestrwes. It also

persevere: First, there must be some short- provement approach places projects at the during the first two phases as well.
term benefit to ongoing projects; second, focal point. This focus on involving proj- During the stabilization phase, th
there must be a meaningful quantification ect personnel also helps to address tech- process is documented, disseminated to a]
ofwhat that benefit is. nology-transition issues - those who practitioners, and institutionalize,
Funding for the Initiative comes from contribute to changing the process essen- through training programs. To get fund
the SSL‘s discretionaly overhead budget. tially buy into the change and so adopt it ing, we knew the SSL had to get it
Over the period of the Initiative (nearly more readily. customers (the seven business areas), whi
five years), these funds have been used to of course would be affected by any proces
support a staff of only one or two full-time Measwing effect. In the second place, in changes, to buy into the process. We hell
personnel, plus many people working l a u n c h g the Initiativewe had to consider a series of briefings for our customer:
part-time - for an average equivalent of how individual small improvements im-
~ each briefing incorporating suggestion
about 15 people per year. At any one time, , plemented more or less in parallel would made at the previous one. In this way, w
about 100 people are involved in one pro- interact to produce a net loss or gain. For got project members to buy into what pro
cess-improvement task or another. example, introducing a new technique cedures were determined to be “best,
Obviously, the Initiative relies heavily ’ probably involves a detailed analysis of al- very early on. When the htiative the]
on the part-time efforts of many people. , ternatives, the development and docu- targeted those best procedures for trainin
Fully 2 5 percent of the SSL staff have mentation of procedures to support it, and courses, we saw an immediate improve
some formal involvement in the Initiative, new training materials. Instead o f w g to ment in project performance.
and 100 percent of the staff work on pro- ascertain the value ofeach change compo- During the control phase, the Initiativ
jects addressed by the Initiative. nent, we decided it would be easier to sees to it that projects are instrumented tl
Using the skills and experiences of so measure the overall effect of a change on obtain process-related measurements i
many project personnel to drive the Initia- the bottom line. preparation for root-cause analysis.Agair
tive guarantees that the resulting process is We also had to address the question of benefits accrued from measurement, eve:
a consensus that reflects the practical con- when we could begin to measure change. at the project level. Individual engineer:
siderations of diverse projects. It also pro- ’ Some believe that the rewards of process when made aware of error-data categori
vides the immediate, short-term benefits improvement are reaped onlywhen an en- zation, tend to be more careful in area
so important to management. tire three-phase cycle has been completed. with the highest frequency of errors.
As Figure 1 shows, our process-im- Our experience shows that benefits accrue Finally, in the change phase, adjust

30 JULY 199:
 helps projects review and tailor which are
the standard process and review
the lessons learned in applymg
the standard process at the
milestone checks.

Ris to grve engmeen a way

to undersand and implement a ’fn 1
uniform dewfopmentd o d .

process automation.
The Ada e&m inchtde ac-

tionofcorpor;tteAdan&t-
teis(theseefforts are indepen-
dent of Ada uaining effons,

ments are made first on pilot projects so as return and continue to monitor it over vention costs is what Crosby calls “the
to confirm the conclusions of the previous time. cost of quality.” The total project cost is
analyses and then on all projects. Major We used Philip Crosby Associates’ ap- simply the cost of quality plus the per-
cost savings should be aclueved in this proach’ to help us qmn@ the benefit of formance costs.
phase. Of course, tlus is the point at which improvements made to ongoing projects. In early 1990, we used the Crosby ap-
careful attention should be paid to auto-
mating any new parts of the process, to
maximize gain.

1 CALCULATINGSAVINGS
During the approximately five years
the Initiativehas been underway, the divi-
~

sion has invested about $1 million in dis-

cretionary funds in process improvement
each year. Tlus investment has moved the
organization to a solid SEI level 3 (De-
fined) maturity level - as determined by
two separate customer evaluations in late + Apprairal.The costs communications, or air-
1991 - and we are approaching level 4 associated with testing the product to de- traffic-control systems using Raytheon
(Managed). termine if it is faulty. hardware; their development schedules
However, from the standpoint of mak- + Rewmk (nonc4mnce). The costs were aggressive but not unachievable;
ing sound investment decisions, you must associated with h g defects in the code they were staffed with a typical mix of soft-
be able to show that achieving level 3 or or design. ware-engineering SUS and experience in
level 4 provides a return that justifies such + Prevention. The costs incurred in at- the application area; and they were led by
a large expenditure. We have been able to tempting to prevent the fault from getting progressive managers. However, two pro-
get management approval to continue this into the product. jects (A and B) were well underway before
investment because we have measured our The sum of appraisal, rework, and pre- the Initiative began, while the other four
-

IEEE SOFTWARE 31
 ! Cost of rework
I

project members were involved and data

collection took longer.
In preparation for instituting formal
inspections, the Policy and Procedures
group wrote detailed procedures, and the
Training group taught a course in inspec-
tions to developers just before the design
stage, using materials tailored to the proj-
ect. This course not only emphasized in-
spection techmques, but it also reinforced
the ideas ofMichael Fagan: The attention
you pay to uncovering errors up front is
guaranteed to save you money in the end.8
We had to convert many skeptics to tlus
doctrine.
Another factor in these increased costs
was that we paid a lot more attention to
getting the requirements solidly in place
before beginning major development. We
achieved tlus by imposing policy; reinforc-
(C, D, E, and E’) were just starting. percent of total project cost. It’s interest- ing the policy in trainingeveryone involved
The initial results of the analysis were ing that, in 1987, Barry Boehm estimated in software,not just front-line developers, on
promising, so we repeated the analysis that eliminating rework might reduce fundamentals;and integrating a computer-
nine months later, near the end of 1990, software costs by 30 to 50 percent.’ In- aided software-engineering tool method-
when projects A, B, C , and D were fin- deed, we have shrunk rework costs during ology into our way of doing business. T h ~ s
ished and projects E and F were about 90 thls period to about one fourth its original last policy did not come cheaply,as the box
and 70 percent complete, respectively. value (from 41 to 11 percent). on p. 3 5 describes.
The second analysis indicated that we had Rework has many components, so it is
eliminated about $9.2 rnillion in rework worthwhile to analyze the cost factors that Integmfion.The largest contributor to re-
C0StS.h contributed most to the overall reduction work costs is the cost associated with h n g
Since then, nine more projects (which, and to try to determine which process source-code problems found during inte-
on average, happen to be smaller in terrns changes might have brought about these gration. This cost decreased to about 20
of delivered source code) have been added reductions. percent of its original value. We believe
to the database,and a new analysis of all I 5 Also, rework savings are often achieved this savings can be credited to the design
projects, shown in Figure 2, indicates we at the expense of a small inn-easein the cost and code inspections, procedures, train-
have eliminated about $15.8 million in re- of other process stages. In our case, the ing, and requirements stability.
work costs through the end of 1992. cost of design and coding rose slightly The greater gain achieved by m a h g
Figure 2 also shows that appraisal costs because formal inspection replaced in-
have increased by about five percent. 4-formal reviews. However, it was this
though the increased rigor with which we very change that enabled us to achieve
conducted design and code inspections in rework savings in uncovering source-
the later stage accounts for some of this code problems before software integra-
the integration phase more efficient is no
doubt due to the fact that h n g problems
this late in the process costs so much more.
A less obvious, but perhaps as important I
factor, is that errors found during integra-
li
increase, most of it is due to a 30-percent tion and eliminating unnecessary retest- tion tend to have a compounding effect on ~

decrease in total project cost, which has ing. integration costs: Very often, discoveringa
pushed up appraisal cost proportionally, as
a percentage ofthat total.
1~
single error a t integration brings the entire
process to a halt until the problem can be
k i g n mdcding. The cost of fixing defects
Prevention costs, on the other hand, found during design rose from about 0.75 resolved.Lost time duringintegration is def-
have been increasingly borne 1-y the Initia- percent to about two percent of the total initelynot a linear function of the number of
tive and so are not subject to the same project cost, and the cost of fixing defects errors in the sobare being integrated.
proportional increases. found during coding rose from about 2.5
percent to about four percent of that total. Reteshg. The second largest contributor
Rework savings. As Figure 2 shows, the These increased costs were undoubt- to rework costs is the cost of retesting,
rework costs at the time the Initiative edly caused by conducting formal inspec- which decreased to about half its original
started (August 1988) averaged about 41 tions instead of informal reviews - more value. First-time testing remained reason

32 JULY 1993
 ably constant, at about nine to 10 percent
of the total effort, because all the software
is tested at least once.
The decrease in retesting indicates that
far fewer problems were found during the
first test, a h e c t effect of removing design 1- . toSm-D#emtM&,
and coding errors found in inspections.
Detailed procedures and training helped
indirectly to decrease the amount of re-
testing, and having solid requirements
e h n a t e s the repetitive tests that a fluid
basehe creates.

The boffom he. The bottom h e is that standard prixess, and the SEI’SCapbili
these reduced rework costs are real. Pro- tools, contents and uses of test
jects E and F completed s o b a r e develop- testers, developers,and software
ment during 1991, with software-only -
managers, eight hours Ada strut-
costs at about $17 million per project. ture and terminology, lessons leamed.
Both hished ahead ofschedule and under Fundmmm3 $A&, middle-leveltask managers, grmp and project managers,20 .
budget; project E was four percent under hours -Lecture course on how to read, but not write, Ada source code and how top”-
budget and project F six percent under ticipate in code reviews.
budget. When project F was delivered to
the customer, Raytheon earned a $9.6 mil- Detcibd(OUE~LWe usually schedule courseson a just-in-time basis to help specific
lion schedule-incentive payment (a bonus projects. In some cases, the tmining materials are tailoredto a project. The goal of these
for meeting or beating a deadhe), which comes is to make the attendee more productive on the project that is fun- his atten-
is not included in any of the return-on-in- dance so that the trainingis self-supporting.
vestment calculations in h s article. Design and codehpectim,senior engineers, 14hours -%pres inspection team
leaders.
RETURN ON INVESTMENT SRsmZs Generatirm zlsing s0fiwap.e Tb-oughM, senior engineers, 32 hours -
Structured analysis for requirements, using the CASE tool SoftwareThrough Pictures;
In calculatingthe annualized return on genemting associated DoD-Std-2167Adocuments, such as software-quirementsspeci-
investment, we made several simplifyrng ficationsand inte&ce-re+ments s~cations.
assumptions. S0jiwu-e C+gwatim ikihqwmt, engineers, 12 hours -Standard policy and pro-
First, the savings in one year is due not cedures, tools, utilities, and resources.
only to investments made that year, but Unit T&g, engineers, 12 hours -When and how to performunit testing, tech-
also to investments made in previous
niques and tools, unit-test plan and procedure preparation, conducting analysis of test-
years. We ignore h s factor, considering
ing, and =-exit criteria. Lab for test plan and procedure.
only the full investment during one year
and the savings in the same year.6 S @ m ppoJectit4z?zugeinent,development managers,40 hours-Skills and tech-
Second, we prorated the process-im- niques to develop softwarewithin budget and on schedule.
provement investment for the six projects Sojiwm Sizing and Tm’ng,senior engineers, eight hours -Estimation techniques
in the 1990 sample. These six projects em- used for proposals and in early development, emphasizingprototypes; trackmg tech-
ployed about 58 percent of the total avail- niques, with examples.
able SSL labor for 1990 and thus received S$wm Engmewig with A&, engineem, 64hours -Lecture-lab courseemphasizes
the benefits of that percentage of the $1 hands-on, interactive exercises in Ada coding. Allaspects ofAda in detail.
million invested. The total savings of the D m Sazrclzg-esUiingAdz,engineers, 32 hours -Lecture-lab course on fundamen-
six projects in reduced rework was $4.48 tal data structuresand how to use Ada p ackages.
million during 1990; thus, the return on
investment was approximately 7.7 to I (a ulder This year the working group is developingmore courses to cover a
$4.48 million return on a $0.58 million broader range of softwareagineerirtgtopics. Cuurses on DesiiTechniqUesUsing
investment). SoftwareThrough Pictures, Software Testing Methods, Object-OrientedMethodology,
The project savings in h s calculation Costing and Scheduling,Measurementand Root-Cause Analysis, and SoftwareIntegra-
represents only the first-order effect of the tion are now being developed.
process improvement. You could argue
~
d
IEEE SOFTWARE 33
 ences in the typesof software applicationswe
develop. Because we have used approach
forsometime-notonlytocountcodeafter
development, but also to monitor code-
growth estimates during development and
estimate code size for bids - it is widely
accepted by managers and developers.
As Figure 3 shows, after computing the
weighted average of the productivity data
using the same technique we used for cost
I data, we found that average productivity
l l
had, in fact, increased substantially from
1 8 , . I 1 1988 to 1990. T h e increase was large
FMAMiJASOND enough to convince the skeptics that the
1992 reduction in project cost from the de-
creased cost of rework was real.
Year
We continue to update weighted average
productivity as projects are added to our
database; we now have data on 18 projects,
not all of them complete. The latest results
are reflected in Figure 3, which shows that
we have obtained an average productivity
that benefits are derived from improved saings a t all? Perhaps the costs expressed increase of about 130 percent over the pe-
competitive position, higher morale, and as a percentage of total project cost are riod of the Initiative. In other words, pro-
the lower absenteeism and amition rates being redistributed, but the absolute proj- ductivity has increased by a factor of about
that typically result from successful pro- ect cost is, in fact, not changing at all or 2.3 in about four and a halfyears.
cess-improvement programs. Vt'e did not even going up! Comparing these 18projects over time
include any of these potential savings in Because the characteristics of the deliv- is a valid way to evaluate productivity im-
our calculations. ered products are unique, it is impossible provement because, although every proj-
Also, most of the process-improve- simply to compare the bottom h e s of ect is unique, they are all of the same type
ment funding comes from overhead, later and earlier projects. So to answer thls - real-time, embedded - and withln a
whereas the sakings comes from lower question, we have obtained productivity reasonable size range - 70,000 to
project costs, which result in lower custo- data associated with all 15 projects in the 300,000 lines of code. The method of
mer billings. Thus, process improvement analysis, in terms of eqriicalent delizwed measuring productivity did not change,
achieves a higher profit only if you con- soiwc'e i?mirL~ionsper. man-month. This is either. Thus, if neither the nature of the
sider that getting the job done on time and our commonly accepted measure of the application nor the measurement method
on budget will bring in more business and engineering effort involved in developing changed in thls time, it is reasonable to
that the business mill, in turn, yield an ad- the product (whether it be all-new, modi- credit the improvement to what did
ditional profit. Of course, this applies only fied, or reused sohare). change, namely, the process.
to cost-plus contracts. For fixed-price If? calculate source-code size with a Schedule and budget performance
contracts, the savings flow directly to the tool that counts lines in most common gains are also impressive. Before the Ini-
bottom line. lanpages (Ada, C , Fortran, and Pascal) tiative, most projects were completed be-
Finally, this calculation implies that and several assembly languages. This tool hind schedule and over budget. In many
improved performance is a direct result of excludes blank and comment lines and in- cases, key personnel were being moved to
process improvement and not simply a re- cludes executable, logical (as opposed to projects on the basis of crisis priorities. In
sult of improving our development envi- physical) lines. Alodified and reused lines other words, we were operating in a fire-
ronment, acquiring more powerful tools, of code are weighted according to the rel- fighting mode. Thls may have been the
and increasing our MU'S per seat, all of ative effort (as compared to new code) of reason our management was so receptive
which we did do. I address this question in niodibing or reusing it (including its re- to the Initiative proposal.
the box on the facing page. lated documentation). Now, most projects are finishing on or
lye realize that this is not a scientifi- ahead of schedule and below budget.
Red savings? The first challenge to the cally accurate exercise because of varia-
analysis of real project savings is usually if tions among projects, but we feel that Less tangible rewords. In addition to the
all costs are normalized, do you really have there are far more similarities than differ- project characteristics that we carefully

34 JULY 1993
 I

track to document process-improvement

success (cost of rework, productivity, and
t program in open fonuns, Iamfie-
cost-performance index), we have ob-
I report d t e d Erom the Initiative,
served less tangible but equally important
t, more powerful d s , and a mani-
results.
Software engineers are spending fewer
late nights and weekends on the job, and
there’s less turnover. There’s less pressure orgmizatMnirnosntsplemaaer:Thereare
being applied to get back on schedule, be- thathavefailedtopruduceimpmve-
cause schedule erosion is occurring much Raytheon take full advantage oft&-
less frequently. fCASEmls:Overtwotothreeyem,the
T h s is certainlynot a phenomenon of
a reduced contract workload; on the con-
trary, in the last six months we have hired
100 SSL staff members to meet the de-
mands of new business.
Communication has improved. The tire organlzatin, which has embraced the idea of wntinuously impraving the way we do
Initiativefunds a quarterly SSL staff news- ourworkandiscommitted to
letter that includes a current list of ongo- t h e p a s . The process was
ing projects and proposal efforts. mtiallyimproved.Why
Because the Initiative is now five years
old, we do not feel these benefits can be
attributed to the Hawthorne effect (which
says that anythmg under study wdl im-
prove simply because it is under study). ACKNOWLEDGMENTS
I thankBob Charette, Watts Humphrey, Blake Ireland, Stan &&in, and Jim Stubbe for their helpful re-
We continue to get enthusiasticp d c i p a - view comments. The comments and suggestions OfZEEESq’harc’r anonymous referees were very helpful.
tion from SSL staff in process-improve-
ment activities. REFERENCES
Of course, it’s difficult to point to direct 1. W.S. Humphrey, Managing the sofrure Pmces, Addison-Wesley, Reading,Mass., 1989.
causal relationships for these h d of kctors, 2. W.S. Humphrey et al., “AMethod for Assessing the Software Engineering Capabilityof Contractors,”
but i n v o h g 2 5 percent of the personnel in Tech. Report CMU/SEI-87-TR-23, Software Engineering Institute, Pittsburgh, 1987.
3. WE. Deming, Out ofthe Crkis,MIT Center for Advanced Engineering Study, Cambridge, Mass., 1982.
improving a process that they themselves 4. J.M. Juran,Frun on Plamzmgfi Quulity,The Free Press, New York, 1988.
apply must make for higher levels of inter- 5. P.B. Crosby, Quahty Whmt Eun,McGraw-Hill, New York, 1984.
personal contact and job satidction. 6. R. Dion, “Elements ofa Process-Improvement Program,”IEEE soffwure,July 1992,pp. 83-85.
We’ve experienced a definite culture 7. B. Boehm, “Improving SoftwareProductivity,” Conputer, Sept. 1987,pp. 43-57,
shift in the area of training. We no longer 8. M. Fagen, “Advanced in SoftwareInspections,”IEEE Puns. SofnuareEng.,July 1988, pp. 75 1-774.
expect practitioners to absorb the SUS
they need on the job (by associating with
other workers). Instead, we have a com-
prehensive, fully funded training pro-
gram. The courses are listed in the box on
k
nyone who has developed software
r the last 10 to 15 years in an o r e -
ization that is in the business of develop-
Raymond Dion is a prina-
pal engineer with Raytheon’s
Equipment Dvision, where
p. 33. Project managers who once insisted ig large, embedded systems can appreci- he has managed the software
that all training be funded by overhead are te the traditional difference in the level of process-improvement pro-
now accepting proposal bids that include espect accorded hardware and software gram since its inception. He
ngineers. has more than 30 years expe-
training costs, because we can demon- rience in software engineer.
strate a direct benefit to the project. We have seen a definite shiftaway from ing.
Naw that customers are using the SETS i s traditional position as the sobare- Dion received a BS in
software-capabjlityevaluations as a discrim- ngineering function has become more phyics from Providence College. He is a member of
inator in contract awards, process capability iature, disciplined, and consistent. In the SoftwareEngineering Institute’sCapabilityMatu-
rity Model Advisory Board and Software Capability
has &en a seat much closer to the action. In ict, our Initiative has become a model Evaluation Review Group and the IEEE.
ha,we have recently been awarded at least 3r hardware engineering, spawning a
two contracts in which our sobare-process Iardware Engineering Initiative that
Address questions about this article to Dion at Ray-
capability was said LDhave made the Mer- as profited from the lessons learned theon Equipment Division, 528 Boston Post Rd., Sud-
ence between winning and losing. -om software. +
bury, MA 01776; Internet [email protected].

IEEE SOFTWARE 35