Rashtreeya Sikshana Samithi Trust

RV Institute of Technology and Management®

(Affiliated to VTU, Belagavi)

JP Nagar, Bengaluru - 560076

Department of Information science & Engineering

Course Name: Software Testing

Course Code: 21IS63

VI Semester
2021 Scheme

Prepared By:
Mrs. Swathi Darla
Assistant Professor,
Department of Information Science and Engineering
RVITM, Bengaluru – 560076
Email: [email protected]
 MODULE IV
Integration and component-based software testing

4.1 Overview
Divides testing into four main levels of granularity: module, integration, system, and acceptance
test. Module or unit test checks module behavior against specifications or expectations;
integration test checks module compatibility; system and acceptance tests check behavior of the
whole system with respect to specifications and user needs, respectively

While integration testing may to some extent act as a process check on module testing (i.e., faults
revealed during integration test can be taken as a signal of unsatisfactory unit testing), thorough
integration testing cannot fully compensate for sloppiness at the module level. In fact, the quality
of a system is limited by the quality of the modules and components from which it is built, and
even apparently noncritical modules can have widespread effects.

Integration faults are ultimately caused by incomplete specifications or faulty implementations of

interfaces, resource usage, or required properties

4.1.1 Integration faults

Inconsistent interpretation of parameters or values each module's interpretation may be

reasonable, but they are incompatible Example: Unit mismatch: A mix of metric and British
measures (meters and yards) is believed to have led to loss of the Mars Climate Orbiter in
September 1999

Violations of value domains or of capacity or size limits implicit assumptions on ranges of

values or sizes Example: Buffer overflow, in which an implicit (unchecked) capacity bound
imposed by one module is violated by another, has become notorious as a security vulnerability.
For example, some versions of the Apache 2 Web server between 2.0.35 and 2.0.50 could
overflow a buffer while expanding environment variables during configuration file parsing

Side-effects on parameters or resources Example: A module often uses resources that are not
explicitly mentioned in its interface. Integration problems arise when these implicit effects of one
module interfere with those of another. For example, using a temporary file "tmp" may be
invisible until integration with another module that also attempts to use a temporary file "tmp" in
the same directory of scratch files Missing or misunderstood functionality

Sem-VI, Software Testing(21IS63) Page 1 of 27

 lead to incorrect assumptions about expected results Example: Counting hits on Web sites maybe done
in many different ways: per unique IP address, per hit, including or excluding spiders, and so on.
Problems arise if the interpretation assumed in the counting module differs from that of its clients
Nonfunctional problems Example: Nonfunctional properties like performance are typically
specified explicitly only when they are expected to be an issue. Even when performance is not
explicitly specified, we expect that software provides results in a reasonable time. Interference
between modules may reduce performance below an acceptable threshold

Dynamic mismatches many languages and frameworks allow for dynamic binding. Problems may
be caused by failures in matching’s when modules are integrated Example: Polymorphic calls
may be dynamically bound to incompatible methods

4.1.2 Integration Testing Strategies

Integration testing proceeds incrementally with assembly of modules into successively larger
subsystems. Incremental testing is preferred, first, to provide the earliest possible feedback on
integration problems. In addition, controlling and observing the behavior of an integrated
collection of modules grows in complexity with the number of modules and the complexity of
their interactions

Big bang testing One extreme approach is to avoid the cost of scaffolding by waiting until all
modules are integrated, and testing them together - essentially merging integration testing into
system testing. In this big bang approach, neither stubs nor drivers need be constructed, nor must
the development be carefully planned to expose well-specified interfaces to each subsystem.
These savings are more than offset by losses in observability, diagnosability, and feedback

Structural integration test strategy: Among strategies for incrementally testing partially
assembled systems, we can distinguish two main classes: structural and feature oriented. In a
structural approach, modules are constructed, assembled, and tested together in an order based on
hierarchical structure in the design. Structural approaches include bottom-up, top-down, and a
combination sometimes referred to as sandwich or backbone strategy. Feature oriented strategies
derive the order of integration from characteristics of the application, and include threads and
critical modules strategies.

Top-down and bottom up testing: A top-down integration strategy begins at the top of the uses
hierarchy, including the interfaces exposed through a user interface or top-level application
program interface (API)

Sem-VI, Software Testing(21IS63) Page 2 of 27

 Bottom-up integration similarly reduces the need to develop stubs, except for breaking circular
relations. Referring again to the example in Figure 21.1, we can start bottom-up by integrating
Slot with Component, using drivers for Model and Order.We can then incrementally add Model
and Order. We can finally add either Package or Account and Customer, before integrating
CustomerCare, without constructing stubs

Sandwich or backbone: Integration may combine elements of the two approaches, starting from
both ends of the hierarchy and proceeding toward the middle. An early top-down approach may
result from developing prototypes for early user feedback, while existing modules may be
integrated bottom-up. This is known as the sandwich or backbone strategy

Sem-VI, Software Testing(21IS63) Page 3 of 27

 Thread testing: The thread integration testing strategy integrates modules according to system
features. Test designers identify threads of execution that correspond to system features, and they
incrementally test each thread. The thread integration strategy emphasizes module interplay for
specific functionality.

Critical module: Critical module integration testing focuses on modules that pose the greatest

risk to the project. Modules are sorted and incrementally integrated according to the associated
risk factor that characterizes the criticality of each module. Both external risks (such as safety)
and project risks (such as schedule) can be considered.

4.1.3 Testing Components and Assemblies

Many software products are constructed, partly or wholly, from assemblies of prebuilt software
components. A key characteristic of software components is that the organization that develops a
component is distinct from the (several) groups of developers who use it to construct system
when reusing a component that has been in use in other applications for some time, one obtains
the benefit not only of test and analysis by component developers, but also of actual operational
use.

These advantages are balanced against two considerable disadvantages. First, a component
designed for wide reuse will usually be much more complex than a module designed for a single
use; a rule of thumb is that the development effort (including analysis and test) for a widely
usable component is at least twice that for a module that provides equivalent functionality for a
single application.

The interface specification of a component should provide all the information required for
reusing the component, including so-called nonfunctional properties such as performance or
capacity limits, in addition to functional behavior

The main problem facing test designers in the organization that produces a component is lack of
information about the ways in which the component will be used. A component may be reused in
many different contexts, including applications for which its functionality is an imperfect fit

Test designers cannot anticipate all possible uses of a component under test, but they can design
test suites for classes of use in the form of scenarios. Test scenarios are closely related to
scenarios or use cases in requirements analysis and design

Sem-VI, Software Testing(21IS63) Page 4 of 27

 4.2 Terminology for Components and Frameworks
Component A software component is a reusable unit of deployment and composition that is
deployed and integrated multiple times and usually by different teams. Components are
characterized by a contract or interface and may or may not have state.
Components are often confused with objects, and a component can be encapsulated by an object
or a set of objects, but they typically differ in many respects: Components typically use
persistent storage, while objects usually have only local state.

Components may be accessed by an extensive set of communication mechanisms, while objects

are activated through method calls.

Components are usually larger grain subsystems than objects.

Component contract or interface The component contract describes the access points and
parameters of the component, and specifies functional and nonfunctional behavior and any
conditions required for using the component.

Framework A framework is a micro-architecture or a skeleton of an application, with hooks for

attaching application-specific functionality or configuration-specific components. A framework
can be seen as a circuit board with empty slots for components.

Frameworks and design patterns Patterns are logical design fragments, while frameworks are
concrete elements of the application. Frameworks often implement patterns.

Component-based system A component-based system is a system built primarily by assembling

software components (and perhaps a small amount of application specific code) connected
through a framework or ad hoc "glue code."

COTS The term commercial off-the-shelf, or COTS, indicates components developed for the
sale to other organizations.

Sem-VI, Software Testing(21IS63) Page 5 of 27

4.3 System, Acceptance, and Regression Testing

System, acceptance, and regression testing are all concerned with the behavior of a software
systemas a whole, but they differ in purpose.

Table: 4.1 Test cases

4.3.1 System testing

System testing can be considered the culmination of integration testing, and passing all system
tests is tantamount to being complete and free of known bugs. The system test suite may share
some test cases with test suites used in integration and even unit testing, particularly when a
thread-based or spiral model of development has been taken and subsystem correctness has been
tested primarily through externally visible features and behavior.

The appropriate notions of thoroughness in system testing are with respect to the system
specification and potential usage scenarios, rather than code or design. Each feature or specified
behavior of the system should be accounted for in one or several test cases

Some system properties, including performance properties like latency between an event and
system response and reliability properties like mean time between failures, are inherently global.
Global properties like performance, security, and safety are difficult to specify precisely and
operationally, and they depend not only on many parts of the system under test, but also on its

Sem-VI, Software Testing(21IS63) Page 6 of 27

 environment and use.

Table: 4.2 Unit, Integration,System Test cases

Sem-VI, Software Testing(21IS63) Page 7 of 27

4.3.2 Acceptance Testing

The purpose of acceptance testing is to guide a decision as to whether the product in its current
state should be released. The decision can be based on measures of the product or process.

Quantitative goals for dependability, including reliability, availability, and mean time between
failure

Systematic testing, which includes all of the testing techniques presented heretofore in this book,
does not draw statistically representative samples. Their purpose is not to fail at a "typical" rate,
but to exhibit as many failures as possible. They are thus unsuitable for statistical testing.

A less formal, but frequently used approach to acceptance testing is testing with users. An early
version of the product is delivered to a sample of users who provide feedback on failures and
usability. Such tests are often called alpha and beta tests. The two terms distinguish between
testing phases. Often the early or alpha phases are performed within the developing organization,
while the later or beta phases are performed at users' sites.
In alpha and beta testing, the user sample determines the operational profile

Usability

A usable product is quickly learned, allows users to work efficiently, and is pleasant to use.
Usability involves objective criteria such as the time and number of operations required to
perform tasks and the frequency of user error, in addition to the overall, subjective satisfaction of
users

Even if usability is largely based on user perception and thus is validated based on user feedback,
it can be verified early in the design and through the whole software life cycle. The process of
verifying and validating usability includes the following main steps:

Inspecting specifications with usability checklists. Inspection provides early feedback on

usability.

Testing early prototypes with end users to explore their mental model (exploratory test),
evaluate alternatives (comparison test), and validate software usability. A prototype for early
assessment of usability may not include any functioning software; a cardboard prototype may be
as simple as a sequence of static images presented to users by the usability tester.

Testing incremental releases with both usability experts and end users to monitor progress and

Sem-VI, Software Testing(21IS63) Page 8 of 27

 anticipate usability problems.

System and acceptance testing that includes expert-based inspection and testing, userbased
testing, comparison testing against competitors, and analysis and checks often done
automatically, such as a check of link connectivity and verification of browser compatibility.

User-based testing (i.e., testing with representatives of the actual end-user population) is
particularly important for validating software usability. It can be applied at different stages, from
early prototyping through incremental releases of the final system, and can be used with different
goals: exploring the mental model of the user, evaluating design alternatives, and validating
against established usability requirements and standards.

Exploratory testing: The purpose of exploratory testing is to investigate the mental model of
end users. It consists of asking users about their approach to interactions with the system. For
example, during an exploratory test for the Chipmunk Web presence, we may provide users with
a generic interface for choosing the model they would like to buy, in order to understand how
users will interact with the system
comparison testing: The purpose of comparison testing is evaluating options. It consists of
observing user reactions to alternative interaction patterns. During comparison test we can, for
example, provide users with different facilities to assemble the desired Chipmunk laptop
configuration, and to identify patterns that facilitate users' interactions.

validation testing: The purpose of validation testing is assessing overall usability. It includes
identifying difficulties and obstacles that users encounter while interacting with the system, as well
as measuring characteristics such as error rate and time to perform a task.

Regression Testing

Testing activities that focus on regression problems are called (non) regression testing. Usually
"non" is omitted and we commonly say regression testing.

A simple approach to regression testing consists of reexecuting all test cases designed for
previous versions. Even this simple retest all approach may present nontrivial problems and
costs. Former test cases may not be re executable on the new version without modification, and

Sem-VI, Software Testing(21IS63) Page 9 of 27

 rerunning all test cases may be too expensive and unnecessary. A good quality test suite must be
maintained across system versions

Test case maintenance: Changes in the new software version may impact the format of inputs
and outputs, and test cases may not be executable without corresponding changes. Even simple
modifications of the data structures, such as the addition of a field or small change of data types,
may invalidate former test cases, or outputs comparable with the new ones.

Scaffolding that interprets test case specifications, rather than fully concrete test data, can reduce
the impact of input and output format changes on regression testing

High-quality test suites can be maintained across versions by identifying and removing obsolete
test cases, and by revealing and suitably marking redundant test cases. Redundant cases differ
from obsolete, being executable but not important with respect to the considered testing criteria

4.3.4 Regression Test Selection Techniques

Test case prioritization orders frequency of test case execution, executing all of them eventually
but reducing the frequency of those deemed least likely to reveal faults by some criterion

Prioritization can be based on the specification and code-based regression test selection
techniques described later in this chapter. In addition, test histories and fault-proneness models
can be incorporated in prioritization schemes. For example, a test case that has previously
revealed a fault in a module that has recently undergone change would receive a very high
priority, while a test case that has never failed (yet) would receive a lower priority, particularly if
it primarily concerns a feature that was not the focus of recent changes.

Regression test selection techniques are based on either code or specifications. Code based
selection techniques select a test case for execution if it exercises a portion of the code that has
been modified. Specification-based criteria select a test case for execution if it is relevant to a
portion of the specification that has been changed. Code based regression test techniques can be
supported by relatively simple tools. They work even when specifications are not
Properly maintained.

Sem-VI, Software Testing(21IS63) Page 10 of 27

 Control flow graph (CFG) regression techniques are based on the differences between the CFGs
of the new and old versions of the software. Let us consider, for example, the C function cgi_
decode from Chapter 12. Figure 22.1 shows the original function as presented in Chapter 12,
while Figure 22.2 shows a revision of the program. We refer to these two versions as 1.0 and 2.0,
respectively. Version 2.0 adds code to fix a fault in interpreting hexadecimal sequences ‘%xy’.
The fault was revealed by testing version 1.0 with input terminated by an erroneous subsequence
‘%x’, causing version 1.0 to read past the end of the input buffer and possibly overflow
theoutput buffer. Version 2.0 contains a new branch to map the unterminated sequence to a
question mark.

Test Case Prioritization and Selective Execution

Regression testing criteria may select a large portion of a test suite. When a regression
test suite is too large, we must further reduce the set of test cases to be executed.

Random sampling is a simple way to reduce the size of the regression test suite. high-
priority test cases are selected more often than low-priority test cases. With a good
selection strategy, all test cases are executed sooner or later
Execution history priority schema Priorities can be assigned in many ways. A simple
priority scheme assigns priority according to the execution history: Recently executed
test cases are given low priority, while test cases that have not been recently executed
are given high priority. In the extreme, heavily weighting execution history
approximates round robin selection.

Fault revealing priority schema Other history-based priority schemes predict fault
detection effectiveness. Test cases that have revealed faults in recent versions are
given high priority. Faults are not evenly distribute

Structural priority schema Structural coverage leads to a set of priority schemes

based on the elements covered by a test case.

Structural priority schemes produce several criteria depending on which elements we

consider: statements, conditions, decisions, functions, files, and so on. The choice of
the element of interest is usually driven by the testing level. Fine-grain elements such
as statements and conditions are typically used in unit testing, while in integration or
system testing one can consider coarser grain elements such as methods, features, and
files.

Sem-VI, Software Testing(21IS63) Page 13 of 27

 4.4. Traditional View of Testing Levels

The traditional model of software development is the Waterfall model, which is drawn as a V
in. In this view, information produced in one of the development phases constitutes the basis
for test case identification at that level. Nothing controversial here: we certainly would hope
that system test cases are somehow correlated with the requirements specification, and that
unit test cases are derived from the detailed design of the unit. Two observations: there is a
clear presumption of functional testing here, and there is an implied “bottom-up” testing
order.
4.4.1 Alternative Life Cycle Models
Since the early 1980s, practitioners have devised alternatives in response to shortcomings of
the traditional waterfall model of software development Common to all of these alternatives
is the shift away from the functional decomposition to an emphasis on composition.
Decomposition is a perfect fit both to the top-down progression of the waterfall model and to
the bottom-up testing order. One of the major weaknesses of waterfall development cited by
[Agresti 86] is the over-reliance on this whole paradigm. Functional decomposition can only
be well done when the system is completely understood, and it promotes analysis to the near
exclusion of synthesis.

The result is a very long separation between requirements specification and a completed
system, and during this interval, there is no opportunity for feedback from the customer.
Composition, on the other hand, is closer the way people work: start with something known
and understood, then add to it gradually, and maybe remove undesired portions. There is a
very nice analogy with positive and negative sculpture. In negative sculpture, work proceeds
by removing unwanted material, as in the mathematician’s view of sculpting Michelangelo’s
David: start with a piece of marble, and simply chip away all non-David. Positive sculpture is
often done with a medium like wax. The central shape is approximated, and then wax is
either added or removed until the desired shape is attained.

Sem-VI, Software Testing(21IS63) Page 14 of 27

 Think about the consequences of a mistake: with negative sculpture, the whole work must be
thrown away, and restarted. (There is a museum in F orlence, Italy that contains half a dozen
such false starts to The David.) With positive sculpture, the erroneous part is simply removed
and replaced. The centrality of composition in the alternative models has a major implication
for integration testing.

4.4.2 Waterfall Spin-offs

There are three mainline derivatives of the waterfall model: incremental development,
evolutionary development, and the Spiral model [Boehm 88]. Each of these involves a series
of increments or builds, Within a build, the normal waterfall phases from detailed design
through testing occur, with one important difference: system testing is split into two steps,
regression and progression testing.

4.5 Specification Based Models

Two other variations are responses to the “complete understanding” problem. (Recall that
functional decomposition is successful only when the system is completely understood.) When
systems are not fully understood (by either the customer or the developer), functional
decomposition is perilous at best. The rapid prototyping life cycle deals with this by
drastically reducing the specification-to-customer feedback loop to produce very early
synthesis. Rather than build a final system, a “quick and dirty” prototype is built and then used
to elicit customer feedback. Depending on the feedback, more prototyping cycles may occur.
Once the developer and the customer agree that a prototype represents the desired system, the
developer goes ahead and builds to a correct specification. At this point, any of the waterfall
spin-offs might also be used.
An Object-Oriented Life Cycle Model
When software is developed with an object orientation, none of our life cycle models fit
very well. The main reasons: the object orientation is highly compositional in nature, and
there is dense interaction among the construction phases of object-oriented analysis, object-
oriented design, and object-oriented programming.

We could show this with pronounced feedback loops among waterfall phases, but the

Sem-VI, Software Testing(21IS63) Page 15 of 27

 fountain model [Henderson-Sellers 90] is a much more appropriate metaphor. In the fountain
model, the foundation is the requirements real world systems.

Fig 4.1 Fountain Model of Object-Oriented Software Development

As the object-oriented paradigm proceeds, details “bubble up” through specification, design,
and coding phases, but at each stage, some of the “flow” drops back to the previous phase(s).
This model captures the reality of the way people actually work (even with the traditional
approaches).
4.6 Formulations of the SATM System
In this and the next three chapters, we will relate our discussion to a higher level example, the
Simple Automatic Teller Machine (SATM) system. there are function buttons B1, B2, and
B3, a digit keypad with a cancel key, slots for printer receipts and ATM cards, and doors for
deposits and cash withdrawals. The SATM system is described here in two ways: with a
structured analysis approach, and with an object-oriented approach. These descriptions are
not complete, but they contain detail sufficient to illustrate the testing techniques under
discussion.

Sem-VI, Software Testing(21IS63) Page 16 of 27

4.6.1 SATM with Structured Analysis
The structured analysis approach to requirements specification is the most widely used
method in the world. It enjoys extensive CASE tool support as well as commercial training,
and is described in numerous texts. The technique is based on three complementary models:
function, data, and control. Here we use data flow diagrams for the functional models,
entity/relationship models for data, and finite state machine models for the control aspect of
the SATM system.

The functional and data models were drawn with the Deft CASE tool from Sybase Inc. That
tool identifies external devices (such as the terminal doors) with lower case letters, and
elements of the functional decomposition with numbers (such as 1.5 for theValidate Card
function). The open and filled arrowheads on flow arrows signify whether the flow item is
simple or compound. The portions of the SATM system shown here pertain generally to the
personal identification number (PIN) verification portion of the system.

Sem-VI, Software Testing(21IS63) Page 17 of 27

 Fig 4.2 Screens for the SATM System
The Deft CASE tool distinguishes between simple and compound flows, where compound
flows may be decomposed into other flows, which may themselves be compound. The
graphic appearance of this choice is that simple flows have filled arrowheads, while
compound flows have open arrowheads. As an example, the compound flow “screen” has
thefollowing decomposition:
Screen 1 Welcome
Screen 2 enter PIN
Screen 3 Wrong PIN
Screen 4 PIN failed, card retained
Screen 5 Select trans type
Screen 6 Select account type
Screen 7 Enter amount
Screen 8 Insufficient funds
Screen 9 Cannot dispense that amount
Screen 10 Cannot process withdrawals
Screen 11 Take your cash
Screen 12 Cannot process deposits
Screen 13 Put dep envelop in slot
Screen 14 Another transaction?
Screen 15 Thanks, take card and receipt.

Sem-VI, Software Testing(21IS63) Page 18 of 27

 As part of the specification and design process, each functional component is normally
expanded to show its inputs, outputs, and mechanism. We do this here with pseudo-code (or
PDL, for program design language) for three modules. This particular PDL is loosely based
on Pascal; the point of ny PDL is to communicate, not to develop something that can be
compiled. The main program description follows the finite state machine description.

State = AwaitCard
AmaitPIN:
ValidatePIN(PiNok, PAN)
IF PINok TEEN ScreenDriver(2, null)
State = AwaitTransELSE
ScreenDriver(4, null)
St at e = Amain Card
AmaitTrans:ManageTransaction
State = CloseSession
CloseSession:IF NewTransactionReguest
TEEN State = AmaitTrans
ELSE PrintReceipt
PoseTransaceionLocal
CloseSession
ControlCardRoller(eject)
State = AmaitCard
End, (CASE State)
END. (Main program SAD)

The ValidatePIN procedure is based on another finite state machine, in which states refer
to the number of PIN entry attempts.
Procedure ValidatePIN(PINok, PAN) GetPINforPAN(PAN, ExpectedPlN)
Try = First
CASE Try OF
First:ScreenDriver(2, null)
GetPIN(EnteredPIN)
IF EnteredPIN = ExpectedPlN THEN PiNok = TRUE
RETURN
ELSE ScreenDriver(3, null)
Try = Second
Second:ScreenDriver(2, null)
GetPIN(EneeredPIN)
IF EnteredPIN = ExpectedPlN THEN PiNok = TRUE
RETURN

Sem-VI, Software Testing(21IS63) Page 19 of 27

 Module Uses Modules
SATM Main WatchCardSlot
Control Card Roller
Screen Driver
Validate Card
Validate PIN
Manage Transaction
New Transaction Request
ValidatePlN GetPlNforPAN
Get PIN
Screen Driver
Get PIN KeySensor
Screen Driver

ELSE ScreenDriver(2, null)

Try =
ThirdThird:ScreenDriver(2,
null)
GetPIN(EneeredPIN)
IF EnteredPIN = ExpectedPlN THEN
P:Nok = TRUERETURN
ELSE ScreenDriver(4, null)
PINok = FALSE
END, (CASE Try)
END. (Procedure ValidatePIN)
Procedure GetPINIEnteredPIN, CancelHitl
Local Data: DigitKeys = (0, 1, 2, 2, 4, 5, 6, 7, 8, 9) BEGIN
CancelHit = FALSE
EnteredPIN = null string
DigitsRcvd=0
WHILE NOT(DigitsRcvd=4 OR
CancelHit) DOBEGIN
KeySensor(KeyHit)
IF KeyHit IN DigitKeys
TEEN BEGIN
EnteredPIN = EnteredPIN +
KeyHit
INCRENENT(DigitsRcv
d)
IF DigitsRcvd=1 THEN ScreenDriver(2,
'X-')IF DigitsRcvd=2 THEN
ScreenDriver(2, 'XX-')IF DigitsRcvd=2
THEN ScreenDriver(2, 'XXX-')IF
DigitsRcvd=4 THEN ScreenDriver(2,
'XXXXI)END
END (WHILE)
END. (Procedure GetPIN)

Sem-VI, Software Testing(21IS63) Page 20 of 27

 If we follow the pseudocode in these three modules, we can identify the “uses”
relationshipamong the modules in the functional decomposition.

4.7 Separating Integration and System Testing

We are almost in a position to make a clear distinction between integration and system
testing. We need this distinction to avoid gaps and redundancies across levels of testing,
to clarify appropriate goals for these levels, and to understand how to identify test cases
at different levels. This whole discussion is facilitated by a concept essential to all
levels of testing: the notion of a “thread”.

A thread is a construct that refers to execution time behavior; when we test a system,
we use test cases to select (and execute) threads. We can speak of levels of threads:
system threads describe system level behavior, integration threads correspond to
integration level behavior, and unit threads correspond to unit level behavior. Many
authors use the term, but few define it, and of those that do, the offered definitions
aren’t very helpful. For now, we take “thread” to be a primitive term, much like
function anddata.

In the next two chapters, we shall see that threads are most often recognized in terms of
the way systems are described and developed. For example, we might think of a thread
as a path through a finite state machine description of a system, or we might think of a
thread as something that is determined by a data context and a sequence of port level
input events, such as those in the context diagram of the SATM system. We could also
think of a thread as a sequence of source statements, or as a sequence of machine
instructions. The point is, threads are a generic concept, and they exist independently of
how a system is described and developed.

Sem-VI, Software Testing(21IS63) Page 21 of 27

 4.7.1 Structural Insights
Everyone agrees that there must be some distinction, and that integration testing is at a
more detailed level than system testing. There is also general agreement that integration
testing can safely assume that the units have been separately tested, and that, taken by
themselves, the units function correctly. One common view, therefore, is that
integration testing is concerned with the interfaces among the units. One possibility is
to fall back on the symmetries in the waterfall life cycle model, and say that integration
testing is concerned with preliminary design information, while system testing is at the
level of the requirements specification.

This is a popular academic view, but it begs an important question: how do we

discriminatebetween specification and preliminary design? The pat academic answer to
this is the what vs. how dichotomy: the requirements specification defines what, and
the preliminary design describes how. While this sounds good at first, it doesn’t stand
up well in practice. Some scholars argue that just the choice of a requirements
specification technique is a design choice The life cycle approach is echoed by
designers who often take a “Don’t Tread On Me” view of a requirements specification:
a requirements specification should neither predispose nor preclude a design option.
With this view, when information in a specification is so detailed that it “steps on the
designer’s toes”, the specification is too detailed. This sounds good, but it still doesn’t
yield an operational way to separate integration and system testing.

Sem-VI, Software Testing(21IS63) Page 22 of 27

 The models used in the development process provide some clues. If we follow the definition
of the SATM system, we could first postulate that system testing should make sure that all
fifteen display screens have been generated. (An output domain based, functional view of
system testing.) The entity/relationship model also helps: the one-to-one and one-to-many
relationships help us understand how much testing must be done. The control model (in this
case, a hierarchy of finite state machines) is the most helpful. We can postulate system test
cases in terms of paths through the finite state machine(s); doing this yields a system level
analog of structural testing. The functional models (dataflow diagrams and structure charts)
move in the direction of levels because both express a functional decomposition. Even with
this, we cannot look at a structure chart and identify where system testing ends and
integration testing starts. The best we can do with structural information is identify the
extremes. For instance, the following threads are all clearly at the system level:

1. Insertion of an invalid card. (this is probably the “shortest” system thread)

2. Insertion of a valid card, followed by three failed PIN entry attempts.

3. Insertion of a valid card, a correct PIN entry attempt, followed by a balance inquiry.

4. Insertion of a valid card, a correct PIN entry attempt, followed by a deposit.

5. Insertion of a valid card, a correct PIN entry attempt, followed by a withdrawal.

6. Insertion
of a valid card, a correct PIN entry attempt, followed by an attempt to withdraw
more cash than the account balance.

Sem-VI, Software Testing(21IS63) Page 23 of 27

4.8 Behavioral Insights
Here is a pragmatic, explicit distinction that has worked well in industrial applications. Think
about a system in terms of its port boundary, which is the location of system level inputs and
outputs.
Every system has a port boundary; the port boundary of the SATM system includes digit
keypad, the function buttons, the screen, the deposit and withdrawal doors, the card and
receipts lots, and so on. Each of these devices can be thought of as a “port”, and events occur
at system ports. The port input and output events are visible to the customer, and the
customer very often understands system behavior in terms of sequences of port events. Given
this, we mandate that system port events are the “primitives” of a system test case, that is, a
system test case (or equivalently, a system thread) is expressed as an interleaved sequence of
port input and port output events. This fits our understanding of a test case, in which we
specify pre-conditions, inputs, outputs, and post-conditions. With this mandate we can
always recognize a level violation: if a test case (thread) ever requires an input (or an output)
that is not visible at the port boundary, the test case cannot be a system level test case
(thread). Notice that this is clear, recognizable, and enforceable. We will refine this inChapter
14 when we discuss threads of system behavior.
4.8.1 Integration Testing
Craftspersons are recognized by two essential characteristics: they have a deep knowledge of the
tools of their trade, and they have a similar knowledge of the medium in which they work, so
that they understand their tools in terms of how they “work” with the medium. In Parts II and
III, we focused on the tools (techniques) available to the testing craftsperson. Our goal there
was to understand testing techniques in terms of their advantages and limitations with respect
to particular types of faults. Here we shift our emphasis to the medium, with the goal that a
better understanding of the medium will improve the testing craftsperson’s judgment.

A Closer Look at the SATM System

This decomposition is the basis for the usual view of integration testing. It is important to
remember that such a decomposition is primarily a packaging partition of the system. As

Sem-VI, Software Testing(21IS63) Page 24 of 27

 software design moves into more detail, the added information lets us refine the functional
decomposition tree into a unit calling graph. The unit calling graph is the directed graph in
which nodes are program units and edges correspond to program calls; that is, if unit A calls
unit B, there is a directed edge from node A to node B. We began the development of the call
graph for the SATM system in Chapter 12 when we examined the calls made by the main
program and the ValidatePIN and GetPIN modules. That information is captured in the
adjacency matrix given below in Table 2. This matrix was created witha spreadsheet; this
turns out to be a handy tool for testers.
Table 1 SATM Units and Abbreviated Names Unit Number
Level Number Unit Name
1 1 SATM System
A 1.1 Device Sense & Control
D 1.1.1 Door Sense & Control
2 1.1.1.1 Get Door Status
3 1.1.1.2 Control Door
4 1.1.1.3 Dispense Cash
E 1.1.2 Slot Sense & Control
5 1.1.2.1 WatchCardSlot
6 1.1.2.2 Get Deposit Slot Status
7 1.1.2.3 Control Card Roller
8 1.1.2.3 Control Envelope Roller
9 1.1.2.5 Read Card Strip

Sem-VI, Software Testing(21IS63) Page 25 of 27

 Some of the hierarchy is obscured to reduce the confusion in the drawing. One thing should be quite
obvious: drawings of call graphs do not scale up well. Both the drawings and the adjacency matrix
provide insights to the tester. Nodes with high degree will be important to integration testing, and paths
from the main program (node 1) to the sink nodes can be used to identify contents of builds for an
incremental development.

4.8.2 Decomposition Based Integration

Fig 4.3 SATM Functional Decomposition Tree

We can dispense with the big bang approach most easily: in this view of integration, all the units are
compiled together and tested at once. The drawback to this is that when (not if!) a failure is observed,
there are few clues to help isolate the location(s) of the fault.

4.8.3 Top-Down Integration

Top-down integration begins with the main program (the root of the tree). Any lower level unit that is
called by the main program appears as a “stub”, where stubs are pieces of throw-away code that emulate
a called unit. If we performed top-down integration testing for the SATM system, the first step would be
to develop stubs for all the units called by the main program: Watch Card Slot, Control Card Roller,
Screen Driver, Validate Card, Validate PIN, Manage Transaction

Once all the stubs for SATM main have been provided, we test the main program as if it were a stand-
alone unit. We could apply any of the appropriate functional and structural techniques, and look for
faults. When we are convinced that the main program logic is correct, we gradually replace stubs with
the actual code. Even this can be problematic. Would we replace all the stubs at once? If we did, we

Sem-VI, Software Testing(21IS63) Page 25 of 27

 Bottom-up Integration
Bottom-up integration is a “mirror image” to the top-down order, with the difference that stubs are
replaced by driver modules that emulate units at the next level up in the tree. In bottom-up integration,
we start with the leaves of the decomposition tree (units like ControlDoor and DispenseCash), and test
them with specially coded drivers. There is probably less throw-away code in drivers than there is in
stubs. Recall we had one stub for each child node in the decomposition tree. Most systems have a fairly
high fan-out near at the leaves, so in the bottom-up integration order, we won’t have as many drivers.
This is partially offset by the fact that the driver modules will be more complicated.
Sandwich Integration
Sandwich integration is a combination of top-down and bottom-up integration. If we think about it in
terms of the decomposition tree, we are really just doing big bang integration on a sub-tree

Call Graph Based Integration

One of the drawbacks of decomposition based integration is that the basis is the functional
decomposition tree. If we use the call graph instead, we mitigate this deficiency; we also move in the
direction of behavioral testing. We are in a position to enjoy the investment we made in the discussion of
graph theory. Since the call graph is a directed graph, why not use it the way we used program graphs?
This leads us to two new approaches to integration testing: we’ll refer to them as pair-wise integration
and neighborhood integration.
Pair-wise Integration
The idea behind pair-wise integration is to eliminate the stub/driver development effort. Rather than
develop tubs and/or drivers, why not use the actual code? At first, this sounds like big bang integration,
but we restrict a session to just a pair of units in the call graph. The end result is that we have one
integration test session for each edge in the call graph
Neighborhood Integration

Sem-VI, Software Testing(21IS63) Page 26 of 27

 the given node. Ina directed graph, this means all the immediate predecessor nodes and all the immediate
successor nodes (notice that these correspond to the set of stubs and drivers of the node). The eleven
neighborhoods for the SATM example (based on the call graph in Figure 4.2) are given in Table 3.

Table 3 SATM Neighborhoods Node

Predecessors Successors

1 1 9, 10, 12
6
1 1 11, 14, 18
7
1 17 14, 15
8
1 1 14, 15
9
23 22 14, 15

24 22 14,15

26 22 14, 15,6, 8, 2,3

27 22 14, 15,2, 3, 4,13

25 22 15

22 1 23, 24,26, 27,25

1 n/a 5, 7, 2,21, 16,17,19,22

We can always compute the number of neighborhoods for a given call graph. There will be one
neighborhood for each interior node, plus one extra in case there are leaf nodes connected directly to
the root node. (An interior node has a non-zero indegree and a non-zero outdegree.)
4.9 Path Based Integrations
Much of the progress in the development of mathematics comes from an elegant pattern: have a clear
idea of where you want to go, and then define the concepts that take you there. We do this here for path
based integration testing, but first we need to motivate the definitions.
When a unit executes, some path of source statements is traversed. Suppose that there is a call to another
unit along such a path: at that point, control is passed from the calling unit to the called unit, where some
other path of source statements is traversed. We cleverly ignored this situation in Part III, because this is
a better place to address the question. There are two possibilities: abandon the single- entry, single exit
precept and treat such calls as an exit followed by an entry, or “suppress” the call statement because
control eventually returns to the calling unit anyway. The suppression choice works well for unit testing,
Sem-VI, Software Testing(21IS63)
Page 27 of 27
 RV Institute of Technology and Management ®

but it is antithetical to integration testing. The first guideline for MM-Paths: points of quiescence are
“natural” endpoints for an MM-Path. Our second guideline also serves to distinguish integration from
system testing.
Our second guideline: atomic system functions are an upper limit for MM-Paths: we don’t want MM-
Paths to cross ASF boundaries. This means that ASFs represent the seam between integration and system
testing. They are the largest item to be tested by integration testing, and the smallest item for system
testing. We can test an ASF at both levels. Again, the digit entry ASF is a good example. During system
testing, the port input event is a physical key press that is detected by KeySensor and sent to GetPIN as a
string variable. (Notice that KeySensor performs the physical to logical transition.) GetPIN determines
whether a digit key or the cancel key was pressed, and responds accordingly. (Notice that button presses
are ignored.) The ASF terminates with either screen 2 or 4 being displayed. Rather than require system
keystrokes and visible screen displays, we could use a driver to provide these, and test the digit entry
ASF via integration testing. We can see this using our continuing example.

Sem-VI, Software Testing(21IS63) Page 28