STM Unit - 1
STM Unit - 1
STM Unit - 1
Testing consumes at least half of the time and work required to produce a
functional program
History reveals that even well written programs still have 1-3 bugs per
hundred statements
Productivity is measured by the sum of the costs of the material, the rework and the
discarded components and the cost of quality assurance and testing
There is a trade-off between quality assurance costs and manufacturing
costs
If sufficient time is not spent in quality assurance, the reject rare is high
and so is the net cost
If inspection is good and all errors are caught as they occur, inspection
costs dominates and again the net cost suffers
Testing and Quality assurance costs for manufactured items can be as low
as 2% in consumer products or as high as 80% in products such as
spaceships, nuclear reactiors and aircrafts, where failures threaten life.
Where as the amnufacturing cost of a software is trivial
The biggest part of software cost is the cost of bugs, the cost of detecting
them, the cost of correcting them, the cost of desinging tests that discover
them and the cost of running those tests
For software, quality and productivity are indistinguishable, because the
cost of a software copy is trivial
Testing and test design are parts of quality assurance that should also
focus on bug prevention
A prevented bug is better than a detected and corrected bug
Phases in testers mental life can be categorized into the following 5 phases:
Test Design
Design Style: While designing the software itself, adopting stylistic objectives
such as testability, openness and clarity can do much, to prevent bugs
Static Analysis Methods: It includes formal analysis of source code during
compilation, In earlier days, this is a routine job of the programmer, Now, the
compilers have taken over this job
Languages: The source language can help reduce certain kinds of bugs.
Programmers find new bugs while using new languages
Dichotomies:
Testing vs Debugging
Function vs Structure
Designer vs Tester
Modularity vs Efficiecy
Small vs Large
Builder vs Buyer
Testing vs Debugging
Testing Debugging
Testing starts with known conditions, uses Debugging starts from possibly unknown
predefined procedures and has predictable initial conditions and the end can not be
outcomes predicted except statistically
Testing can and should be planned, designed Procedure and duration of debugging cannot
and scheduled be so constrained
Much testing can be done without design Debugging is impossible without detailed
knowledge design knowledge
Much of test execution and design can be Automated debugging is still a dream.
automated
Function vs Structure
Function Structure
It takes the user point of view-bother about It looks at the implementation details
functionality and features and not the
programs implementation
Both structural and functional tests are useful, both have limitations, and
both target different kinds of bugs
Functional tests can detect all bugs but would take massive time to do so
Structural tests are inherently finite, but cannot detect all errors, even if
completely executed
Designer vs Tester
Test designer is the person who designs the test, whereas the tester is the
one who actually tests the code
During functional testing, the designer and tester are probably different
persons. During unit testing, the tester and the programmer merge into
one person
Tests that are designed and executed by software designers are more
likely inclined to structural considerations
Structural testing restricts these tests to convey its meaning correctly in a
simple lucid (easy) manner
Modularity vs Efficiency
Small vs Large
Most of the software's are written and used by the same organizations
Unfortunately, this situation id dishonest, because it clouds accountability
If there is no separation between builder and buyer, there can be no
accountability
Unit/Component Testing
Integration Testing
System Testing
Unit/Component Testing
Integration testing
System Testing
Consequences of Bugs
The consequences of a bug can be measured in terms of human, rather than
machine, some consequences of a bug on a scale of one to ten are:
1. Mild: The symptoms of the bug offend us gently; a misspelled output
or a misaligned printout.
2. Moderate: Outputs are misleading or redundant. The bug impacts the
systems performance.
5. Serious: It loses track of its transaction. Not just the transaction itself
but the fact that the transaction occurred and accountability is lost.
6. Very Serious: The bug causes the system to do the wrong transactions.
Instead of gaining a pay check, the system credits it to another account or
converts deposits to withdrawals.
10. Infectious: What can be worse than a failed system? One that corrupts
other systems even though it does not fail in itself; that erodes the social
physical environment; that melts reactors and starts a war.
Taxonomy of Bugs
Path Testing is a method that is used to design the test cases. In path testing
method, the control flow graph of a program is designed to find a set of linearly
independent paths of execution. In this method Cyclomatic Complexity is used
to determine the number of linearly independent paths and then test cases are
generated for each path.
It gives complete branch coverage but achieves that without covering all
possible paths of the control flow graph. McCabe’s Cyclomatic Complexity is
used in path testing. It is a structural testing method that uses the source code of
a program to find every possible executable path.
Path Testing Process:
Make Set:
Make a set of all the path according to the control flow graph and
calculated cyclomatic complexity. The cardinality of set is equal to the
calculated cyclomatic complexity.
Independent paths:
Independent path is a path through a Decision to Decision path graph
which cannot be reproduced from other paths by other methods.
The specifications that don't have flaws in them may change while the design is in progress.
The features are added, modified and deleted.
The range is from a few percentage to more than 50%, depending on the application and
environment.
What hurts most about the bugs is that they are the earliest to invade the system and the last
to leave
2. Feature Bugs:
Removing the features might complicate the software, consume more resources, and foster
more bugs.
Providing correct, clear, implementable and testable feature specifications is not enough.
Features usually come in groups or related features. The features of each group and the
interaction of features with in the group are usually well tested.
Every application has its peculiar set of features and a much bigger set of unspecified feature
interaction potentials and there fore result in feature interaction bugs.
STRUCTURAL BUGS:
Various categories in Structural bugs include:
Control and sequence bugs include paths left out, unreachable code, improper nesting of
loops, loop-back or loop termination criteria incorrect, missing process steps, duplicated
processing, unnecessary processing, rampaging, GOTO's, ill-conceived (not properly
planned) switches, sphagetti code, and worst of all, pachinko code.
One reason for control flow bugs is that this area is amenable (supportive) to theoretical
treatment.
Most of the control flow bugs are easily tested and caught inunit testing.
Another reason for control flow bugs is that use of old code especially ALP &
COBOL code are dominated by control flow bugs.
Control and sequence bugs at all levels are caught by testing, especially structural testing,
more specifically path testing combined with a bottom line functional test based on a
specification.
DATA BUGS:
Data bugs include all bugs that arise from the specification of data objects, their formats, the
number of such objects, and their initial values.
Data Bugs are atleast as common as bugs in code, but they arefoten treated as if they did not
exist at all.
Because of this, there is an increasing awareness that bugs in code are only half the battle
and the data problems should be given equal attention.
CODING BUGS:
Coding errors of all kinds can create any of the other kind of bugs.
Syntax errors are generally not important in the scheme of things if the source language
translator has adequate syntax checking.
If a program has many syntax errors, then we should expect many logic and coding bugs.
The documentation bugs are also considered as coding bugs which may mislead the
maintenance programmers.
1. External Interfaces:
The external interfaces are the means used to communicate with the world.
These include devices, actuators, sensors, input terminals, printers, and communication
lines.
The primary design criterion for an interface without side world should be robustness.
All external interfaces, human or machine should employ a protocol. The protocol
may be wrong or incorrectly implemented.
Other external interface bugs are: invalid timing or sequence assumptions related to external
signals
2. Internal Interfaces:
Internal interfaces are in principle not different from external interfaces but they are more
controlled.
The external environment is fixed and the system must adapt to it but the internal
environment, which consists of interfaces with other components, can be negotiated.
3. Hardware Architecture:
Bugs related to hardware architecture originate mostly from misunderstanding how the
hardware works.
Examples of hardware architecture bugs: address generation error, i/o device operation /
instruction error, waiting too long for a response, incorrect interrupt handling etc.
The remedy for hardware architecture and interface problems is two fold: (1) Good
Programming and
Use operating system interface specialists, and use explicit interface modules or macros for
all operating system calls.
This approach may not eliminate the bugs but at least will localize them and make testing
easier.
Integration Bugs:
Integration bugs are bugs having to do with the integration of, and with the interfaces
between, working and tested components.
The communication methods include data structures, call sequences, registers, semaphores,
communication links and protocols results in integration bugs.
The integration bugs do not constitute a big bug category(9%) they are expensive category
because they are usually caught late in the game and because they force changes in several
components and/or data structures.9.
System Bugs:
System bugs covering all kinds of bugs that cannot be ascribed to a component or to their
simple interactions, but result from the totality of interactions between many components
such as programs, data, hardware, and the operating systems.
There can be no meaningful system testing until there has been thorough component and
integration testing.
System bugs are infrequent(1.7%) but very important because they are often found only
after the system has been fielded.
Testing: testers have no immunity to bugs. Tests require complicated scenarios and
databases.
They require code or the equivalent to execute and consequently they can have bugs.
Test criteria: if the specification is correct, it is correctly interpreted and implemented, and a
proper test has been designed; but the criterion by which the software's behaviour is judged
may be incorrect or impossible. So, a proper test criteria has to be designed. The more
complicated the criteria, the likelier they are to have bugs.
Remedies:
1. Test Debugging:
The first remedy for test bugs istesting and debugging the tests. Test debugging, when
compared to program debugging, is easier because tests, when properly designed are
simpler than programs and donot have to make concessions to efficiency.
Programmers have the right to ask how quality in independent testing ismonitored.
The history of software bug removal and prevention is indistinguishable from the history of
programming automation aids. Assemblers, loaders, compilers are developed to
reduce the incidence of programming and operation errors. Test execution bugs are virtually
eliminated byvarious test execution automation tools.
Just as much of software development has been automated, much test design can be and has
been automated. For a given productivity rate, automation reduces the bug count -be it for
software or be it for tests.
Sequential Statements:
If – Then – Else:
Do – While:
While – Do:
Switch – Case:
Advantages of Basic Path Testing
Path Predicate:
AND
"W is either negative or equal to 10" is true
Multiway Branches:
The path taken through a multiway branch such as computed GOTO, case
statement, or jump tables cannot be directly expressed in TRUE/FALSE terms.
Each product term denotes a set of inequalities, that if solved will yield an
input vector that will drive the routine along the designated path
Solve any one of the inequality sets for the chosen path, and you have
found a set of input values for the path
If you can find a solution, the the path is achievable
If you cannot find a solution, then the path is achievable
If you cannot find a solution to any of the sets of inequalities, the path is
not achievable
The act of finding a set of solutions to the path predicate expression is
called path sensitization
Path Instrumentation:
Path Instrumentation is 4 types as follows:
Interpretive Trace Program
Traversal Marker or Link Marker
Two Link Marker Method
Link Counter
The solution to the problem of single link marker method is to implement two
markers per link, one at the beginning of each link and one at the end
The two link markers now specify the path name and confirm both the
beginning and end of the link
Link Counter
Instead of a unique link name to be pushed into a string when the link traversed,
we simply increment a link counter, we now confirm that the path length is
expected
The same problem that led tus to double link markers also leads us to double
link counters
Path testing methods are mainly used in unit testing, especially for new software
The new component is first tested as an independent unit with all called
components and co-requisite components replaced by stubs. A simulator of low-
level components that is more reliable than the actual component
Note:
New Code:
Typically, we will try to use the shortest entry/exit path that will do the task
Maintenance:
There is a great difference between maintenance testing and new code testing
Maintenance testing is a completely different situation
Rehosting:
Path testing with C1+C2 coverage is a powerful tool for rehosting old software
The objective of rehosting is to change the operating environment and not the
rehosted software
Rehosting from one COBOL environment to another is easy by comparison
The test suites(collection) and all outcomes of the old environment become the
specification for the rehosted software