Chapter 3 Software Testing Methods v1.1
Chapter 3 Software Testing Methods v1.1
Chapter 3 Software Testing Methods v1.1
Page 1 of 25
Confidentiality Statement
This document should not be carried outside the physical and virtual boundaries of TCS and
its client work locations. Sharing this document with any person other than a TCS associate
would tantamount to violation of confidentiality agreement signed by you while joining
TCS.
Notice
The information given in this course material is merely for reference. Certain third party
terminologies or matter that may be appearing in the course are used only for contextual
identification and explanation, without an intention to infringe.
Certificate in Software Testing Skill TCS Business Domain Academy
Contents
Chapter-3 Software Testing Methods .............................................................................4
3.1 Introduction to Static and Dynamic testing ........................................................... 5
3.2 Static Testing in detail ........................................................................................... 7
3.3 Box Approaches of Testing .................................................................................. 15
Summary ........................................................................................................................ 23
Page 3 of 25
Certificate in Software Testing Skill TCS Business Domain Academy
Introduction
There are many approaches available in software testing. Testing the software by
performing a manual inspection, review or walkthrough is called Static Testing, whereas
executing programmed code with a given set of test cases is called dynamic testing.
Learning Objective
After reading this chapter, you will be able to understand:
• Static Testing
• Dynamic Testing
• Difference between Static and Dynamic Testing
• Static Testing Methods and technologies
• List of static testing tools language wise
• Black Box Testing & its techniques
• White Box Testing & its techniques
• Grey Box Testing
Page 4 of 25
Certificate in Software Testing Skill TCS Business Domain Academy
The Verification activities into the category of Static Testing. During static testing, you have
a checklist to check whether the work you are doing is going as per set standards of the
organization or not. These standards can be for Coding, Integrating and Deployment.
Reviews, Inspection's and Walkthrough's are static testing methodologies.
Static testing is carried out without executing the application under test whereas dynamic
testing requires one or more executions of application. Static testing is useful in discovery of
faults in the application, ambiguities and errors in requirements and other application
related documents, at relatively very low cost then dynamic expensive testing. While doing
static testing we can find Unreachable Code, Undeclared values, Syntax errors using its
methods and technology for finding defects.
Static testing is accompanied out by an individual who did not write code or by a team of
individuals. A sample process of static testing is shown in below figure. The test team
responsible for static testing has access to requirements document, application and all
other application related documents such as design documents and user manuals and also
have access of static testing tools. A static testing tools takes application code as input and
generates a variety of data useful in test process as output.
Page 5 of 25
Certificate in Software Testing Skill TCS Business Domain Academy
Dynamic testing takes place during run time. Dynamic testing may begin before the
program is fully (100 %) completed in order to test particular sections of code and are
applied to distinct functions or modules. Classic techniques for this are either using
stubs/drivers or execution from a debugger environment.
Dynamic Testing contains working with the software, giving input values and checking if the
output is as expected. These are the Validation activities. Unit Tests, Integration Tests,
System Tests and Acceptance Tests are few of the Dynamic Testing methodologies.
Tester may muddy between static and dynamic testing methods as both of them have
similar purposes. There are number of visible differences between static and dynamic
testing.
The simplest difference is that static testing evaluates the software when it is idle, while
dynamic testing operates the software to see if it functions properly.
Static testing looks for bugs/defects in the software when it is in an inactive state while
dynamic software checks for errors when the software is running or under execution. So
each of these tests have an approach that is unique from each other but will eventually help
the software to have the functionality it should have.
Static and dynamic testing accompaniment each other because they have similar goal of
searching for bugs and errors in the application. Static testing has the ability to help the
Page 6 of 25
Certificate in Software Testing Skill TCS Business Domain Academy
Static testing involves verification, whereas dynamic testing involves validation. And
together they help improve software quality. Among the techniques for static analysis,
mutation testing can be used to ensure the test-cases will detect errors which are
introduced by mutating the source code.
In this section we will know about Static Testing Methods and technologies, Goal and List of
static testing tools.
Reviews, Inspection's and Walkthrough's are static testing methodologies. Static testing
could be manual or automated.
Inspection
It is a technique in which the work product is examined for its compliance to specific
standards and checked against a history of common errors.
Inspection is a more appropriately defined process than a walkthrough and usually is
associated with code. Several organizations following formal code inspections to improve
the code quality at a lower cost than suffered when dynamic testing is used. Organization
experienced substantial increases in productivity and software quality due to code
inspections.
Code inspection carried out by experienced team members and works according to an
inspection plan which consists of following elements.
• Statement of purpose
• Work product to be inspected, this includes code and associated documents needed
for inspection
• Team formation, roles, and tasks to be performed
• Rate at which the inspection task is to be completed
• Data collection forms where the team will record its findings such as defects
discovered, coding standard violations, and time spent in each task.
Page 7 of 25
Certificate in Software Testing Skill TCS Business Domain Academy
Members of the inspection team are assigned roles of moderator, reader, author and
recorder. The moderator is responsible of the process and leads the review. Actual code is
read by the reader, possibly with the help of a code browser and with large monitors for all
in the team to view the code. The recorder records any issues or errors discovered to be
looked into. The author is the actual developer whose main task is to help others in
understanding code. It is important that the inspection process be friendly and non-
confrontational.
Review
It is a technique in which the work product is discussed by a group of two or more persons
and re-examined for possible corrections. In other words review is an organised
examination of a document by one or more people with the aim of finding and removing
errors early in the SDLC. Reviews are used to verify documents like requirement docs,
system designs, code, test plans and test cases.
Review processes can differ usually in their levels of formality, where formality relates to
the level of structure and documentation associated with the activity. Some types of review
are fully informal while others are formal. Planning, Kick-off, Individual preparation, Review
Meeting, Rework and follow up are phase of formal review.
Page 8 of 25
Certificate in Software Testing Skill TCS Business Domain Academy
The decision on the appropriate level for a review is generally based on combinations of the
following factors.
• Maturity of the development process. The more mature the process tend more
mature reviews.
• Regulatory or legal requirements. These are used to govern the software
development activities in certain industries, particularly in safety-critical areas such
as railway signalling, and determine what kinds of review should take place.
• Need of audit trail. Formal review processes ensure that is it is possible to trace
backwards throughout the SDLC. Level of formalities in the types of review can help
to raise the level of audit trial.
Reviewer can have many review objective and this identifies main focus of review. Usually
the review object include.
• Finding defects.
• Gaining understanding.
• Generating discussion.
• Decision making by consensus.
All reviews (Formal and informal) exhibit the same basic elements of process.
Page 9 of 25
Certificate in Software Testing Skill TCS Business Domain Academy
The role of each reviewer is to look at documents belonging to them from their assigned
perspective, this may include the use of checklists. In addition to assigned review roles, the
review process itself define specific roles and responsibilities that should be fulfilled for
formal review.
• Manager: The manager decides on what is to be reviewed (if not already defined),
ensures there is sufficient time allocated in the project plan for all of the required
review activities and also determines if the review objectives have been met.
Managers do not usually get involved in the actual review process unless they can
add real value, for example they have technical knowledge key to the review.
• Moderator: The moderator is sometimes known as the review leader. This is the
person who leads the reviews of the document or set of documents, including
planning the review, running the meeting, and follow-ups after the meeting. If
necessary, the moderator may mediate between the various points of view and is
often the person on whom the success of the review rests. The moderator will also
make the final decision as to whether to release an updated document.
Page 10 of 25
Certificate in Software Testing Skill TCS Business Domain Academy
• Author: Author is the writer or person with chief responsibility for the development
of the document(s) to be reviewed. The author will in most instances also take
responsibility for fixing any agreed defects.
• Reviewers: These are individuals with a specific technical or business background
(called checkers or inspectors). As discussed above, reviewers should be chosen to
represent different perspectives and roles in the review process and take part in any
review meetings.
• Scribe (or recorder): The scribe attends all review meetings and documents all of
the issues and defects, problems and open points that were identified during the
meeting.
The following suggested success factors should be considered while measuring the success
of a review.
• Each review should have a clearly predefined and agreed objective and the right
people should be involved to ensure that the objective is met. For example, in an
inspection, if the objective is technically based, each reviewer will have a defined
role and have the experience to complete the technical review, this should reflect
testers as valued reviewers.
• Any defects found are welcomed, and expressed objectively.
• The review should be seen as being conducted within an atmosphere of trust, so
that the output will not be used for the evaluation of the participants, and that the
people issues and psychological aspects are dealt with (e.g. making it a positive
experience for the author and all participants).
• Review techniques (both formal and informal) that are suitable to the type and level
of software work-products and reviewers.
• Checklists or roles should be used, where suitable, to increase effectiveness of
defect identification. For example, in an inspection, roles such as data entry clerk or
technical architect may be required to review a specific document.
• Management support is essential for a good review process (e.g. by incorporating
adequate time for review activities in project schedules).
• There should be an emphasis on learning and process improvement.
Page 11 of 25
Certificate in Software Testing Skill TCS Business Domain Academy
Walkthrough
It is a technique mostly done on the code developed, where the code is outlined manually to
monitor the state of the program variables as a way of analysing the logic.
Walkthroughs and inspections are an integral part of static testing. Walkthrough is a casual
process to review any application-related document. For example, requirements are
reviewed using a process termed requirements-walkthrough. Code is reviewed using code-
walkthrough, also known as peer code review. A detailed report contains list of concerned
items regarding the document reviewed.
While requirement walkthrough, the test team must review the requirements document to
ensure that the requirements matches the user expectation and from inconsistencies and
ambiguities. Requirements review process also improves the understanding of the test
team regarding what is preferred of the application. Both functional and non-functional
requirements are reviewed and a list of items of concern regarding requirement is
generated in a detailed report. The Key Characteristics of Walk Through are Scenario, Dry
Run and Peer Group.
Static analysis tools add the highest value when used during component and integration
testing and will normally involves their use by developers to check against predefined rules
or standards, and by designers during software modelling.
A static analysis tools runs automatically and reports all defects it recognises, few of which
may be insignificant and require small work to correct, while other could be critical and
need urgent correction. These defects therefore require strong planning to ensure that the
full benefit is obtained from using the tool in priority.
Page 12 of 25
Certificate in Software Testing Skill TCS Business Domain Academy
The purpose of both validation and verifications in software testing are nor similar. For
example validation ensures that the development of precise software by enabling the
developers to track back through the system to meet particular user requirements.
Meanwhile the purpose of the validation is to ensure that the software will pass a particular
phase during the time of development whereas verification means ensuring that the
software development is happening by using the right method.
Page 13 of 25
Certificate in Software Testing Skill TCS Business Domain Academy
Static testing also ensures the software quality of the product with without executing the
application using various methods. In software development, the core definition of quality is
subjective as different software applications have different purposes and goals. There is
three rules that the software needs to pass in order to considered as a quality software, the
failure rate of the software in actual use, and of course client satisfaction.
Below are some issues that will help you determine whether the software under testing can
pass the quality assurance or not.
• Software encounters nominal failures in testing and also during actual use.
• Software should be reliable means that the software needs to demonstrate
extended.
• The final consideration is the satisfaction of the end-users. This is itself an indication
of whether the software is of good quality or not.
Below are list of some popular static analysis tools language wise.
• Multiple language : Coverity Code Advisor, Black Duck Software Suite, CAST
Application Intelligence Platform, Cigital SecureAssist, ConQAT, HP Fortify
Software Static Code Analyser, IBM Rational AppScan Source Edition etc..
• Dot Net: .NET Compiler Platform, StyleCop, FxCop, CodeRush, CodeIt.Right,
Parasoft dotTEST, etc.
• Java: Jtest, ThreadSafe, FindBugs, Checkstyle, PMD, JArchitect, SourceMeter,
Squale, soot etc.
• Java Scripts: JSHint, ESLint, JSCS, JSHint, JSLint, Plato etc...
• C/C++: Parasoft C/C++test, BLAST, Astree, Clang, Eclipse, Frama-C, ECLAIR,
Polyspace, PVS-Studio, Splint etc…
• PHP: RIPS
• PowerBuilder: PB Code Analyser, Visual Expert for PowerBuilder etc…
• PL/SQL: TOAD, Visual Expert for PL/SQL etc…
• Python: PyCharm, Pylint, Quantified Code etc…
• Perl: Padre, PerlTidy etc.
• Objective-C : Clang
Page 14 of 25
Certificate in Software Testing Skill TCS Business Domain Academy
White box testing is also known as clear box testing, Glass box testing, Transport box
testing, or Structure testing and is also a method of testing software that test internal
structures or workings of an applications. In white-box testing an internal perspective of the
system, as well as programming skills, are required and used to design test cases. The tester
chooses inputs to exercise paths through the code and determine the appropriate outputs.
It can be applied at unit, integration and system levels of software testing process. It is
designed to expose many error and problem and might not detect unimplemented parts of
the specification or missing requirements.
White box testing completely trusts on the identified structure of the software as we can
see in the following examples
• Component level: the structure is that of the code itself, i.e. statements, decisions
or branches
Page 15 of 25
Certificate in Software Testing Skill TCS Business Domain Academy
• Integration level: the structure may be a call tree (a diagram in which modules call
other modules). Modules are checked for proper and correct integration with the
other modules
• System level: the structure may be a menu structure, business process or web page
structure
White box techniques include
• Data flow testing
• Decision testing and coverage
• Path testing
• Statement testing and coverage
Path testing
• In component testing, statement coverage is the assessment of the percentage of
executable statements that have been exercised by a test case suite.
• It indicates the total number of statements being executed.
• Statement testing derives test cases to execute specific statements, normally to
increase statement coverage.
Page 16 of 25
Certificate in Software Testing Skill TCS Business Domain Academy
• Since tests can be very complex, highly skilled resources are required, with
thorough knowledge of programming and implementation.
• Test script maintenance can be a burden if the implementation changes too
frequently.
• Since this method of testing it closely tied with the application being testing, tools
to cater to every kind of implementation/platform may not be readily available.
Page 17 of 25
Certificate in Software Testing Skill TCS Business Domain Academy
Black Box Testing is also known as Behavioural Testing, is a software testing method in
which the internal structure/ design/ implementation of the item being tested is not known
to the tester. These tests can be functional or non-functional, though usually functional.
Black box testing method that tests the functionality of an application as opposed to its
internal structures or workings. Specific knowledge of the application's code/internal
structure and programming knowledge in general is not required. Test cases are built
around specifications and requirements, i.e., what the application is supposed to do. It uses
external descriptions of the software, including specifications, requirements, and design to
derive test cases. These tests can be functional or non-functional, though usually functional.
The test designer selects valid and invalid inputs and determines the correct output. There
is no knowledge of the test object's internal structure.
It typically comprises most if not all testing at higher levels, but can also dominate unit
testing as well. It can be applied at all levels of software testing
In case of Requirements based technique (RBT), Models, either formal or informal, are used
for the specification of the problem to be solved, the software or its components. From
these models test cases can be derived systematically
Page 18 of 25
Certificate in Software Testing Skill TCS Business Domain Academy
Black Box testing method is applicable to the Integration, System and Acceptance levels of
software testing. The higher the level, and hence the bigger and more complex the box, the
more black box testing method comes into use.
• Here Test cases are derived in order to execute the combination of inputs.
• Decision tables are a good way to capture system requirements that contain logical
conditions, and to document internal system design.
• They may also be used to record complex business rules that a system is to
implement. The specification is analysed, and conditions and actions of the system
are identified.
• The input conditions and actions are most often stated in such a way that they can
either be true or false (Boolean).
• The decision table contains the triggering conditions, often combinations of true
and false for all input conditions, and the resulting actions for each combination of
conditions.
• Each column of the table corresponds to a business rule that defines a unique
combination of conditions that result in the execution of the actions associated with
that rule.
• The coverage standard commonly used with decision table testing is to have at
least one test per column, which typically involves covering all combinations of
triggering conditions.
• The strength of decision table testing is that it creates combinations of conditions
that might not otherwise have been exercised during testing. It may be applied to
all situations when the action of the software depends on several logical decisions
• The four quadrants of decision table are
Page 19 of 25
Certificate in Software Testing Skill TCS Business Domain Academy
Page 20 of 25
Certificate in Software Testing Skill TCS Business Domain Academy
• Tests can be designed to cover a typical sequence of states, to cover every state, to
exercise every transition, to exercise specific sequences of transitions or to test
invalid transitions
• This technique is suitable for modelling a business object having specific states or
testing screen-dialogue flows (e.g. for internet applications or business scenarios)
Equivalence partitioning
• This technique divides the input data of a software unit into partitions of data from
which test cases can be derived. In principle, test cases are designed to cover each
partition at least once
• This technique tries to define test cases that uncover classes of errors, thereby
reducing the total number of test cases that must be developed
• In rare cases equivalence partitioning is also applied to outputs of a software
component, typically it is applied to the inputs of a tested component
• The equivalence partitions are usually derived from the requirements specification
for input attributes that influence the processing of the test object. An input has
certain ranges which are valid and other ranges which are invalid
• Invalid data here does not mean that the data is incorrect, it means that this data
lies outside of specific partition
Page 21 of 25
Certificate in Software Testing Skill TCS Business Domain Academy
Grey box testing is a method to test applications with a limited knowledge of the internal
workings of an application. Mastering the domain of the system always gives the tester an
advantage over someone with limited domain knowledge.
Unlike black box testing, where the tester only tests the applications user interface, in grey
box testing, the tester has access to design documents and database.
Advantages of Grey Box
• Offers combined benefits of black-box and white-box testing wherever possible.
• Grey box testers don't rely on the source code; instead they rely on interface
definition and functional specifications.
• Based on the limited information available, a grey-box tester can design excellent
test scenarios especially around communication protocols and data type handling.
• The test is done from the point of view of the user and not the designer.
Page 22 of 25
Certificate in Software Testing Skill TCS Business Domain Academy
Summary
Page 23 of 25
Certificate in Software Testing Skill TCS Business Domain Academy
• Black box testing method that tests the functionality of an application as opposed
to its internal structures or workings. Specific knowledge of the application's
code/internal structure and programming knowledge in general is not required.
• Grey box testing is a method to test the application with a limited knowledge of the
internal workings of an application. Unlike black box testing, where the tester only
tests the applications user interface, in grey box testing, the tester has access to
design documents and database.
Page 24 of 25