SOFTWARE TESTING

UNIT III
Test Management

Presented by
M S RATHOD
LIF, GPA
Unit Outcomes
Apply specified testing level for a web
based application.
Apply Acceptance testing for given web
based application.
Apply the given performance testing for
an application.
Generate test cases for the given
application using regression and GUI
testing.
Overview

1) Test Planning

2) Test Management

3) Test Process

4) Test Reporting
Overview

1) Test Planning

2) Test Management
Bank Website

3) Test Process

4) Test Reporting
Test Planning
1) Preparing a Test Plan
2) Scope Management
3) Deciding Test Approach
4) Setting up criteria for testing
5) Identifying Responsibilities
6) Staffing
7) Training needs
8) Resource requirements
9) Test Deliverables
10) Testing Tasks
Preparing a Test Plan
A Test Plan is a detailed document that
describes the test strategy, objectives, schedule,
estimation, deliverables, and resources required
to perform testing for a software product.
Test plan acts as anchor for the
execution ,tracking, and reporting of the entire
testing projects.
Test Plan helps us determine the effort needed
to validate the quality of the application under
test.
Preparing a Test Plan
It covers following points.
1) What needs to be tested?
2) How the testing is going to be performed?
3) What resources are needed for testing?
4) The time lines by which the testing
activities will be performed.
5) Risks that may be faced in all of the
above, with appropriate mitigation and
contingency plans
Scope Management
Before the start of any test activity, scope
of the testing should be known. You must
think hard about it.
The components of the system to be
tested (hardware, software, middleware,
etc.) are defined as “in scope“
The components of the system that will
not be tested also need to be clearly
defined as being “out of scope.”
Scope Management
For example, a website only focus on
testing all the functions and external
interface of website (in scope)
Nonfunctional testing such
as stress, performance or logical
database currently will not be tested. (out
of scope)
Scope Management
Understanding what constitutes a release
of a product;
2)Breaking down the release into
features;
3)Prioritizing the features for testing;
4)Deciding which features will be tested
and which will not be; and
5)Gathering details to prepare for
estimation of resources for testing.
Scope Management
Choice & Prioritization of features to be
tested:
1)Features that are new and critical for the
release.
2)Features whose failures can be
catastrophic.
3)Features that are expected to be complex to
test.
4)Features which are extensions of earlier
features that have been defect prone.
Deciding Test Approach
What type of testing would you use for
testing the functionality?
What are the configurations or scenarios
for testing the features?
What integration testing would you do to
ensure these features work together?
What “non-functional” tests would you
need to do?
Setting up criteria for testing
 Encountering more than a certain number of defects,
causing frequent stoppage of testing activity;
 Hitting show stoppers that prevent further progress
of testing ( for example, if a database does not start,
further tests of query , data manipulation, and so on
are is simply not possible to execute); and
 Developers releasing a new version which they
advice should be used in instead of the product under
test( because of some critical defect fixes. )
 When such conditions are addressed, the tests can
resume..
Identifying Responsibilities
Ensure there is clear accountability for a given
task, so that each person knows what he or she
has to do;
Clearly list the responsibilities for various
functions to various people, so that everyone
know how his or her work fits into the entire
project;
Complement each other, ensuring no one steps
on an others toes; and
Supplement each other, so that no task is left
unassigned
Training needs
It may not always be possible to find the
perfect fit between the requirement and the
skills available .
In case there are gaps between the
requirements and availability of skills, they
should be addressed with appropriate
training programs.
It is important to plan for such training
programs upfront as they are usually are
de-prioritized under project pressures.
Resource requirements
Resource could be human, equipment and
materials needed to complete a project.
Human Resource
System Resource
Documents
 Human Resource
No. Member Tasks
Manage the whole project
1. Test Manager Define project directions
Acquire appropriate resources
Identifying and describing appropriate test techniques/tools/automation
architecture
Verify and assess the Test Approach
Execute the tests, Log results, Report the defects.
2. Tester
Tester could be in-sourced or out-sourced members, base on the project
budget
For the task which required low skill, I recommend you
choose outsourced members to save project cost.
3. Developer in Test Implement the test cases, test program, test suite etc.
Builds up and ensures Test Environment and assets
4. Test Administrator are managed and maintained
SupportTester to use the test environment for test execution
Take in charge of quality assurance
5. SQA members Check to confirm whether the testing process is meeting specified
requirements
 System Resource
No. Resources Descriptions

Install the web application under test

1. Server This includes a separate web server, database server, and application
server if applicable

The testing tool is to automate the testing, simulate the user operation,
generate the test results
2. Test tool
There are tons of test tools you can use for this project such as Selenium,
QTP…etc.

You need a Network include LAN and Internet to simulate the real
3. Network
business and user environment

4. Computer The PC which users often use to connect the web server
Test Deliverables
Test Deliverables is a list of all the
documents, tools and other components
that has to be developed and maintained
in support of the testing effort.
There are different test deliverables at
every phase of the software development
lifecycle.
 Test Deliverables
Test deliverables are provided before testing phase.
 Test plans document.
 Test cases documents
 Test Design specifications.

Test deliverables are provided during the testing

 Test Scripts
 Simulators.
 Test Data
 Test Traceability Matrix
 Error logs and execution logs.

Test deliverables are provided after the testing cycles is over.

 Test Results/reports
 Defect Report
 Installation/ Test procedures guidelines
 Release notes
Test Management
Test infrastructure
management
Test people management
Test Infrastructure Management
Testing requires a robust infrastructure to
be planned upfront.
This infrastructure is made up of three
essential elements:
1)A Test Case Database (TCDB)
2)A Defect Repository (DR)
3)Software Configuration Management
repository and tool. (SCM)
 A Test Case Database
Entity Purpose Attributes

Test Case Records all the ‘static’ information about the tests. • Test Case ID
• Test case name (file
name)
• Test case owner
• Associated files for the test
case.

Test Case : Provides a mapping between the tests and the • Test Case ID
Product corresponding product features; enables identification • Module ID
Cross- of tests for a given feature.
reference
Test Case Run Gives the history of when a test was run and what • Test Case ID
history was the result; provides inputs on selection of tests • Run date
for regression runs. • Time taken
• Run Status(success/failure)

Test Case: Gives details of test cases introduced to test certain • Test Case ID
Defect specific defects detected in the product; provides • Defect Reference # (points to a
Cross- inputs on the selection of tests for regression runs. record in the defect
reference repository.)

2
A Defect Repository
A defect repository captures all the relevant
details of defects reported for a product .
The defect repository is an important vehicle
of communication that influence the work flow
within a software organization.
It also provides the data in arriving at several
of the metrics.
Most of the metrics classified as testing defects
metrics and development defect metrics are
derived out of the data in defect repository.
 A Defect Repository
Entity Purpose Attributes

Defects details Records all the “static” information about the tests. • Defect ID
• Defect priority/ severity
• Defect Description
• Affected product.

Defect Provide details of test cases for a given defect. • Defect ID

test Cross-reference the TCDB. • Test Case ID
details

Fix Details Providers details of fixes for a given defect; cross- • Defect ID
references the configuration management repository. • Fix details (file changed, fix release
information).

Communication Captures all the details of the communication that • Test Case ID
transpired for this defect among the various • Defect reference #
stakeholders. These could include communication • Details of communication.
between the testing team and development team,
and so on.
Provide insights into effectiveness of
communication.
Configuration management repository
 CM repository is also(Software Configuration Management)SCM.
 SCM is process of tracking and controlling the software changes.

 Change control ensures that:

 1)Changes to test files are made in controlled fashion and only
with proper approvals.
 2)Changes made by one test engineer are not accidentally lost or
overwritten by other changes.
 3)Each change produces a distinct version of the files that is re-
creatable at any point of time.
 4)At any point of time, everyone gets access to only the most
recent version of the test files. ( except in exceptional cases).
 Version control ensures that the test scripts associated with a given
release of a product are base lined along with the product files.
Test Infrastructure Management
 TCDB, defect repository, and SCM repository should
complement each other and work together in an integrated
fashion as shown in fig.
Test Infrastructure Management
 The defect repository links the defects, fixes , and tests.

 The files for all these will be in the SCM.

 The meta data about the modified test files will be in the
TCDB.

 Thus, starting with a given defect, one can trace all the test
case that test the defect ( from the TCDB) and then find the
corresponding test case files and source files from the SCM
repository.
Test Infrastructure Management
 Similarly, in order to decide which tests to run for given
regression run:
 The defects recently fixed can be obtained from the
defects repository and test for these can be obtained
from the TCDB and included in the regression tests.
 The list of files changed since the last regression run can
be obtained from the SCM repository and the
corresponding test files traced from the TCDB.
 The set of tests not run recently can be obtained from
the TCDB and these can become potential candidates to
be run at certain frequencies.
Test People Management
1) People management is an integral
part of any project management.
2) People management requires the
ability to hire, motivate and retain
the right people.
3) These skills are seldom formally
taught. (unlike technical skills. )
4) Project manager often learn these
skills in a “sink or swim” mode,
being thrown head-on into the task.
Test People Management
1) These team-building exercises should be
ongoing and sustained, rather than be done
in one burst.
2) The effort of these exercises tend to wear
out under the pressure of deadlines of
delivery and quality.
3) The common goals and the spirit of
teamwork have to be internalized by all
stakeholders.
4) Such an internalization and upfront team
building has to be part of the planning
process the team to succeed.
Test Process
Testing is not a single activity , instead it
consists of number of processes.
Activities:
1) Base Lining a Test Plan

2) Test Case Specification

3) Executing Test cases

Base Lining a Test Plan
Generally a baseline is defined as a line that
forms the base for any construction or
measurement or comparisons or calculations.
It is one of the types of non-functional testing.
It refers to the validation of documents and
specifications on which test case would be
designed.
The requirement specification validation is
base line testing.
Base Lining a Test Plan
It is developed by competent people and
given to higher authorities for approval.
It accepted the followed for further all
activities.
If changes are made then it is made to test
plan and approval is needed.
A majority of issues are solved through
baseline testing.
Test Case Specification
Test Case Specification document described
detailed summary of what scenarios will be
tested, how they will be tested, how often
they will be tested for a given feature.
It specifies the purpose of a specific test,
identifies the required inputs and expected
results, provides step-by-step procedures for
executing the test, and outlines the pass/fail
criteria for determining acceptance.
Test Case Specification
Test Case Specification has to be done
separately for each unit.
Based on the approach specified in the
test plan, the feature to be tested for each
unit must be determined.
Based on these, the test cases are
specified for the testing unit.
Test Case Specification
What is Test Case Specification Identifiers?
The way to uniquely identify a test case is as follows:
Test Case Objectives: Purpose of the test
Test Items: Items (e.g., requirement specifications,
design specifications, code, etc.) required to run a
particular test case. This should be provided in "Notes”
or “Attachment” feature. It describes the features and
conditions required for testing.
Input Specifications: Description of what is required
(step-by-step) to execute the test case (e.g., input files,
values that must be entered into a field, etc.). This
should be provided in “Action” field.
Test Case Specification
Output Specifications: Description of what
the system should look like after the test case
is run. This should be provided in the
“Expected Results” field.
Environmental Needs: Description of any
special environmental needs. This includes
system architectures, Hardware & Software
tools, records or files, interfaces, etc
To sum up, Test Case Specification defines
the exact set up and inputs for one Test Case.
Executing Test Case
Proper execution of test cases is essential
which will minimize the work and reduce
the time to release s/w product.
Test Execution task:
1. Follow the test procedures to execute
test cases.
2. Do the confirmation testing for the
failed test cases.
3. Log the result after test execution.
Executing Test Case
Proper execution of test cases is essential
which will minimize the work and reduce
the time to release s/w product.
Test Execution task:
1. Follow the test procedures to execute
test cases.
2. Do the confirmation testing for the
failed test cases.
3. Log the result after test execution.
Executing Test Case
4.Compare actual and expected results. In
case of difference, defect occurrence is
detected.
Update the defect database which is used
to communicate between developer and
testing team.
So the execution of test cases will decide
the suspension or resumption of further
test cases.
Test Reporting: Preparing Test summary
report
The definition of a Test Summary is as
simple as the name suggests.
It is also known as a Test Closure Report.
It provides the relevant stakeholders with
a detailed account of the overall test
results and the defects found.
It aims to formally summarize the results
of the entire testing process.
What to include in a Test Summary
Report?
An informative test summary report
should be concise and relevant.
What to include in a Test Summary
Report?
 Below is a list of what should be included:
 Test Objective – Include the purpose of the testing,
this shows that the test object and the requirement
were clear with the testing team.
 Areas Covered – Include the areas and the
functionalities of the product that were covered in
testing.
 Areas Not Covered – It is very essential to capture
the areas of the product that were not covered in
testing.
 Testing Approach – This is important to cover since
it indicates what and how the testing was performed.
What to include in a Test Summary
Report?
 Defect Report – Defect Report, though it usually gets captured
already in the bug report, having it included in the test summary
report can give an additional advantage to your report.
 Platform Details – Nowadays, products are tested across multiple
platforms. With the growing demand, testing is not just limited to
multiple devices or browsers but different versions as well. So, let’s
ensure to include details of every single platform and environment
on which the product was tested.
 Overall Summary – This section is mainly for us to provide our
feedback on the overall status of the application under test. It
should inform the client about the critical issues that were
discovered and how many are still open so that they can estimate
how close they are to shipping the product to the customer.