Test Strategy
Test Strategy
Test Strategy
A Test Strategy document is a high level document and normally developed by project
manager. This document defines Software Testing Approach to achieve testing
objectives. The Test Strategy is normally derived from the Business Requirement
Specification document.
The Test Strategy document is a static document meaning that it is not updated too often. It
sets the standards for testing processes and activities and other documents such as the Test
Plan draws its contents from those standards set in the Test Strategy Document.
Some companies include the Test Approach or Strategy inside the Test Plan, which is fine
and it is usually the case for small projects. However, for larger projects, there is one Test
Strategy document and different number of Test Plans for each phase or level of testing.
Components of the Test Strategy document
Business issues
Test deliverability
Training plan
Test Plan
The Test Plan document on the other hand, is derived from the Product Description,
Software Requirement Specification SRS, or Use Case Documents.
The Test Plan document is usually prepared by the Test Lead or Test Manager and the focus
of the document is to describe what to test, how to test, when to test and who will do what
test.
It is not uncommon to have one Master Test Plan which is a common document for the test
phases and each test phase have their own Test Plan documents.
There is much debate, as to whether the Test Plan document should also be a static
document like the Test Strategy document mentioned above or should it be updated every
often to reflect changes according to the direction of the project and activities.
My own personal view is that when a testing phase starts and the Test Manager is
controlling the activities, the test plan should be updated to reflect any deviation from the
original plan. After all, Planning and Control are continuous activities in the formal test
process.
Test Plan id
Introduction
Test items
Features to be tested
Test techniques
Testing tasks
Suspension criteria
Test deliverables
Responsibilities
Schedule
This is a standard approach to prepare test plan and test strategy documents, but things
can vary company-to-company
Test Plan: Test plan is a Document, developed by the Test Lead, which contains "What to
Test", "How to Test", "When to Test", "Who to Test".
Test Strategy: Test Strategy is a Document, developed by the Project manager, which
contains what type of technique to follow and which module to test.
Test Scenario: A name given to Test Cases is called Test Scenario. These
Test Scenario was deal by the Test Engineer.
Test Cases: It is also document and it specifies a Testable condition to validate
functionality. These Test Cases are deal by the Test Engineer
Order of STLC:
Test Strategy, Test Plan, Test Scenario, Test Cases.
1) Severity: It is the extent to which the defect can affect the software. In other words
it defines the impact that a given defect has on the system.
2) For example: If an application or web page crashes when a remote link is clicked,
in this case clicking the remote link by an user is rare but the impact of application
crashing is severe. So the severity is high but priority is low.
3. Priority: Priority defines the order in which we should resolve a defect. Should we fix it
now, or can it wait? This priority status is set by the tester to the developer mentioning the
time frame to fix the defect. If high priority is mentioned then the developer has to fix it at
the earliest.
Few very important scenarios related to the severity and priority which are asked
during the interview:
High Priority & High Severity: An error which occurs on the basic functionality of the
application and will not allow the user to use the system. (Eg. A site maintaining the student
details, on saving record if it, doesnt allow to save the record then this is high priority and
high severity bug.)
High Priority & Low Severity: The spelling mistakes that happens on the cover page or
heading or title of an application.
High Severity & Low Priority: An error which occurs on the functionality of the
application (for which there is no workaround) and will not allow the user to use the system
but on click of link which is rarely used by the end user.
Low Priority and Low Severity: Any cosmetic or spelling issues which is within a
paragraph or in the report (Not on cover page, heading, title).
3. Navigation Testing
Good Navigation is an essential part of a website, especially those that are complex and
provide a lot of information. Assessing navigation is a major part of usability testing.
Test leaders tend to be involved in the planning, monitoring, and control of the
testing activities and tasks.
At the outset of the project, test leaders, in collaboration with the other
stakeholders, devise the test objectives, organizational test policies, test strategies and
test plans.
They estimate the testing to be done and negotiate with management to acquire the
necessary resources.
They recognize when test automation is appropriate and, if it is, they plan the effort,
select the tools, and ensure training of the team. They may consult with other groups
e.g., programmers to help them with their testing.
They lead, guide and monitor the analysis, design, implementation and execution of
the test cases, test procedures and test suites.
As test execution comes near, they make sure the test environment is put into place
report on the test progress, the product quality status and the test results, adapting the
test plan and compensating as needed to adjust to evolving conditions.
During test execution and as the project winds down, they write summary reports on
test status.
Sometimes test leaders wear different titles, such as test manager or test
coordinator. Alternatively, the test leader role may wind up assigned to a project
manager, a development manager or a quality assurance manager. Whoever is playing
the role, expect them to plan, monitor and control the testing work.
Along with the test leaders testers should also be included from the beginning of the
projects, although most of the time the project doesnt need a full complement of testers
until the test execution period. So, now we will see testers responsibilities.
Role
Responsibilities
Project Manager
(2 to 5 hours per week)
Role
Responsibilities
Oversees, point of escalation for the Jira
(defect tracking) database for all testing phases
(update, follow-up and escalate overdue issues).
QA issue prioritization and resolution
facilitation.
Facilitates weekly Quality Assurance meetings
and maintains agenda.
Module Leads
(1 to 4 hours per week/per
person)
Role
Responsibilities
overdue issues).
Work with the QA Lead to improve KFS's
quality assurance capabilities and improve testing
processes.
Attend testing meetings and may travel to
group face-to-face meetings.
Campus Testers
Super Users/Champions/Volunteer
Testers/DFA,Depts & Units (nonproject staff)
(2 - 5 hours)
Role
Responsibilities
best ways to leverage test tool on the project.
Staying current on latest test approaches and
tools, and transferring this knowledge to test team.
Conducting test-design and test-procedure
walk throughs and inspections.
Implementing test-process improvements
resulting from lessons learned and benefits surveys.
Test Traceability Matrix (tracing the testprocedures to the test requirements).
Test-process implementation.
Ensuring that test-product documentation is
complete.
Attend local testing meetings, and may travel
to group face-to-face testing meetings.
Business Analyst
(3 @ 33% time, or 1 @ 75%100%)
Configuration Management
(2 - 15 hours)
Technical Support
(includes development, conversion
Role
Responsibilities
and environment)
(2 - 15 hours)
Training
(2 - 15 hours)
Internal Audit
(1 - 2 hours)
What is Estimation?
Estimation is the process of finding an estimate, or approximation, which is a value that
is usable for some purpose even if input data may be incomplete, uncertain, or unstable.
[Wiki Definition]
The Estimate is prediction or a rough idea to determine how much effort would take to
complete a defined task. Here the effort could be time or cost. An estimate is a forecast or
prediction and approximate of what it would Cost. A rough idea how long a task would take
to complete. An estimate is especially an approximate computation of the probable cost of a
piece of work.
The calculation of test estimation techniques is based on:
Available documents/Knowledge
Assumptions
Calculated risks
Before starting one common question arises in the testers mind is that Why do we
estimate? The answer to this question is pretty simple, it is to avoid the exceeding
timescales and overshooting budgets for testing activities we estimate the task.
Few points need to be considered before estimating testing activities:
Check if before estimating task is all assumptions and risks are documented.
Delphi Technique
Work Breakdown Structure (WBS)
Three Point Estimation
Functional Point Method
1) Delphi Technique:
Delphi technique This is one of the widely used software testing estimation technique.
In the Delphi Method is based on surveys and basically collects the information from
participants who are experts. In this estimation technique each task is assigned to each
team member & over multiple rounds surveys are conduct unless & until a final estimation
of task is not finalized. In each round the thought about task are gathered & feedback is
provided. By using this method, you can get quantitative and qualitative results.
In overall techniques this technique gives good confidence in the estimation. This technique
can be used with the combination of the other techniques.
2) Work Breakdown Structure (WBS):
A big project is made manageable by first breaking it down into individual components in a
hierarchical structure, known as the Work breakdown structure, or the WBS.
The WBS helps to project manager and the team to create the task scheduling, detailed cost
estimation of the project. By using the WBS motions, the project manager and team will
have a pretty good idea whether or not theyve captured all the necessary tasks, based on
the project requirements, which are going to need to happen to get the job done.
In this technique the complex project is divided into smaller pieces. The modules are divided
into smaller sub-modules. Each sub-modules are further divided into functionality. And each
functionality can be divided into sub-functionalities. After breakdown the work all
functionality should review to check whether each & every functionality is covered in the
WBS.
Using this you can easily figure out the what all task needs to completed & they are
breakdown into details task so estimation to details task would be more easier than
estimating overall Complex project at one shot.
Work Breakdown Structure has four key benefits:
Weightage
Function Points
Total
Complex
Medium
Simple
25
20
60
35
35
120
4.15
498
It is basd on thinking
Involved Risk
Bare to losing
9. Use standard template for documentation. Like any excel sheet template or doc file
template and stick to it for all your document needs.
10. Share all project related documents at single location, accessible to every team member
for reference as well to update whenever required.
I am not saying that by applying above steps you will get sudden results. I know this
change wont happen in a day or two, but at least we can start so that these changes start
happening slowly.
After all the documentation needs documentation. Isnt it?
Example: Total test script developed 1360, total test script executed 1280, total test script
passed 1065, and total test script failed 215
So, test case defect density is
215 X 100
---------------------------- = 16.8%
1280
This 16.8% value can also be called as test case efficiency %, which is depends upon total
number of test cases which uncovered defects
Example: Customer filed defects are 21, total defect found while testing are 267, total
number of invalid defects are 17
So, Slippage Ratio is
[21/(267-17) ] X 100 = 8.4%
Requirement Volatility
Ensure that the requirements are normalized or defined properly while estimating
Example: VSS 1.3 release had total 67 requirements initially, later they added another 7
new requirements and removed 3 from initial requirements and modified 11 requirements
So, requirement Volatility is
(7 + 3 + 11) * 100/67 = 31.34%
Review Efficiency:
The Review Efficiency is a metric that offers insight on the review quality and testing
Some organization also use this term as Static Testing efficiency and they are aiming to
get min of 30% defects in static testing
Review efficiency=100*Total number of defects found by reviews/Total number of project
defects
Example: A project found total 269 defects in different reviews, which were fixed and test
team got 476 defects which were reported and valid
So, Review efficiency is [269/(269+476)] X 100 = 36.1%
Effectiveness: Doing the right thing. It deals with meeting the desirable
attributes that are expected by the customer.
Efficiency: Doing the thing right. It concerns the resources used for the service to
be rendered
for a duration (say, the first month, the quarter, or the year).
WEB TESTING
This is done for 3 tier applications (developed for Internet / intranet / xtranet)
Here we will be having Browser, web server and DB server.
The applications accessible in browser would be developed in HTML, DHTML, XML, JavaScript etc. (We
can monitor through these applications)
Applications for the web server would be developed in Java, ASP, JSP, VBScript, JavaScript, Perl, Cold
Fusion, PHP etc. (All the manipulations are done on the web server with the help of these programs
developed)
The DBserver would be having oracle, sql server, sybase, mysql etc. (All data is stored in the database
available on the DB server)
Suppose, in general, Kilogram is a metric for measuring the attribute Weight. Similarly, in software,
How many issues are found in thousand lines of code?, here No. of issues is one measurement & No.
of lines of code is another measurement. Metric is defined from these two measurements.
Test metrics example:
Take the decision for next phase of activities such as, estimate the cost & schedule of future
projects.
2.
3.
2.
3.
Log the defects & need to fail the related test cases
4.
After the defect is resolved, need to re-test the defect & re-execute the corresponding failed
test case.
In above scenario, if metrics are not followed, then the work completed by the test analyst will be
subjective i.e. the test reportwill not have the proper information to know the status of his
work/project.
If Metrics are involved in the project, then the exact status of his/her work with proper numbers/data
can be published.
I.e. in the Test report, we can publish:
1. How many test cases have been designed per requirement?
2. How many test cases are yet to design?
3. How many test cases are executed?
4. How many test cases are passed/failed/blocked?
5. How many test cases are not yet executed?
6. How many defects are identified & what is the severity of those defects?
7. How many test cases are failed due to one particular defect? etc.
Based on the project needs we can have more metrics than above mentioned list, to know the status
of the project in detail.
Based on the above metrics, test lead/manager will get the understanding of the below
mentioned key points.
a) %ge of work completed
b) %ge of work yet to be completed
c) Time to complete the remaining work
d) Whether the project is going as per the schedule or lagging? etc.