QA Booklet
QA Booklet
QA Booklet
1.2.2Sanity testing
Sanity Testing is a special type of software testing performed after receiving a software build with
little changes in code or functionality to ascertain that certain bugs have been fixed in advance to
resolve functionality issues. The goal of sanity testing is to determine that the proposed
functionalities are working roughly as expected. If sanity testing fails then the build is rejected
directly to save time and costs that are involved in more rigorous testing.
The objective of sanity testing is not to verify the core functionalities thoroughly but to determine
that the developer has applied some rationality while building a software program. For example, if
your scientific calculator fives the result of 2+2=5! for the instance, then there is no need to check
the advanced functionalities like trigonometry calculations or more.
Sanity testing is performed during the release phase to check the main functionalities of an
application without going into depth. It is named as the subset of regression testing. There are
certain cases when regression testing is not done to the build due to time constraints and sanity
testing is considered more suitable to check the main functionalities.
The major difference between both types of testing can be quickly understood by the diagram
given below.
Smoke Testing is performed to ascertain Sanity Testing is done to check the new
that the critical functionalities of the program functionality/bugs have been fixed
is working fine
The objective of this testing is to verify the The objective of the testing is to verify the
"stability" of the system in order to proceed "rationality" of the system in order to proceed
with more rigorous testing with more rigorous testing
This testing is performed by the developers Sanity testing is usually performed by testers
or testers
Smoke testing is usually documented or Sanity testing is usually not documented and is
scripted unscripted
Smoke testing exercises the entire system Sanity testing exercises only the particular
from end to end component of the entire system
Smoke testing is like General Health Check Sanity Testing is like specialized health check
Up up
1.4.2Regression Testing
Regression Testing is a type of software testing executed to check whether a code change has
not unfavorably disturbed current features & functions of an Application
Defect verification is not the part of Defect verification is the part of re-
Regression Testing testing
You can do automation for You cannot automate the test cases
regression testing, Manual for Retesting
Testing could be expensive and
time-consuming
Regression testing is done for Retesting is done only for failed test
passed test cases cases
1.7.2Non-Functional Testing
Non-functional testing is a type of testing to check non-functional aspects (performance, usability,
reliability, etc.) of a software application. It is explicitly designed to test the readiness of a system
as per nonfunctional parameters which are never addressed by functional testing.
A good example of non-functional test would be to check how many people can simultaneously
login into a software.
Non-functional testing is equally important as functional testing and affects client satisfaction.
Requirements Functional testing is carried out This kind of testing is carried out by
using the functional performance specifications
specification.
Manual testing Functional testing is easy to It's very hard to perform non-
execute by manual testing. functional testing manually.
Functionality It describes what the product It describes how the product works.
does.
Load Testing is to test the system Stress testing is to test the system behavior
behavior under normal workload under extreme conditions and is carried out till
conditions, and it is just testing or the system failure.
simulating with the actual workload
Load testing does not break the Stress testing tries to break the system by
system testing with overwhelming data or resources.
Speech Recognition Software - It will convert the spoken word to text, which serves as
input to the computer.
Screen reader software - Used to read out the text that is displayed on the screen
Screen Magnification Software- Used to enlarge the monitor and make reading easy for
vision-impaired users.
Special keyboard made for the users for easy typing who have motor control difficulties
Alpha testing involves both the white Beta Testing typically uses Black Box
box and black box techniques
Testing
Alpha testing requires a lab Beta testing doesn't require any lab
environment or testing environment environment or testing environment. The
software is made available to the public
and is said to be real time environment
Long execution cycle may be required Only a few weeks of execution are
for Alpha testing required for Beta testing
Alpha testing is to ensure the quality of Beta testing also concentrates on the
the product before moving to Beta quality of the product, but gathers users
testing input on the product and ensures that the
product is ready for real time users.
Static testing covers the structural and Dynamic testing covers the executable file
statement coverage testing of the code
Cost of finding defects and fixing is less Cost of finding and fixing defects is high
Return on investment will be low as this
Return on investment will be high as process involves after the development
this process involved at an early stage phase
More reviews comments are highly More defects are highly recommended for
recommended for good quality good quality.
The main focus of black box testing is White Box Testing (Unit Testing)
on the validation of your functional validates internal structure and working of
requirements. your software code
Black box testing gives abstraction To conduct White Box Testing,
from code and focuses on testing effort knowledge of underlying programming
on the software system behavior. language is essential. Current day
software systems use a variety of
programming languages and
technologies and its not possible to know
all of them.
Black box testing facilitates testing White box testing does not facilitate
communication amongst modules testing communication amongst modules
Exploratory Techniques
Inadequate specification
Quality Control
Quality control popularly abbreviated as QC. It is a Software Engineering process used
to ensure quality in a product or a service. It does not deal with the processes used to
create a product; rather it examines the quality of the "end products" and the final
outcome.
Maintenance Regression
Maintenance
This is not the complete list as there are more than 150 types of testing types and
still adding. Also, note that not all testing types are applicable to all projects but
depend on the nature & scope of the project.
Verification Validation
Verification uses methods like reviews, It uses methods like Black Box
walkthroughs, inspections, and desk- Testing, White Box Testing, and non-
checking etc. functional testing
It finds bugs early in the development It can find bugs that the verification
cycle process cannot catch
QA team does verification and make With the involvement of testing team
sure that the software is as per the validation is executed on software
requirement in the SRS document. code.
cases are covered so that no functionality should miss while doing Software testing.
Why RTM is Important?
The main agenda of every tester should be to understand the client’s requirement
and make sure that the output product should be defect-free. To achieve this goal,
every QA should understand the requirement thoroughly and create positive and
negative test cases.
This would mean that the software requirements provided by the client have to be
further split into different scenarios and further to test cases. Each of this case has
to be executed individually.
Parameters to include in Requirement Traceability Matrix
Requirement ID
Requirement Type and Description
Test Cases with Status
1.7. Defect lifecycle
Defect Life Cycle or Bug Life Cycle is the specific set of states that a Bug goes
through from discovery to defect fixation.
Bug Life Cycle Status
The number of states that a defect goes through varies from project to project.
Below lifecycle diagram, covers all possible states
New: When a new defect is logged and posted for the first time. It is assigned a
status as NEW.
Assigned: Once the bug is posted by the tester, the lead of the tester approves the
bug and assigns the bug to the developer team
Open: The developer starts analyzing and works on the defect fix
Fixed: When a developer makes a necessary code change and verifies the change,
he or she can make bug status as "Fixed."
Retest: Tester does the retesting of the code at this stage to check whether the
defect is fixed by the developer or not and changes the status to "Re-test."
Verified: The tester re-tests the bug after it got fixed by the developer. If there is no
bug detected in the software, then the bug is fixed and the status assigned is
"verified."
Reopen: If the bug persists even after the developer has fixed the bug, the tester
changes the status to "reopened". Once again the bug goes through the life cycle.
Closed: If the bug is no longer exists then tester assigns the status "Closed."
Duplicate: If the defect is repeated twice or the defect corresponds to the same
concept of the bug, the status is changed to "duplicate."
Rejected: If the developer feels the defect is not a genuine defect then it changes
the defect to "rejected."
Deferred: If the present bug is not of a prime priority and if it is expected to get fixed
in the next release, then status "Deferred" is assigned to such bugs
Not a bug: If it does not affect the functionality of the application then the status
assigned to a bug is "Not a bug".
Priority
Priority is defined as the order in which a defect should be fixed. Higher the priority
the sooner the defect should be resolved.
Defects that leave the software system unusable are given higher priority over
defects that cause a small functionality of the software to fail.
Defect Severity and Priority Types
In Software Testing, Defect severity can be categorized into four classes
Critical: This defect indicates complete shut-down of the process, nothing can
proceed further
Major: It is a highly severe defect and collapses the system. However, certain parts
of the system remain functional
Medium: It causes some undesirable behavior, but the system is still functional
Low: It won't cause any major break-down of the system
Defect priority can be categorized into three classes
Low: The Defect is an irritant but repair can be done once the more serious Defect
has been fixed
Medium: During the normal course of the development activities defect should be
resolved. It can wait until a new version is created
High: The defect must be resolved as soon as possible as it affects the system
severely and cannot be used until it is fixed
Priority Vs Severity: Key Difference
Priority Severity
Defect Priority has defined the order in Defect Severity is defined as the
which the developer should resolve a degree of impact that a defect has on
defect the operation of the product
Priority is categorized into three types Severity is categorized into five types
o Low o Critical
o Medium o Major
o High o Moderate
o Minor
o Cosmetic
Priority indicates how soon the bug Severity indicates the seriousness of
should be fixed the defect on the product functionality
Its value is subjective and can change Its value is objective and less likely to
over a period of time depending on the
change in the project situation change
High priority and low severity status High severity and low priority status
indicates, defect have to be fixed on indicates defect have to be fixed but
immediate bases but does not affect the not on immediate bases
application
During UAT the development team fix During SIT, the development team will
defects based on priority fix defects based on the severity and
then priority
1.10. Difference between Test scenario, Test case and Test script
Test Scenario
A Test Scenario is defined as any functionality that can be tested. It is also called Test
Condition or Test Possibility. As a tester, you may put yourself in the end user’s shoes
and figure out the real-world scenarios and use cases of the Application under Test.
Tips to Create Test Scenarios
Each Test Scenario should be tied to a minimum of one Requirement or User Story as
per the Project Methodology.
Before creating a Test Scenario that verifies multiple Requirements at once, ensure you
have a Test Scenario that checks that requirement in isolation.
Avoid creating overly complicated Test Scenarios spanning multiple Requirements.
The number of scenarios may be large, and it is expensive to run them all. Based on
customer priorities only run selected Test Scenarios
2.2 Test Case
A test case includes specific variables or conditions, using which a test engineer can
determine as to whether a software product is functioning as per the requirements of
the client or the customer.
Test case
The objective or summary of the test case
description:
Pass, Fail, ‘Not executed’ when test case is not executed and
Status:
‘Blocked’ when high severity bug is found
Created By: Name of the person who wrote the test case
Date of creation: The date on which the test case was authored
Executed By: Name of the person who ran or executed the test case
Date of execution: The date on which the test case was executed
Formal test cases: Formal test cases are those test cases which are authored as per
the test case format. It has all the information like preconditions, input data, output data,
post conditions, etc. It has a defined set of inputs which will provide the expected
output.
Informal test cases: Informal test cases are authored for such requirements where the
exact input and output are not known. In order to test them the formal test cases are
not authored but the activities done and the outcomes are reported once the tests are
run.
3 Test Script
A TEST SCRIPT is a set of instructions (written using a scripting/programming
language) that is performed on a system under test to verify that the system
performs as expected. Test scripts are used in automated testing.
Sometimes, a set of instructions (written in a human language), used in manual
testing, is also called a Test Script but a better term for that would be a Test Case.
Some scripting languages used in automated testing are:
JavaScript
Perl
Python
Ruby
Tcl
Unix Shell Script
VBScript
The requirement is the first stage in the SDLC process. It is conducted by the senior team
members with inputs from all the stakeholders and domain experts in the industry.
Planning for the quality assurance requirements and recognition of the risks involved is
also done at this stage.
This stage gives a clearer picture of the scope of the entire project and the anticipated
issues, opportunities, and directives which triggered the project.
Requirements Gathering stage need teams to get detailed and precise requirements. This
helps companies to finalize the necessary timeline to finish the work of that system.
Design:
In this phase, the system and software design documents are prepared as per the
requirement specification document. This helps define overall system architecture.
This design phase serves as input for the next phase of the model.
Coding:
Once the system design phase is over, the next phase is coding. In this phase, developers
start build the entire system by writing code using the chosen programming language. In
the coding phase, tasks are divided into units or modules and assigned to the various
developers. It is the longest phase of the Software Development Life Cycle process.
In this phase, Developer needs to follow certain predefined coding guidelines. They also
need to use programming tools like compiler, interpreters, debugger to generate and
implement the code.
Testing:
Once the software is complete, and it is deployed in the testing environment. The testing
team starts testing the functionality of the entire system. This is done to verify that the
entire application works according to the customer requirement.
During this phase, QA and testing team may find some bugs/defects which they
communicate to developers. The development team fixes the bug and sends back to QA
for a re-test. This process continues until the software is bug-free, stable, and working
according to the business needs of that system.
Installation/Deployment:
Once the software testing phase is over and no bugs or errors left in the system then the
final deployment process starts. Based on the feedback given by the project manager, the
final software is released and checked for deployment issues if any.
Maintenance:
Once the system is deployed, and customers start using the developed system, following 3
activities occur
Bug fixing - bugs are reported because of some scenarios which are not tested at all.
Upgrade - Upgrading the application to the newer versions of the Software.
Enhancement - Adding some new features into the existing software.
The main focus of this SDLC phase is to ensure that needs continue to be met and that
the system continues to perform as per the specification mentioned in the first phase.
2.2. Models
2.2.1. Waterfall
Waterfall – is a cascade SDLC model, in which development process looks like the
flow, moving step by step through the phases of analysis, projecting, realization,
testing, implementation, and support. This SDLC model includes gradual execution
of every stage completely. This process is strictly documented and predefined with
features expected to every phase of this software development life cycle model.
ADVANTAGES DISADVANTAGES
Simple to use and understand The software is ready only after the
stage is over
Development stages go one by one Not the best choice for complex and
oriented projects
Perfect for the small or mid-sized Inappropriate for the long-term proje
projects where requirements are clear
and not equivocal
ADVANTAGES DISADVANTAGES
Easy to determine the key points in the The progress of the stage is hard to
development cycle measure while it is still in the develo
Easy to classify and prioritize tasks Integration is done at the very end, w
does not give the option of identifyin
problem in advance
2.2.2. V-shaped
V-shaped SDLC model is an expansion of classic waterfall model and it’s
based on associated test stage for the every development stage. This is a very
strict model and the next stage is started only after the previous phase. This is also
called “Validation and verification” model. Every stage has the current process
control, to make sure that the conversion to the next stage is possible.
ADVANTAGES DISAD
2.2.3. Iterative
The Iterative SDLC model does not need the full list of requirements before the project
starts. The development process may start with the requirements to the functional part,
which can be expanded later. The process is repetitive, allowing to make new versions
of the product for every cycle. Every iteration (which last from two to six weeks)
includes the development of a separate component of the system, and after that, this
component is added to the functional developed earlier. Speaking with math
terminology, the iterative model is a realization of the sequential approximation method;
that means a gradual closeness to the planned final product shape.
ADVANTAGES DISADVANTAGES
The shorter iteration is - the easier Bad choice for the small projects
testing and debugging stages are
Problems and risks defined within The risks may not be completely determ
one iteration can be prevented in even at the final stage of the project
the next sprints
2.2.4. Incremental
What is the Incremental Model?
incremental-
model
The incremental build model is a method of software development where the
model is designed, implemented and tested incrementally (a little more is added
each time) until the product is finished. It involves both development and
maintenance. The product is defined as finished when it satisfies all of its
requirements. This model combines the elements of the waterfall model with the
iterative philosophy of prototyping.
The product is decomposed into a number of components, each of which are
designed and built separately (termed as builds). Each component is delivered to
the client when it is complete. This allows partial utilisation of product and avoids a
long development time. It also creates a large initial capital outlay with the
subsequent long wait avoided. This model of development also helps ease the
traumatic effect of introducing completely new system all at once.
There are some problems with this model. One is that each new build must be
integrated with previous builds and any existing systems. The task of decomposing
product into builds not trivial either. If there are too few few builds and each build
degenerates this turns into Build-And-Fix model. However if there are too many
builds then there is little added utility from each build.
Advantages of Incremental Model
Generates working software quickly and early during the software life cycle.
More flexible – less costly to change scope and requirements.
Easier to test and debug during a smaller iteration.
Easier to manage risk because risky pieces are identified and handled during its
iteration.
Each iteration is an easily managed milestone.
Build a Prototype
Project is divided by short and transparent The team should be highly profe
iterations and client-oriented
Risks are minimized thanks to the flexible New requirements may conflict w
change process existing architecture
Fast release of the first product version With all the corrections and chan
there is possibility that the projec
exceed expected time
2.2.6.1.https://blog.trello.com/beginners-guide-scrum-and-agile-
project-management
2.2.6.2.https://en.wikipedia.org/wiki/Scrum_(software_development)
3. STLC Phases
3.1. What is STLC
Software Testing Life Cycle (STLC) is defined as a sequence of activities
conducted to perform Software Testing.
Contrary to popular belief, Software Testing is not a just a single activity. It consists of
a series of activities carried out methodologically to help certify your software product.
During this phase, test team studies the requirements from a testing point of view to identify
the testable requirements.
The QA team may interact with various stakeholders (Client, Business Analyst, Technical
Leads, System Architects etc) to understand the requirements in detail.
Requirements could be either Functional (defining what the software must do) or Non
Functional (defining system performance /security availability )
Activities
Identify types of tests to be performed.
Gather details about testing priorities and focus.
Prepare Requirement Traceability Matrix (RTM).
Identify test environment details where testing is supposed to be carried out.
Automation feasibility analysis (if required).
Deliverables
RTM
Automation feasibility report. (if applicable)
Test Planning
Typically, in this stage, a Senior QA manager will determine effort and cost estimates
for the project and would prepare and finalize the Test Plan. In this phase, Test
Strategy is also determined.
Activities
Preparation of test plan/strategy document for various types of testing
Test tool selection
Test effort estimation
Resource planning and determining roles and responsibilities.
Training requirement
Deliverables
Test plan /strategy document.
Effort estimation document.
Test Case Development
This phase involves the creation, verification and rework of test cases & test
scripts. Test data, is identified/created and is reviewed and then reworked as well.
Activities
Create test cases, automation scripts (if applicable)
Review and baseline test cases and scripts
Create test data (If Test Environment is available)
Deliverables
Test cases/scripts
Test data
Test Environment Setup
Test environment decides the software and hardware conditions under which a work
product is tested. Test environment set-up is one of the critical aspects of testing
process and can be done in parallel with Test Case Development Stage. Test team
may not be involved in this activity if the customer/development team provides the
test environment in which case the test team is required to do a readiness check
(smoke testing) of the given environment.
Activities
Understand the required architecture, environment set-up and prepare hardware and
software requirement list for the Test Environment.
Setup test Environment and test data
Perform smoke test on the build
Deliverables
Environment ready with test data set up
Smoke Test Results.
Test Execution
During this phase, the testers will carry out the testing based on the test plans and the
test cases prepared. Bugs will be reported back to the development team for correction
and retesting will be performed.
Activities
Execute tests as per plan
Document test results, and log defects for failed cases
Map defects to test cases in RTM
Retest the Defect fixes
Track the defects to closure
Deliverables
Completed RTM with the execution status
Test cases updated with results
Defect reports
Test Cycle Closure
Testing team will meet, discuss and analyze testing artifacts to identify strategies that
have to be implemented in the future, taking lessons from the current test cycle. The
idea is to remove the process bottlenecks for future test cycles and share best
practices for any similar projects in the future.
Activities
Evaluate cycle completion criteria based on Time, Test coverage, Cost,Software,
Critical Business Objectives, Quality
Prepare test metrics based on the above parameters.
Document the learning out of the project
Prepare Test closure report
Qualitative and quantitative reporting of quality of the work product to the customer.
Test result analysis to find out the defect distribution by type and severity.
Deliverables
Test Closure report
Test metrics
4. Reviews
A review in a Static Testing is a process or meeting conducted to find the potential
defects in the design of any program. Another significance of review is that all the team
members get to know about the progress of the project and sometimes the diversity of
thoughts may result in excellent suggestions. Documents are directly examined by
people and discrepancies are sorted out.
During the Review process four types of participants that take part in testing are:
Moderator: Performs entry check, follow up on rework, coaching team member,
schedule the meeting.
Author: Takes responsibility for fixing the defect found and improves the quality of the
document
Scribe: It does the logging of the defect during a review and attends the review
meeting
Reviewer: Check material for defects and inspects
Manager: Decide on the execution of reviews and ensures the review process
objectives are met.
Identify
errors and
issues.
Evaluate conformance to specification and
Objective Examine
plans. Ensure change integrity.
alternatives.
Forum for
learning.
Delegated Controls
The
designer
makes all
decisions. Review team petitions management or
Decision
Change is technical leadership to act on
Making
the recommendations.
prerogative
of the
designer.
Change
verification Leader verifies that action items are
Change
left to other documented; incorporated into external
Verification
project processes.
controls.
Group Dynamics
Recommended
2-7 people. 3 or more people.
Group Size
The
designer
and any
Attendance Management and engineering leadership.
other
interested
party.
The
Leadership designer or Lead engineer.
designate.
Procedures
The
Presenter Engineering team lead or representative.
designer.
Outputs and
As desired. Review report.
Artifacts
http://learnthebasicsofsoftwaretesting.blogspot.com/2013/04/types-of-
review.html
http://tryqa.com/what-are-the-types-of-review/
10. Automation Testing
Selenium is not just a single tool but a suite of software's, each catering to different testing
needs of an organization. It has four components.
The WebDriver proves itself to be better than both Selenium IDE and Selenium
RC in many aspects. It implements a more modern and stable approach in automating
the browser's actions. WebDriver, unlike Selenium RC, does not rely on JavaScript for
Automation. It controls the browser by directly communicating with it.
Java
C#
PHP
Python
Perl
Ruby