Non Functional Testing
Non Functional Testing
Non Functional Testing
A Complete Guide to Non-Functional Testing: Its Purpose, Types, Tool, Test Cases with Examples
What is Non-Functional Testing?
Non-functional testing is done to verify the non-functional requirement of the application like Performance,
Usability, etc.
It verifies if the behavior of the system is as per the requirement or not. It covers all the aspects which are
not covered in functional testing. In our day to day testing, a lot of attention is given to functional testing
and functional requirements.
The clients are also interested in fulfilling the functional requirements which are directly related to the
functionality of an application. But in the actual phase i.e. when you’re functionally tested, the software
comes into the market and is used by the real end-users, and there are chances for it to face some issues
related to the performance.
These issues are not related to the functionality of the system, but they can affect the user experience in a
negative manner. Hence it is important for the software or application to be tested for Non-Functional
requirements as well in order to avoid negative customer experience.
The Non Functional requirements were also not given proper attention in the earlier test cycles. However,
this has changed now. Non-functional tests are now most important as they consider all the application
performance and security issues these days.
This testing has a greater impact on applications when it comes to the performance of the application under
high user traffic. This testing ensures that your application is stable and is able to handle load under extreme
conditions.
As the name itself depicts, this testing concentrates on the non-functional aspect of the application. So what
are the non-functional aspects? Or should I say what are the features which are not related to the
functionality of the application?
Purpose
The sole purpose of this type of testing is to ensure that the non-functional aspects of the application are
tested and the application works well in context to the same.
The purpose is to cover the testing of all the characteristics of the application which help to provide an
application that meets the business expectation.
Example
This is an important testing type.
Functional testing tests the application’s functionality and ensures that it works as expected but the non-
functional testing ensures that the application works good enough to meet the business expectations.
Meanwhile, when the application goes live, it might result in critical or major issues like when the load is
increased on the application, it becomes too slow and takes a lot of time to open.
Response time might increase or when the load is increased to an extent, the application might crash. This
shows how important it is to test an application’s non-functional aspects.
Advantages
Given below some of the advantages of a non-functional test:
It covers the testing which cannot be covered in functional testing.
It ensures that the application runs efficiently and is reliable enough.
It ensures the security of the application.
While we perform testing, the focus is mainly on functional testing which tests the functionality of the
product. But non-functional testing is as important as functional testing and its requirement should be taken
into consideration right from the inception of the product.
Non-Functional requirements are used to perform Non-Functional testing. These requirements include the
performance output that is expected from the application or the software under test. This basically includes
the time taken by the software to operate a particular system.
Non-Functional requirements also capture the behavior when a large number of people are using the
software at the same time. Most of the time it is experienced that the servers are busy or unavailable due to
heavy load (i.e. more people are using it at the same time). Booking online railway tickets can be the
best example of such a situation.
Hence documenting the Non-Functional requirement properly and performing the testing correctly will
ensure high satisfaction in terms of usability by the potential customers.
Though this testing does not have a direct business impact on the functionality of the system, it can increase
the user experience and user-friendliness to a higher extent which in turn will have a greater impact on the
quality of the software.
Example:
Consider the same Facebook login page example. In this case, the scope of Non-Functional testing is to note
the time required by the system to log in to Facebook after entering the valid credentials.
Also, it can be tested as when (let’s say 100) the users log in at the same time, how much time does it take to
log in the user on Facebook.
This ensures that the system can handle load and traffic which in turn has a good user experience.
#3) In Artifacts
A separate artifact should be prepared for the non-functional requirements, this, in turn, would help to have
a better idea of what needs to be tested and how it can be done in iterations.
5 The implementation plan for The implementation plan for non functional
functional requirement is defined in requirement is defined in system architecture.
system design document.
A checklist is generally used when there is no time for documentation and the product has to be tested or
when there is a time constraint, a checklist can be used to ensure that all the important aspects have been
covered.
Test Scope
Test Metrics
Test Tools
Key Dates and Deliverables
Test Scope
Conduct Performance Testing from different perspectives, such as user performance, business processes,
system stability, resource consumption, and so on. Types of Performance Testing to execute are discussed in
the above section of the article (like Load test, Stress test, etc.)
Test Metrics
The Test approach refines the metrics to measure and report during Testing, such as:
Response time (online)
Batch window (batch)
Throughput (For Example, the number of transactions per unit of time)
Utilization (For Example, the percentage of resources utilized)
Test Tools
Mostly Performance Testing requires the usage of appropriate tools:
Load generation tools
Performance monitoring tools
Performance analysis tools
Application profiling tools
Base-lining tools.
Key Dates and Deliverables
The Performance Test Approach Document should describe the following:
Date and time of each Performance Test conduct.
Types of tests and functionality mix to be included in each Performance Test conduct.
Performance Test completion dates.
Non-Functional Testing Types
Performance Testing:
Evaluates the overall performance of the system.
Key elements are as follows:
Validates that the system meets the expected response time.
Evaluates that the significant elements of the application meet the desired response time.
It can also be conducted as a part of integration testing and system testing.
Load Testing:
Evaluates whether the system’s performance is as expected under normal and expected conditions.
Stress Testing:
Evaluates whether the system’s performance is as expected when it is low on resources.
Usability Testing:
Evaluates the system for human use or checks if it is fit for use.
Compatibility Testing:
Evaluates that the application is compatible with other hardware /software with minimum and maximum
configuration.
Recovery Testing:
Evaluates that the application terminates gracefully in case of any failure and the data is recovered
appropriately from any hardware and software failures.
Instability Testing:
Evaluates and confirms if the software installs and uninstalls correctly.
Documentation Testing:
Evaluates the documents and other user manuals.
Failover Testing:
Failover testing is done in order to verify that in case of a system failure the system is capable enough to
handle extra resources like servers.
In order to prevent such a situation, backup testing plays a big role. Creating a backup system is what the
process is all about. If the backup is available, then it helps to get the system back.
Security Testing:
Security testing is done to ensure that the application has no loopholes which could lead to any data loss or
threats. It is one of the important aspects of non-functional testing and if not performed properly, it can lead
to security threats.
It includes testing authentication, authorization, integrity, and availability.
Scalability Testing:
Scalability testing is done to verify if the application is capable enough to handle increased traffic, number
of transactions, data volume, etc. The system should work as expected when the volume of data or change in
the size of data is done.
Compliance Testing:
Compliance testing is done to verify if the standards defined are being followed or not. Audits are done to
verify the same.
For Example, Audits are done to verify the process of creating test cases/test plans and placing them in the
shared location with the standard name that is being done or not. In QC, while naming the test cases the
standard test case name is being followed or not. Documentation is complete and approved or not.
These are the few pointers that are covered while auditing.
Endurance Testing:
Endurance Testing is done to verify the system’s behavior when a load is increased to an extent for a long
time.
It is also called as Soak testing & Capacity testing. It helps to verify if there are any memory leaks in the
system. Endurance testing is a subset of load testing.
Localization Testing:
Localization testing is done to verify the application in different languages i.e. different locales. The
application should be verified for a particular culture or locale. The main focus is to test the content, GUI of
the application.
Internationalization Testing:
Internationalization testing is also known as i18n testing.
I18n represents I –eighteen letters- N. It is done to verify if the application works as expected across all the
language settings. It verifies that any functionality or application itself does not break i.e. the application
should be capable enough to handle all the international settings.
It also verifies that the application gets installed without any issues.
Reliability Testing:
Reliability testing is done to verify if the application is reliable and is tested for a specific period of time in
the defined environment. An application should give the same output as expected every time, only then it
can be considered as reliable.
Portability Testing:
Portability testing is done to verify if in case a software/application is installed on a different system or on a
different platform it should be able to run as expected i.e. no functionality should be affected because of a
change in the environment.
While testing, it is also required to test the change with the hardware configuration such as the hard disk
space, Processor, and also with different operating systems to ensure the application’s correct behavior and
expected functionality are intact.
Baseline Testing:
Baseline testing is also known as benchmark testing as it creates a base for any new application to be tested.
For Example: In the first iteration, the response time for an application was 3 seconds. Now, this has been
set as a benchmark for the next iteration and in the next iteration, the response time changes to 2 seconds. It
is basically a validation document that is used as a base for future references.
Efficiency Testing:
Efficiency testing is done to verify if the application works efficiently and the number of resources required,
tools required, complexity, customer requirement, environment required, time, what kind of project it is, etc.
These are some of the pointers that would help to define how efficiently an application would work if all the
considered parameters work as expected.
Disaster Recovery Testing:
This testing is done to verify the success rate of recovery of an application or system if any critical failure
happens and whether the system is able to restore the data and application or the system could cope up
easily to return the way it was working earlier i.e. from the operational front.
Maintainability Testing:
Once the application/Product goes live, then there are chances for an issue to come up in the live
environment or the customer may want an enhancement for the application which is already live.
In this case, maintenance testing team is available to test the above scenarios mentioned. Once the
application goes live it still needs maintenance for which the maintenance testing team works.
Example: The following are the steps to be performed on an Online Banking application in order to
perform a transfer. Let us use that as our test for reference.
1. Login to the site.
2. Choose the bank account.
3. Choose the payee (this payee could belong to the same bank or a different one- this depends on your
data choice to execute this step. In any case, choose one. Also, we are going to assume that the payee
is already added.).
4. Enter the amount to be transferred (positive value, within the limit, correct format, etc.).
5. Click Transfer and check if confirmation is received, account balance has been updated and all that.
This is the functional test case, correct?
On the same application, on the same transfers’ page, let us say we are performing Performance, Security
and Usability Testing. These are non-functional types, correct?
How would we write the test cases?
#1) Usability Testing Test cases
Usability testing is a genre of software testing that deals with user experience. These are some of the
questions that we try to answer.
The user would perform the exact same steps to make as in the functional test case. Am I right?
#2) Performance Testing Test cases
There are several variations to performance testing, but at its core, it is used to get statistics about the
system, its resource utilization, response time, network consumption, etc. at various load points.
What steps would each user perform in order to use the transfer while the performance test is in progress?
When do I want to check if the transfers are hack-proof and are directed correctly to the intended recipients
and that there are no black spots in the entire process? I would perform the transfer while the monitoring
process for security leaks goes on in parallel.
Therefore, in effect, I am carrying out the exact same steps that I would normally do in case of a
functional test case.
I guess, we have enough to establish that the steps in all the situations are the same. The method and the
intention behind the process are what is different.
Efficiency
Business
applicability
Type of
Who? Why? Intention
Testing
Ease of learning
Efficiency
Performance Performance testers via a tool that treats the operation as a Response time
transaction performed by a certain number of concurrent
users while the tester is analyzing all the logistics
Security Scanning tools and other monitoring system by specialized Hack safe
security experts