Test Environment Management

ABOUT US
Kanoah is an innovative company
providing ground-breaking solutions to
software testing professionals on the
Atlassian JIRA platform

www.kanoah.com
 About Kanoah Tests

Kanoah Tests is a full featured test management, integrated seamless into

JIRA with the same look-n-feel. No need to learn or switch between different
applications

Coordinate all test management activities including test planning, authoring,

execution, tracking and reporting from a central location

Kanoah Tests enables you to track testing progress and quality to foster
collaboration and visibility across traditional and agile teams

Get real-time insights into your testing progress with out of the box
reports

Easily integrate your automated tests and submit test results with Kanoah
Tests’ powerful REST API or use the API to automate many areas of the
application
 Reviews

After looking for several years at plugins for test management we finally found
Kanoah Tests. The other solutions were either too complex, didn't integrate well
with Jira, or were focused on a single project. Kanoah Tests proved to be an Don
elegant solution that allowed linking between any project. Kanoah has been
very responsive to feedback, requests, suggestions as well bugs. The customer
service is awesome. I'd highly recommend Kanoah Tests to teams of any size
Pierce
looking to simplify test management and consolidate tools.

Liked Kanoah from the moment I discovered it. Integrates nicely with JIRA and
Robert especially with Agile. Test cases can be authored right from the story level, but has
all other functionality a Test Case Management solution would need to have,
ranging from creating test plans, executing test cases, importing test cases, API for
Murhamer automation, a.s.o. Additionally the team at Kanoah is amazing and responding to
any question very quickly. It wasn't hard to sell to my management to purchase
Kanoah. Will highly recommend Kanoah to anybody.

I just start working with Kanoah and I am impressed how it's simple to manage tests
without unlimited non-used features like in most of the test management tools and
still to get the needed functionality and results. The integration with Jira is a great
working solution that enable to share testing and development in simple way on
Zour
one system. I recommend to use Kanoah for testing management. It will help to do
Kanoah as much as possible customizable like Jira to match each group
Brosh
methodology

For more reviews, visit: https://marketplace.atlassian.com/plugins/com.kanoah.test-manager/server/reviews

Key Features

Native seamless integration with JIRA Powerful REST API

No need to learn or switch between Establish clear traceability between

different applications requirements, test cases, and defects

Perfect for agile & traditional testing Execute test cases and track results that
approaches matter

Manage, organize and track all your testing Get real-time insights into your testing
efforts in a central place progress with out of the box reports

Reuse test cases across your projects Live statistics accessible to your entire
team
Benefits for the testers Benefits for the teams

No need to lear or switch between Informed decisions based on real-time

different applications insights

Reuse test cases across projects for End-to-end traceability and impact
regression analysis

Link test cases to requirements and Centralized Test Management

defects
Save time and increase productivity
API support for automated efforts
Why choose Kanoah Tests

1 Testing right
inside JIRA
Coordinate all test
2 Make informed
decisions
Take advantage of the built-in
3 Stelar
support
Kanoah Tests users receive
management activities right reports to track the results and priority support, even
inside JIRA measure progress during trials
Test Environment Management
What is Test Environment Management?
Test Environment Management (TEM) is a function in the software delivery process
which aids the software testing cycle by providing a validated, stable and usable
test-environment to execute the test scenarios or replicate bugs. [1]

Test environment management is the process by which test machines (clients,

servers, or both) are managed. It’s super-critical to have clean machines when
running automated and manual tests.

Test environments have always played a ma jor role in software development. With a
wider range of operating systems, hardware and software options available to
consumers than ever before, maintaining a stable and accurate environment is
becoming even more critical.

the biggest constraint in virtualized test labs is hard disk space; most organizations
find they need to invest either in a SAN or some very serious RAID array technologies.
[2]
What is Test Environment Management?
While test environment management will always require a certain amount of
overhead, adopting a few simple practices can help you streamline the process while
simultaneously increasing the quality and stability of your test environment. [3]

Maintaining a central repository of test-environments in scope with their latest

version and connectivity details (Information management).

Allocation of test environments to teams as per requirement (Demand management).

Creation of new test environments as per requirement (Supply management).

Environment Monitoring (Monitoring). [1]

Activities performed by TEM
Deleting/ updating outdated test-environments and its details (Housekeeping).

Preliminary investigation of issues on the environment and sometimes co-ordination

till an issue resolution (Incident Management).

Analyzing data for environment issues, identifying trends and taking pro-active steps
to resolve issues / co-coordinating for a long term fix (Problem Management). [1]
Disadvantages of Virtualization
Impossibility of emulation of all devices. At the current moment, all main devices of
hardware platforms are supported by vendors of virtualization systems. But if some
controllers or devices, which are not supported, are used, we have to refuse the
virtualization of such environment.

Virtual test environment requires additional hardware resources. Existing

virtualization technologies make it possible to bring performance of virtual machines
to real. But to start some definite configuration with a number of machines, we need
a physical host to allow this.

Equipment conflict. Drivers used by a virtualization system can conflict with drivers
of tested equipment.

In case of low disk space on the hard disk of the virtual machine, we cannot increase
it if the virtual machine contains snapshots.

Final testing has to be performed on real machines because no virtual

environment can completely replace the real one.

There are training costs on working with virtual machines. [4]

Disadvantages of Virtualization

When a virus is detected on one of snapshots, it is hard to define on which stage it

appeared. We have to check all snapshots because the antivirus that can detect and
delete viruses on all snapshots of the virtual machine isn’t developed yet.

Single destruction point. If a virtualization system fails, it is impossible to start virtual

machines till the reason of failure is found and system is recovered.

It is not recommended to apply virtual systems while testing the performance and
also while testing applications with high requirements to the physical resources of
the computer.

The final testing of the developed software must be performed on real machines. [4]
 Commonly faced challenges
1. Remote environment: The test assets or environments are mostly placed
geographically in sites that are remote to the teams. This is one of the most commonly
faced challenges for test teams as in case of any problems that may arise pertaining to
hardware, firmware, software, networking, etc.

2. Combined usage between teams: Most often than not, development and test teams
use the same environment assets. It becomes extremely cost unfriendly for organizations
to procure separate resources for each team.

3. Ineffective planning for resource usage for integration: Ineffective planning with
respect to usage is a large contributor of environment turning instable, besides conflict
between teams. The most evident effect of this is that an issue that is noticed for a
particular once or twice may produce a completely different behavior in the following
runs for the same scenario. [3]
 Commonly faced challenges
4. Complex test configuration: The test bed or test environment configuration is
sometimes too complex. This will pose several challenges as the test team will need the
required skills to understand the needed configurations. In such cases the testers may
themselves induce an error in the test-bed by configuring it incorrectly. This would
greatly impact the test case and results it produces.

5. Elaborate setup time: In certain other times, for each test case the test setup may be
far too elaborate each test case identified. This could be due to a wide variety of co-
existent technologies that need to be coupled together or multiple components to work
together in cases of integration testing. In these cases, each of the components have to
be working perfectly to ensure consistent results, as one component may form an input
to the next. [3]
 Best Practices from TEM
Analyze these questions before getting started:

Do you need to be cross-platform, or do you support/test only on one platform?

Do you require complete automation, or can some manual intervention be tolerated?

How many virtualized machines do you need, and what are the required hardware
specs? Can you run the automation you need in this environment?
 Best Practices from TEM
The following practices have been compiled from several sources [3] [5] [6] [7].

Capture your Requirements

When a specifications document or a use case document is rolled out by the

development team, the invariable step for the test team is to understand the line item
requirements and then prepare a test case document detailing the test cases.

While the test planning is being carried out, it is the best practice to also include the
detailed test environment information in the test case document.

Assign Test Environment Management Ownership

The first step is establishing a healthy test environment management policy is

assigning ownership of the environment to an individual, team, or department.

Sharing ownership across an organization or multiple teams typically leads to

accountability concerns. The result is a test environment that is poorly managed and
irregularly maintained. Having a single manager or management group eliminates
these problems.
 Best Practices from TEM
Virtualize Your Test Environment

Virtualization technology allows testers and developers to create custom

instances or choose from existing template images dynamically.

Virtualization offers is the ability to take ‘snapshots’ of the test environment.

These snapshots contain configuration information and are extremely useful in
helping testers and developers recreate bugs.

Test Environment Assets Utilization

Consolidating environment estate is an exercise which consists of working

information of all assets, owners, contact details, vendor details, physical
location details, etc. This is a tedious exercise and till date one of the biggest
challenges at the enterprise level.

After consolidating the assets, the organization will have to focus on creating
application constellation based on patterns and should know how to simplify the
test environment landscape. Sharing test environments and ensuring that
resources, skills knowledge and assets are reused will minimize the operations
cost.
 Best Practices from TEM
Regularly Reassess Your Test Environment

Get into the habit of reassessing your test environment every ma jor test cycle.
Determine what is working, what is not working, what is being underutilized, and
what is being over utilized.

Keeping up with your test environment helps to ensure that it remains as accurate
and realistic as possible. Assigning ownership, maintaining detailed documentation,
and performing regular reassessments are three simple ways to streamline test
environment management.

Regression Testing

The test environments for regression testing seem to be running on the same test
setup with the same data, in actuality they are constantly evolving every release in
accordance to the features being implemented as well.

Every product release cycle would have one or more rounds of regression testing.
Thus establishing regression test environments for every product release cycle and
re-using them within the cycle, would definitely portray the stability of the test
environment.
 Best Practices from TEM
Automation

Developing automation frameworks and using automation for regressive tests, also
helps in improving the efficiency of a test environment because automation will
assume that the environment is stable and the defects that are originated are purely
feature/code oriented.

Automation strategies like Cloud technology and Service Virtualization allows the
organization can fulfil ad-hoc environment requests and ensure ‘on–time’ test
environment availability with lesser cost and effort.
 References
[1] https://en.wikipedia.org/wiki/Test_Environment_Management

[2] http://searchsoftwarequality.techtarget.com/answer/What-is-test-environment-
management

[3] https://leantesting.com/resources/test-environment-management-best-practices/

[4] https://www.apriorit.com/qa-blog/223-virtualization-in-testing

[5] http://www.computerworlduk.com/infrastructure/testing-environment-management-
challenges-with-virtualisation-1690/

[6] http://www.softwaretestinghelp.com/test-bed-test-environment-management-best-
practices/

[7] Critical Success Factors for a Successfull Test Environment Management by Wipro.
Next Steps

GETTING STARTED RESOURCES FREE TRIAL

Getting Started Guide Tutorials Start you 30-day free trial now
Documentation What we're reading this week
Support Blog