TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING

Service Validation and Testing:

A CA Service Management
Process Map

Malcolm Ryder
CA SERVICES / ARCHITECT
MARCH 2010
 Executive Summary
Challenge
The Information Technology Infrastructure Library version 3 (ITIL® V3) process
Table of Contents framework approaches IT Service Management (ITSM) in terms of the lifecycle of a
service. The Service Lifecycle is an organization model providing insight into the way
ITSM is structured, and embodies critical guidance for IT organizations seeking to
Executive Summary 1 improve service quality and align more closely with business goals to create value for the
business and its customers.
SECTION 1:
Finding the Right Path to IT Service Excellence 3
However, ITIL V3 best practice guidelines across the five phases of the service lifecycle
SECTION 2: are complex and challenging to interpret, and they are not designed to provide definitive
Service Validation and Testing 5 advice about implementing ITSM processes. Many IT organizations consequently
Adopt Best Practices undertake an ITIL journey without a firm idea of their goals and the path to achieve
Validate & Verify those goals.
Tracking Validations Versus Requirements
Perform Tests
Report / Closure
One of the key ITIL management processes, Service Validation and Testing, presents
its own special challenges. Participants in the service quality assurance effort must
SECTION 3: coordinate the priorities and extent of testing with the type and timing of business needs
Benefits 9 to be met; consequently, the measure of success in validation and testing extends beyond
A Key to Achieving IT Service Excellence the technical aspect to the matter of business value.

SECTION 4:
Conclusions 10

SECTION 5: Opportunity
About the Author 10
The primary objective of the Service Validation and Testing process is to ensure that a
service offering, as designed, deployed and supported, has the quality expected to meet
customer expectations.This quality is a strategic process outcome. CA has developed a
unique approach to representing the ITIL framework and its interdependent IT Service
Management (ITSM) processes at a high level in the form of an easy-to-use subway map.
This map is an ideal starting point for understanding and communicating about ITIL in
support of successful program planning and implementation.

Benefits
Following the Validation and Testing map provides guidance to:

• Improve business process capabilities

• Effectively upgrade the IT infrastructure in timely alignment with business needs

• Foster the efficient delivery of multiple IT services

• Improve the quality of technical support

Copyright © 2010 CA. All rights reserved. All trademarks, trade names, service marks and logos referenced herein belong to their respective companies. This document is for your infor-
mational purposes only. To the extent permitted by applicable law, CA provides this document “As Is” without warranty of any kind, including, without limitation, any implied warranties
of merchantability or fitness for a particular purpose, or non-infringement. In no event will CA be liable for any loss or damage, direct or indirect, from the use of this document including,
without limitation, lost profits, business interruption, goodwill or lost data, even if CA is expressly advised of such damages. TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING   1
 Simplifying ITIL
The ITIL V3 process framework focuses on the service lifecycle and the way that service
SECTION 1: CHALLENGE Finding the Right Path to IT Service Excellence
management components are structured and linked. It embodies critical guidance for
IT organizations that are seeking to improve service quality and align more closely with FIGURE A
business goals. CA has developed three maps:
Service Design Service Catalog
Service Transition Service Operation
Service Design, Service Transition and
Management
Schedule Maintain Accurate Service
Change Service Configurations
Availability Validation Incident
and Testing

But, the ITIL V3 best-practice guidelines across the five phases of the service lifecycle are
Release and Management
Management Maintain
Deployment CAB Review Secure
Service Access
Business
Management

Service Operation since most ITSM

Services Execute Prevent and
(Emergency/
Audit
Eliminate
Standard) Problems
Adopt
Best Practices
Test Evaluation/ Monitor / Track
Deliver Decision
Required Raise Incident
Monitor Resources Business
Services Planning Responsiveness Informed
Decisions Resolve
Validate &
Verify Event
Build
Request

complex and challenging to interpret. Moreover, they are not designed to provide definitive
Record
Management

discussions focus on these critical

Impact
Methods/
Specify
Continuity
Adjust and Tune Analysis Fulfillment
Techniques Requirements Transition
Planning and Status
Reports
Service Request
(Incl. Self-Service) Error
Set Security Support CH Control CH
Controls O Service Portfolio
Management
O Service Portfolio
Filter /
Correlate
Detect
(Incl. Fault Detection)

D
Management

EC

EC
Develop
Strategy Ensure Provide Executive Select Response

K
Assess & Forecast Model/Trend
Classify Assets Requirements Preparation Release Rights Policy
Transfer/

ITIL disciplines
Value Disseminate Review/
Document Action
Perform

advice about implementing IT Service Management (ITSM) processes. Many IT organizations

Transform
Service
Definition CH Preparation
and Planning
Tests Verify to Usable Record
Info. Security O Service Portfolio Build Verify
Knowledge

D
Management

EC
Maintain Schedule
Management Policy
Automate Investigate

K
Analyze Knowledge
Performance Access Request and Control
Build Catalog Transfer
Analyze Contents
Store Info.
Test
Manage
Security Incidents
Diagnose
Problem
Assess Risk Demand Financial Deploy Demand Financial
Management

AN
AN
Management Management Management Management
Proactive

A
Escalate

A
Review Prioritize Configuration CT CT
Capacity Determine
Management and Audit Change PL Control
PL
Management Build Plan Vulnerabilities Management Manage Access

consequently undertake an ITIL journey without a firm idea of their goals and the path to
Mitigate Risk Categorize
Build/Release
Business
RFC
Analysis Management Problem Known
Demand
Service
Views Financial Service Level Control Errors

AN
Monitor

A
Management Management
Management
CT and Report
PL Technical Optimize
Monitor Service Availability
Demand Views
Report/
Coordinate Closure Capture Info. Work
Resources Approval Around
(Financial, Compliance)
Revise Service Customer Monitor Design SLA
SLAs / OLAs Review Satisfaction Performance Identify
Framework Configurations Fulfillment
Manage Publish
Live Services

achieve those goals.

Issues Resolve/
Manage
Deliver Recover
and Plan
Assure Standardized
Catalog Quality Services
Operational Service Catalog Management Key Intersections Service Validation and Testing Key Intersections Problem Management Key Intersections
IT Service Services
Report
Continuity
Service Level Management
Strategic Controls
Service Asset and Configuration Incident Management Restore
Service
Knowledge
Achievements Management Strategic Controls Strategic Controls
Management
IT Service Continuity Management
Event Management Management
Capacity Management Strategic Inputs Change Management
Strategic Inputs
Service Asset and
Strategic Inputs
Meet Business Transition Planning and Support Configuration Request Fulfillment
Info. Security Management
Requirements Continual Service
Release and Deployment
Continual Service Management Knowledge Management Continual Service
Improvement Improvement
Availability Management Management Improvement
Access Management

CA has developed a unique approach to charting the ITIL journey through a visual
representation of the ITIL framework and its interdependent ITSM processes modeled after FIGURE B
an urban subway system. This three-part map (Figure A) presents an easy-to-navigate,
The Service Transition map represents
high-level view of the ITIL terrain. IT executives, strategists and implementers can use these a journey of developing and improving Service Transition
Service Management process maps along with the family of CA Service Management process capabilities for the transition of new Release and
Schedule
Change
Maintain Accurate
Service Configurations
Service
Validation
and Testing

map technology briefs that expand on them. The maps and technology briefs provide a
CAB Review
Deployment
and modified services to production. Management

Evaluation/
Execute
(Emergency/
Standard) Audit
Adopt
Best Practices

common reference point for understanding and communicating about ITIL and help you with
Decision
Business
Planning Responsiveness
Validate &
Verify

program planning and implementation.

Impact
Analysis
Transition
Planning and Status
Reports

Support CH
O Service Portfolio
Management

EC
Develop
Strategy Ensure

K
Preparation Release
Value
Perform
Preparation Tests
and Planning

How to Use the CA Service Management Process Maps

Build Verify
Schedule

Knowledge
Transfer

CA’s Service Management process maps (Figure A) illustrate every process (or track), each
Demand Financial Deploy

AN
Management Management

A
Prioritize Configuration CT
Change PL Control

Management Manage
Build/Release
RFC Categorize

activity (or station) and the key relationships that are relevant to navigating continuous IT
Analysis

Monitor
and Report

Report/
Coordinate

service improvement. The ITIL quality cycle takes the form of a “circle” with each Plan-Do-
Closure
Resources
Identify
Configurations

Check-Act (P-D-C-A) step as a process integration point (junction) on the line. Junctions
Manage Assure
and Plan
Quality
Service Validation and Testing Key Intersections

Service Asset and Configuration

Management Strategic Controls

serve both as reference points when assessing process maturity, and as a means to consider Service Asset and
Change Management
Strategic Inputs
Transition Planning and Support Configuration
Release and Deployment
Continual Service Management
Improvement

the implications of implementing a process in isolation.

Management

Strategic controls (Service Portfolio Management, Demand Management and Financial

Management) are needed to reduce risk and optimize integration across the service lifecycle, Most organizations are highly sensitive to quality levels in IT deployments, and they already
as illustrated on the three points of the triangle centered in the P-D-C-A quality circle (seen rely on continuously managed processes and practices that are expected to provide some
more easily in Figure B). These strategic controls help in evaluating, prioritizing and assuring assurance of the fitness of deployed components and systems for their purpose. For that
the appropriate levels of financial and human resources for existing and new services. matter, validation most often means verifying that component and systems specifications are
appropriate to established norms for operational risk. Testing takes place under the watchful
This paper is part of a series of Service Management Process Map technology briefs. Each eye of domain and subject matter experts in the line of duty for resolving problems and
brief explains how to navigate a particular ITIL process journey, reviewing each process executing changes.
activity that must be addressed in order to achieve process objectives. Along each journey
careful attention is paid to how technology plays a critical role in both integrating ITIL Yet as we see in the process interactions shown on our subway map, the circuit from planned
processes and automating ITIL process activities. modifications to deployment releases is an incomplete journey. This is because builds and
installations may or may not go according to expectations. But once utilization begins, quality
requirements differ again, and acceptance of the deployments and releases depends on
further steps defined more from a business user’s perspective than from a technical one.

2  TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING  3
 Alignment and compliance with the business is the overall goal of validation and testing,
mandating explicit awareness of business requirements and coordination with them. Key SECTION 2: OPPORTUNITY Service Validation and Testing
points in this coordination include: The Service Validation and Testing process is responsible for planning and coordinating tests
• Cooperation of specific business and technical roles to ensure that specifications for the service design are met and validated through delivery
of the service and, including co-operation with the Release and Deployment process, to
• Documentation of requirements at different levels of service, and corresponding manage and limit risks that could result from insufficient utility and warranty of the service in
documentation of the plan for validations operation. Service Validation and Testing takes the technical proficiencies and plans of quality
• Tracking the validations versus the requirements assurance for IT and strategically places them in the larger context of service quality for
the business.
Another question that comes up is about how much testing is enough. Too much is difficult
to justify, and too little is too risky. One of the main outcomes of the Service Validation and The strategic issue is that the business must be able to grow and adapt in order to maintain
Testing process is an increase in the ability to recognize a return on the investment in testing, its success and critical competencies in its market ecosystem of customers, partners and
which encourages the inclusion of right-sized effective testing in the business-level planning competitors. This makes both Capacity and Change strategic areas of risk, and consequently
and budgeting for IT Services development. it makes investments in service deliveries and service modifications part of financial
management as well as business risk management.

The ultimate goal of the Service Validation and Testing track (Figure C) is a full alignment of
Service characteristics to the utility and warranty requirements of the business. The subway
map for this journey plots the key stations along the course in this way:
• Adopt Best Practices
• Validate and Verify
• Perform Tests
• Closure and Reporting

FIGURE C
The Service Validation and Testing
track represents management’s
attention to how verification of service
characteristics is proactively aligned
with the requirements for utilization
and warranty of an upcoming new
deployment of service

4  TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING  5
 Adopt Best Practices Tracking Validations Versus Requirements
The key practices of Service Validation and Testing will be • Specs decompose from top down—business to service to supplier
• A model for identifying business alignment • Tests build from bottom up—components to functions to business agreements
• An integrated set of test plans
• Training, knowledge transfer, and communications
SPECIFICATIONS VALIDATION TESTS KEY ROLES
• Verification of results through pre-defined tracking and reporting
Business Requirements Alignment to Business •• Customer
These practices are represented in the following table, which illustrates the incorporation Agreements
of testing within a larger process framework (inputs, activities, and eventual outcomes) of •• Compliance Manager
identifying business value from quality assurance. Service Requirements User Acceptance •• Service Level Manager
The inputs for this high-level process express two things: a proactive identification of the •• Test Manager
customer’s view of a service, and the perspectives of the testing organization as driven by •• Test Manager User
that view. Explicit definitions and standardized formulations of the relevant factors across
the process are reusable, which allows for direct comparisons of different testing occasions. Service Solution Design Operational Readiness •• Change Manager
The results of such comparisons create an awareness that supports prioritization of service •• Security Manager
deliveries; and they inform analyses for the improvement of quality management procedures.
Service Release Release Package •• Release Manger
QUALITY ASSURANCE Requirements
Inputs Validations Outcomes
Development Components with Assembly •• Technical and Subject Matter
Service Quality Policy Service Package Delivered Value Experts
• Value Expectations • Description of Designed IT Service • Available and Supportable

Service Level Package Utility Level Warranty Level Release for Targeted Qualified Supplier(s)
• Intended Utility and (fit for purpose) (fit for use) Line of Service
Warranty

Requirements and Targets

• Measures
Usability Compliance Certification Perform Tests
The essence of manageable testing is the ability to define tests as repeatable,
measurable procedures.
Test Strategy Specifications Buisness Needs Quality Rating and
• Plans, Model Results Report
• Resources This introduces the aspect of Key Performance Indicators (KPIs). As noted in most literature
• Methods, Scripts
on performance management, KPIs are metrics that address efficiency, effectiveness and
cost issues traceable to the design of an executing process and related to the outputs of that
process. Indicators identify output characteristics that correlate to desired outcomes, and the
In the table above, items such as Service Level Packages (SLPs), Test Plans, and Reports are
means of driving those indicators will be seen as Critical Success Factors (CSFs).
typical artifacts of the ITIL management process and can be researched in detail within most
substantive ITIL publications. A service management organization will want to reach a level of Accordingly, testing will feature standards, scripts (methods) and metrics that are defined and
confidence that the various items exist and are clearly related to each other. This will call for logically related to each other before test execution actually begins.
assigning clear responsibilities for routine production and review of the items.
Further, as a matter of setting scope to testing as well as methodological compliance, policies
will be established to govern the decisions about when testing must occur and what type
Validate & Verify
must occur. As identified in the table above, key roles in the organization take on responsibility
The outstanding feature of Service Validation and Testing is its deliberate synchronization of
and authority as stakeholders in the specifications to be validated. In that light, cooperation
test types with defined business requirements, agnostic to particular technologies and to the
amongst the roles is expected and directed by policies, while also being institutionalized as
variations of hardware and software. The following illustration presents an overview of
integrations of their respective management processes. At points of process integration, the
this synchronization.
processes may feed each other as participants in shared workflow. In the ITIL subway map of
processes, this is notable especially where the Build and Release of a service package involves
validation and testing.

6  TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING  7
 One of the most important aspects of these process integration points is the documentation
that occurs and is used from validation and testing. For example, a test plan usually precedes SECTION 3: BENEFITS
release and deployment activities, and an update of service records in a configuration
management database (CMDB) usually follows deployment but precedes a post- ITIL v3 describes Service Design, Service Transition (deployment), and Service Operation as
implementation review (PIR). This arrangement highlights the cooperation expected between three major phases in the service lifecycle. Within Service Transition, the Service Validation
managers of services, configurations, changes and releases. and Testing effort is one of several high-level management processes that link the appropriate
design of the service to the smooth operability of the service.
Report / Closure IT Services are the building blocks of business processes and business services. A successful
Documentation of results is also a fundamental deliverable of the Service Validation and delivery of a service will feature the right thing arriving at the right place at the right time for
Testing effort, as testing itself is expected to be the source of evidence that the service is the right reason. Validation and Testing confirms that the service is fit to its target purpose
not ready for delivery and may go through another iteration of development or modification and fit to its offered use. The business uses multiple services to support its own performance
before it can succeed. and growth. Executives own the service portfolio, set service goals and control the investment
in the service. This executive demand and influence thus should sponsor the qualification of
As part of proper management of testing itself, these cycles of tests need directives
the service as a primary concern of financial and risk management for the business.
identifying and justifying whether the scope and depth of testing is to be different or not in
the next cycle versus the prior one. Because test cycling directly affects the turnaround time The executive view of a successful service implementation includes concerns about the
between development and release, the management of the test iterations affects the ability of control, compatibility and delivery of the service. These three factors translate, respectively,
the organization to meet service level agreements with customers. into management of the quality, risk, and value of the provided service, as deliberately
generated from end-to-end throughout the service lifecycle .
Test reporting explicitly establishes the reasons why something is tested and why it is deemed
suitable for release or not. As material for decision support, the reporting affects resource The goal of Service Validation and Testing is to ensure that specifications for the service
allocations as well as readiness for compliance auditing. design are met and validated through delivery of the service and, including co-operation
with the Release and Deployment process, to manage and limit risks that could result from
insufficient utility and warranty of the service in operation.

A Key to Achieving IT Service Excellence

Automating ITSM through technology can help your organization reduce the amount
ofresources required to achieve ITIL v3 best practices. This assists your IT department in
improving the quality of its services while embarking upon a continuous IT service excellence
program focused on fostering business growth.

As you reach the end point of the Validation and Testing journey outlined in the CA Service
Transition process map, your organization should have a better handle on the steps needed to
promote successful service deployments. Specifically, bringing deployment efforts in line with
ITIL best practices can help you:
• Improve business process capabilities
• Effectively upgrade the IT infrastructure in timely alignment with business needs
• Foster the efficient delivery of multiple IT services
• Improve the quality of technical support

8  TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING  9
SECTION 4: CONCLUSION
Service Validation and Testing facilitates ITSM by managing the certification of IT in the
form of services needed by the business. However, doing so requires rigorous processes,
appropriate communication plans and support commitments—as well as an outline detailing About CA: CA (NASDAQ: CA) is the world’s leading
where to start and how to proceed through the journey. independent IT management software company. With
CA’s Enterprise IT Management (EITM) vision and
Following the steps outlined in the CA Service Transition process map gives organizations a
clear view of how their Service Validation and Testing journey will take shape, illustrating the expertise, organizations can more effectively govern,
key stops en route to achieving effective requirements alignment. This journey results in: manage and secure IT to optimize business performance
• Strategic re-orientation of testing towards business value and sustain competitive advantage. For more information,
visit www.ca.com.
• Better prioritization and investment in services
• Improved risk management
• Stronger integration with other ITSM and ITIL best practices

SECTION 5: ABOUT THE AUTHOR

Malcolm Ryder has over 25 years experience in the IT industry, with expertise in the areas
of service transition, IT value management, and IT strategy. For the last 15 years, Malcolm
has worked in consulting and solution strategy roles with a heavy emphasis on service
management systems with vendors, service providers and end-user customers. Malcolm has
been a co-developer of multiple market-leading commercial ITSM solutions since the
mid ’80s.

This document shall not serve to (i) affect the rights and/or obligations of CA or its licensees under any existing or future license agreement or services agreement relating to any CA
software product; or (ii) amend any product documentation or specifications for any CA software product. The development, release and timing of any features or functionality described
in this document remain at CA’s sole discretion. Notwithstanding anything in this document to the contrary, upon the general availability of any future CA product release referenced in
this document, CA may make such release available to new licensees in the form of a regularly scheduled major product release. Such releases may be made available to current licensees
of the product who are active subscribers to CA maintenance and support, on a when and if-available basis. In the event of a conflict between the terms of this paragraph and any other
information contained in this document, the terms of this paragraph shall govern.

Copyright © 2010 CA. All rights reserved. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Intel and Intel Itanium are trademarks or
registered trademarks of Intel Corporation in the United States and other countries. Microsoft, SQL Server .Net , Visual Studio, Internet Explorer, Windows Vista and Windows are either
registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. UNIX is a registered trademark of The Open Group in the United States and
other countries. Java and all Java-based trademarks and logos are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. ITIL® is a Registered Trade Mark, and
a Registered Community Trade Mark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office. Red Hat® and Red Hat® Enterprise Linux® are trade-
marks or registered trademarks of Red Hat, Inc in the United States and/or other countries. SUSE is a registered trademark of Novell, Inc., in the United States and other countries. Eclipse
is a trademark of the Eclipse Foundation, Inc. SPARC is a trademark or registered trademark of SPARC International, Inc. in the United States and other countries. Adobe and Dreamweaver
are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States and/or other countries. Mozilla® and Firefox® are registered trademarks of The Mozilla
Foundation in the United States and/or other countries. Solaris is a register trademark of Sun Microsystems in the United States and other countries. IBM, z/OS, z/Architecture, CICS,
WebSphere. DB2, MQSeries, System p5 , System p6, AIX and DB2 Universal Database are trademarks or registered trademarks of International Business Machines Corporation in the
United States, other countries, or both. All trademarks, trade names, service marks and logos referenced herein belong to their respective companies. THIS DOCUMENT IS FOR YOUR
INFORMATIONAL PURPOSES ONLY. TO THE EXTENT PERMITTED BY APPLICABLE LAW, CA PROVIDES THIS DOCUMENT “AS IS” WITHOUT WARRANTY OF ANY KIND, INCLUDING,
WITHOUT LIMITATION, ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. IN NO EVENT WILL CA BE LIABLE
FOR ANY LOSS OR DAMAGE, DIRECT OR INDIRECT, FROM THE USE OF THIS DOCUMENT, INCLUDING, WITHOUT LIMITATION, LOST PROFITS, LOST INVESTMENT, BUSINESS
INTERRUPTION, GOODWILL OR LOST DATA, EVEN IF CA IS EXPRESSLY ADVISED IN ADVANCE OF THE POSSIBILITY OF SUCH DAMAGES.
ID1688-032910

10  TECHNOLOGY BRIEF: SERVICE VALIDATION AND TESTING