2

Introduction
Thank you for downloading this guide, which aims to help with the
development of a plan for a data migration. The guide is based on our
years of work in the data movement industry, where we provide off-theshelf software and consultancy for organisations across the world.
Data migration is a complex undertaking, and the processes and software
used are continually evolving. The approach in this guide incorporates
data migration best practice, with the aim of making the data migration
process a little more straightforward.
We should start with a quick definition of what we mean by data
migration. The term usually refers to the movement of data from an old
or legacy system to a new system. Data migration is typically part of a
larger programme and is often triggered by a merger or acquisition, a
business decision to standardise systems, or modernisation of an
organisations systems. The data migration planning outlined in this
guide dovetails neatly into the overall requirements of an organisation.
Dont hesitate to get in touch with us at [email protected] if you have
any questions.

www.etlsolutions.com

Contents
Project scopingPage 5
Methodology.Page 8
Data preparationPage 11
Data security.Page 14
Business engagement.Page 17
About ETL Solutions.Page 20

www.etlsolutions.com

Chapter 1
Project Scoping

www.etlsolutions.com

Preparing a Data Migration Plan | Project Scoping

While staff and systems play an important role in reducing the risks
involved with data migration, early stage planning can also help. It
identifies potential issues that may occur later in the project, enabling the
organisation to plan the mitigation of risk.
Our consultants thoroughly review and scope a project before it starts.
We find its practical to divide the review into two parts: the projects
structure and its technical aspects.

The project review should evaluate the following areas:

Are the deliverables and deadlines clearly defined?

Is the budget sufficient?
Have all potential stakeholders been included in the plan?
Are there communication plans in place, and do they include all
stakeholders, senior management and, if necessary, the wider
organisation?
Are there personnel in the right number and with the right skills?
Will they be available for the duration of the project? Specifically,
are there sufficient:
o

o
o

Business domain experts?

System experts?
Data migration experts?

www.etlsolutions.com

Preparing a Data Migration Plan | Project Scoping

The technical review usually assesses the quality and fit of:

The proposed migration methodology and workflow

The data security plan
The software available:
o

o
o

Technical features
Flexibility
Fit with the skills of the people working on the project.

The volume and cleanliness of the data to be migrated.

Analysing these aspects in the early stages of a project will help to

reduce risk and realise best practice. It also provides supporting
evidence when requesting additional funding or other resources.

www.etlsolutions.com

Chapter 2
Methodology

www.etlsolutions.com

Preparing a Data Migration Plan | Methodology

A clear methodology is essential for a staged, well-managed and robust

approach to data migration. According to a 2011 report by Bloor, 38% of
data migration projects run over time or budget. The report identifies an
effective methodology as one of the ways to minimise these risks.
However, industry-standard data migration methodologies are scarce.
One option is the Practical Data Migration methodology developed by
industry expert Johny Morris, which consists of training and certification.
Alternatively, most companies who provide data migration services have
their own methodology; ours consists of pre-migration scoping, project
assessments and a core migration process.
The complexity of data migration means that a chosen methodology can
seem like a sea of options, which can be difficult to get all the
stakeholders to buy into. Focus on the most startling element of the
migration the fact that the legacy system will be turned off and the
attention of the stakeholders is guaranteed.

www.etlsolutions.com

Preparing a Data Migration Plan | Methodology

A robust methodology should include:

Extract design: how the data is extracted, held and verified

Migration design: how data is transformed into the target structure
Mapping rules: the details of the migration
Test overview: tools, reporting, structure and constraints
Unit test: unit test specification
Integration test: integration test specification
Recovery plan: recovery options for each stage of the migration
Go live plan: actions required to go live.

Standards are used to identify problem areas early on, making sure that
the project dont reach the final stages with a hundred different issues to
sort out. For instance, at ETL Solutions we have the Prince2
management standard, and use ISO standards where appropriate to
underpin our data migration methodology.

www.etlsolutions.com

10

Chapter 3
Data Preparation

www.etlsolutions.com

11

Preparing a Data Migration Plan | Data Preparation

It is crucial to thoroughly prepare data and systems before a migration

takes place. In particular, landscape analysis is an important part of
preparing for a data migration. It provides an overview of the source and
target systems, enabling the project team to understand how each
system works and how the data within each system is structured. These
areas should be reviewed systematically to ensure that potential errors
are identified in advance of the migration. Ideally, the team should model
the links and interactions between the different systems involved, along
with the data structures within each system.
Another important component of thorough preparation is data assurance.
This procedure validates the data discovered in the landscape analysis
and ensures that all data is fit for purpose. By validating the data, the
migration team are then free to focus solely on structural manipulation
and movement. Data assurance has several phases: data profiling; data
quality definition; and data cleansing.
Data profiling
The aim of the data profiling phase is to ensure that any historical data
due to be migrated is suitable for the changes that are taking place in the
organisation. Profiling should be carried out to identify areas of the data
which may not be of sufficient quality. It should include comprehensive
checks of existing model structure, data format and data conformance.
A retirement plan should be used to define the data no longer required.
Any data to be retired should be recorded, along with a description of
what replaces it or why it can be removed. The data that is no longer
needed may have to be archived for tax purposes or to meet the
requirements of an industrys governing bodies.

www.etlsolutions.com

12

Preparing a Data Migration Plan | Data Preparation

Data quality definition

Data quality definitions state the quality that must be attained by
elements, attributes and relationships in the source system. The
definitions or rules should be used during profiling to identify whether or
not the data is of the correct quality and format. All data quality rules
should be listed at element level, such as data table or flat file. All data
quality issues and queries should be tracked and stored.
Data cleansing
The first stage in data cleansing is to define which cleansing rules will be
carried out manually and which will be automated. Splitting the rules into
two enables the organisations domain experts to concentrate on the
manual process, while the migration experts design and develop the
automated cleansing. Typically, the manual cleansing will be carried out
before the migration, while the automated cleansing may be carried out
before the migration or as part of the migrations initial phase.
Data verification is the part of the data cleansing process that checks
that the data is available, accessible, complete and in the correct format.
Our consultants often continue to carry out verification once a migration
has begun, ensuring that the information is optimised prior to each stage
of the migration.
We find that data impact analysis is a crucial part of data cleansing.
Because cleansing data adds or alters values, data impact analysis
ensures that these changes do not have a knock-on effect on other
elements within the source and target systems. It also checks the impact
of data cleansing on other systems which currently use the data, and on
systems which may use the data once the migration is complete.

www.etlsolutions.com

13

Chapter 4
Data Security

www.etlsolutions.com

14

Preparing a Data Migration Plan | Data Security

Data security has become a political and legal issue, particularly with
continuing high-profile data losses. Carrying out a data migration is likely
to require access to corporate or customer data that is likely to be
sensitive and business critical.
It is crucial that all data is treated with respect. All sensitive information,
including customer data, should have detailed levels of security in place.
Before you start any data migration, check exactly what levels are in
place, and who is allowed access to the data and when.
Assess the value of the data to the business, in addition to the costs that
could arise from a security breach. Then make sure that the security
requirements of the migration reflect this value. They should be costeffective and not outweigh the risks highlighted in the assessment.
Legal obligations should be thoroughly checked. Statutory measures
covering data breach and data protection are now in place in many
sectors. These often outline the areas of security that have to be in
place, as well as stipulating operating procedures to keep the data
secure.

www.etlsolutions.com

15

Preparing a Data Migration Plan | Data Security

Draw up data security plans early on and embed them in the data
migration plan. Areas to consider include:

How to ensure secure data transfer

How to create secure server access
How to ensure secure data access
Whether or not to increase the number of permissions required to
transfer data
Clearance and vetting of personnel, including outside consultants
and partners
The training or information sessions required by personnel
Vetting of the software that will be used for the migration.
Protocols for the use of email and portable storage devices.

www.etlsolutions.com

16

Chapter 5
Business Engagement

www.etlsolutions.com

17

Preparing a Data Migration Plan | Business Engagement

The backing of senior business leaders will improve the chances of a data
migration project going smoothly and ensure that you have the resources
you need. The key is to remember that the purpose of the migration is to
make the overall business more effective and efficient, and to ensure that
this is communicated properly.
Here are a number of ways to encourage the endorsement of senior staff.
Align the project with business priorities
The project results should reflect the areas on which business leaders
tend to focus. These are predominantly revenue and cost. Senior
managers dont always want to spend the budget (and staff resources)
required for an effective migration, so they need to be convinced that
real, monetary gain lies in project success.
Manage expectations
Be honest about how long the project is going to take and what will be
asked of management along the way. Make it easy for senior managers
to delegate by ensuring regular communication and sign-off stages along
the way. In this way, business leaders feel in control without having to
micro-manage.
Link the benefits to specific business issues
Show how current challenges within the business will be helped by the
data migration project. Illustrate how you are following a methodology
that will not only satisfy the data requirements of the new system, but
also the business requirements for the new data within the system. And
talk in terminology that management can understand!

www.etlsolutions.com

18

Preparing a Data Migration Plan | Business Engagement

Promote best practice

Great processes can reflect positively on a companys senior
management. Show in the scoping and strategy documents at the outset
how the migration process uses best practice and even, where
applicable, accreditations. Best practice can tend to fade during a
project as time constraints loom, so make sure that the right processes
are followed all the way through. Take a look at our methodology to get
ideas on best practice implementation.
Build in short and long-term gains
Senior business leaders are likely to want to see short-term value added
to their bottom line after making an investment in data migration. Often,
data migration projects reveal their value over time, so make sure you
build in some quick wins to satisfy business objectives.
Communicate the system retirement plan
Be clear about what will happen to existing business resources after the
migration. Explain how any changes can mitigate the costs of the
migration itself:

Recovering licences for software that is no longer used

Machine disposal
Cancellation of support contracts
Redeployment of staff
Closure of buildings.

The retirement plan will provide reassurance that the project can be
closed without any issues or additional cost. A quick shutdown of the
project post-migration enables the freeing of resources, including
personnel, for other projects.

www.etlsolutions.com

19

About ETL Solutions

www.etlsolutions.com

20

Preparing a Data Migration Plan | About ETL Solutions

We are experts in complex data migration and integration

At ETL Solutions, we design software to help developers tackle difficult
data transformations. We deliver ready-to-use products and services
based on Transformation Manager, a robust integration toolkit.
Transformation Manager is an off-the-shelf
software suite for developers. It has been proven
across multiple industries, even in the most
demanding data movement situations.
Transformation Manager provides a complete
Integrated Development Environment for
specifying, developing, testing, debugging and
deploying transforms.

Click here to request an

online demo of
Transformation Manager,
our data migration software

Our staff are experts in data migration

and integration it is all that we do
and our team of consultants deliver costeffective solutions to clients worldwide.

www.etlsolutions.com
Images from freedigitalphotos.net

21