Portals eNewsletters Web Seminars dataWarehouse.com DM Review Magazine
DM Review | Information Is Your Business
   Information Is Your Business Advanced Search

Business Intelligence
Corporate Performance Management
Data Integration
Data Quality
Data Warehousing Basics
Master Data Management
View all Portals

Scheduled Events

White Paper Library
Research Papers



DM Review Home
Current Magazine Issue
Magazine Archives
DM Review Extended Edition
Online Columnists
Ask the Experts
Industry News
Search DM Review

Tech Evaluation Center:
Evaluate IT solutions
Buyer's Guide
Industry Events Calendar
Software Demo Lab
Vendor Listings

About Us
Press Releases
Advertising/Media Kit
Magazine Subscriptions
Editorial Calendar
Contact Us
Customer Service

Data Migration: Plan to Succeed

  Article published in DM Direct Newsletter
January 19, 2007 Issue
  By Ashish Nadkarni

Data migration is the process of migrating data from one location to another. It is not something new to the storage industry. Almost every IT organization has had to move data at some point or the other. There are different reasons for doing so: hardware gets obsolete and has to be upgraded, data gets bigger and cannot be handled by the array, the environment is being relocated, etc. As data becomes more complex and difficult to manage, the process of migrating data has taken on a life of its own. Consulting services organizations have made a business out of it. Data migration is no longer about knowing the technology, but about managing the process properly, in a manner that meets the requirements of your internal and external customers. Typical requirements include minimum downtime, (who likes downtime these days, anyway?) reconfiguration of any affected component, and the availability of resources and skills to adequately perform the migration. Whatever the reason may be, there are specific aspects of a data center move that need to be addressed to ensure success.

Before listing the specifics of what data migration entails, let us first define it in the context of data storage. Data migration is a one-time activity that is undertaken for a specific purpose. The purpose can vary from organization to organization, but it must justify the need to undertake the endeavor in the first place. Because any time you migrate data there is an element of risk to the business, if you find yourself migrating the same data all the time, you have other issues in the environment that warrant examination.

The second important point in any migration is that the type of data you are migrating influences what migration method you select. For example, if your database is being migrated using simple copy tools available to any user, and it is not shut down during the process, whatever you have copied may not be usable. The method must align with the data you are migrating.

What is a successful data migration? The ultimate goal is to get the job done with minimum impact to your environment.

Plan for success. A data migration exercise, no matter how small it is, can have undesired consequences if not planned properly. No IT manager likes surprises, least of all being up at night worrying about data loss or unexpected downtime because of poor planning. Don't cut corners - get a project manager involved . If your company does not have a project manager who is familiar data migration processes, look at bringing in a resource. Project managers with data migration expertise are out there. A project management expert should create a detailed project plan that includes tasks, subtasks timelines, roles, responsibilities and dependencies. All the parties involved should have an opportunity to voice their concerns (in a controlled manner) and key requirements are factored in while building the plan. A critical component of the project plan is also a well-planned test and back-out plan.

Get the experts. While moving data may sound like any other IT project that can be handled internally, the reality is, it is not. This is especially true if the migration involves moving data from one physical resource to another or when you are moving it from one geographical location to another (i.e., a data center migration). Bringing in outsourced experts does not mean they replace the value that your internal resources bring to the table. Bringing in an outside resource that specializes in that technology allows your staff to focus on the day-to-day operational activities while the added, dedicated resource focuses on the details around the migration itself.

Know your objectives. This may sound repetitive, but before you embark on a data migration project, you must be clear about the reasons you are crossing that bridge. Is it technical or purely financial? How much money do I have to spend on this project? What cost savings will the migration effort bring about? What sort of impact does the business experience when I do the migration? How much downtime can the business tolerate? Have I evaluated all my options before making the decision? A detailed analysis of all the reasons for the data migration, and their impact on the business, will drive you to the set of objectives to be applied to planning the process. Do not start looking at the technical options before you have completed this exercise. Do not even engage any third-party organization before you are clear about what the objectives are and what they will help you accomplish in the short, medium and long term.

Know your options. There are many different ways to migrate data. There is no such thing as the right way or the wrong way. The method selected depends entirely on your requirements, such as downtime, resource availability and the amount of data you're migrating. You don't need to know every method in minute detail, but knowing the options will help you make an informed decision about how best to accomplish your objectives. If you are bringing in an outside resource to help you make the decision, keep in mind that while they may bring a fresh perspective, that resource may also prefer certain migration techniques if they are aligned with a storage vendor. In such cases, vendor-independent organizations may be the best suited to provide an unbiased analysis of your environment and the methods that are most appropriate to migrate your data. Also, keep in mind that there is no such thing as a one-size-fits-all data migration technique for all data types. The one-size-fits-all approach has its advantages and disadvantages. Among the advantages is the ability to use the same consistent migration process across all platforms and applications. The disadvantages are the lack of flexibility to downtime and lack of customization. This approach may be best suited for large-scale migration efforts such as a data center move or an array-to-array migration, whereas other customized approaches will be preferred for small migrations, such as an application or database migration for a very specific purpose (like a server upgrade or platform migration).

Get an executive sponsor. I am generally not into "marketing speak," but having a senior member of management involved in a large-scale migration effort helps with visibility. It focuses the proper attention from all the internal groups, and priorities for all the individuals working on the project, internal or external. The executive sponsor assists during vendor escalations and facilitates quorum in the case of a difficult situation. He/she can also empower the project team to do the right thing by providing oversight during planning and execution.

Don't be intimidated (take your time). Yes, there are timelines and deadlines, but if success is what you are after, do not let anyone dictate timelines to you. This is especially true in situations where you have to move off some equipment because its lease is about to expire and the vendor is on your doorstep asking for their equipment. Get an extension on the lease so you don't have to hurry up. A little money spent extending the lease is worth it, considering the amount of money your business may lose because of unexpected downtime or data loss. On the other hand, you have to be proactive in identifying potential data migration issues in your environment before they become "hot." As part of your IT best practices, keep a record of leases, performance metrics and service level agreements (SLAs), which help you make a pre-emptive decision to move your data to a safer location before there's a threat!

Now that we have looked at the soft side of data migration, let us examine the technology behind the mobility. One word of caution - technology is constantly changing, so what works best today may not be true tomorrow.

Online migration. At the very center of the data migration and, to a large extent marketing hype from vendors, is the ability to do an online versus offline data migration. Online migration allows you to keep your applications online and running (in normal or slightly degraded operating conditions) while the migration is taking place. This is possible only if the technology used is capable of moving data in a manner transparent to the application accessing data. This happens when the technology operates at a level below the file system on which the application objects reside or if the application itself has the ability to move data to another location by "duplexing" its writes (i.e., writing to the source and destination at the same time). After the source and destination are in a synchronized state, the technology then switches the source and destination flags so that all new writes go to the new location. Examples of this type of migration are host-based volume managers.

Offline migration. Offline migration, on the other hand, is the antithesis of what we described above. In these migrations, all data is assumed to be at rest; that is, it is not changing nor is any application accessing it. On the face of it, such migrations are the easiest to perform. You simply shut down your entire environment, copy all the data from the old to the new location by whatever method you please and then once the new location is ready, you simply switch the pointers to the new location. Pointers can be anything: mount points, drive letters, symbolic links, IP addresses, etc. In reality, however, this migration is not feasible if you are migrating large amounts of data in a mission-critical environment. The quantity of data can require extended downtime, a luxury you do not always have.

Hybrid migrations. In an ideal world, all migrations could be done as pure online migrations without any outage at all. Reality is usually different. Many times, you have to update software on the host to facilitate the change, and sometimes the host itself may require a reboot to "see" the new storage. Sometimes an online migration can require a small outage to switch the source and destination. Usually, such outages are only to facilitate the source and destination switch. Examples of the latter are array-based replication technologies, appliance and/or software-based migration solutions.

Here are some of the most commonly used techniques for a migration:

Host based: All operating systems provide basic tools to copy data from one location to another whether the location is on the same server or on another server that is accessible via IP. Additionally, a lot of operating systems these days have IO abstraction software known as logical volume management (LVM). Logical volume managers are great for seamless data migrations because they allow you to add "mirrors" to existing volumes (objects that are used to perform file system operations instead of raw devices). A volume-manager based migration can literally be performed as a zero-downtime migration if the host itself is not being migrated.

Application based: Applications themselves can have a built-in copy function that allows you to duplex incoming writes to another location. Examples of such applications are databases like Oracle. Oracle Dataguard can be used to create an exact (and synchronized) database copy at a remote location, at a predetermined time, and all pointers can be switched from the local to the remote copy.

Appliance based: In such migrations, an appliance is usually inserted in the data path between the host and the storage array. The appliance then transparently duplexes data access to another location. During a predetermined outage window, the host is then made to point to the new location.

Array based: There are usually two subcategories here: array to array techniques are when you are migrating off an older array; or within the array itself, if you are migrating from older sets of disks to newer ones. Both methods are vendor-proprietary and, in most cases, require special licenses and services to implement. However from a one-size-fits-all perspective, these types of solutions tend to be more robust and predictable. For large, scale data migration projects such as data center migrations, these techniques are highly recommended. They do suffer from the typical "vendor lock in" syndrome. However, more vendors are beginning to offer cross-platform replication solutions that allow you to go from one vendor array to another one using the same piece of software.

Data migration does not have to be as strenuous or painful as moving a mountain, but it can be. To ensure that it does not, you have to plan it well. A well-organized IT shop will take data migration in its stride and plan it using the tight methodology and processes they apply to plan all other projects. Planning and using the proper set of methodologies is key to a successful migration.


For more information on related topics visit the following related portals...
Data Integration and Data Management.

Ashish Nadkarni is a principal consultant at Glasshouse Technologies Inc. and specializes in data migration projects. Glasshouse Technologies Inc. is a vendor independent storage services company and provides a wide array of storage services to Fortune 1000 clients. He may be reached at ashish.nadkarni@glasshouse.com.

Solutions Marketplace
Provided by IndustryBrains

Backup SQL Server or Exchange Continuously
FREE WHITE PAPER. Recover SQL Server, Exchange or NTFS data within minutes with TimeSpring?s continuous data protection (CDP) software. No protection gaps, no scheduling requirements, no backup related slowdowns and no backup windows to manage.

Database Access Tool
Reverse engineers relational schemas and automatically generates data access code that can be used in Service Oriented Architectures.

Validate Data at Entry. Free Trial of Web Tools
Protect against fraud, waste and excess marketing costs by cleaning your customer database of inaccurate, incomplete or undeliverable addresses. Add on phone check, name parsing and geo-coding as needed. FREE trial of Data Quality dev tools here.

DeZign for Databases - Database Design Made Easy
Create, design & reverse engineer databases with DeZign for Databases, a database design tool for developers and DBA's with support for Oracle, MySQL, MS SQL, MS Access, DB2, PostgreSQL, InterBase, Firebird, NexusDB, dBase and Pervasive.

Free EII Buyer's Guide
Understand EII - Trends. Tech. Apps. Calculate ROI. Download Now.

Click here to advertise in this space

E-mail This Article E-Mail This Article
Printer Friendly Version Printer-Friendly Version
Related Content Related Content
Request Reprints Request Reprints
Site Map Terms of Use Privacy Policy
SourceMedia (c) 2007 DM Review and SourceMedia, Inc. All rights reserved.
SourceMedia is an Investcorp company.
Use, duplication, or sale of this service, or data contained herein, is strictly prohibited.