10 Tips for Flawless Data and Application Migration
IT modernization and digital transformation usually involve incremental changes in existing systems. Occasionally, though, a rip-and-replace strategy may be necessary, one that requires migration of data from an old software system to a new one. Switching enterprise resource planning (ERP), CRM, or other central applications will involve such data migration between systems, which is never a simple task; the complexity of data-migration jobs can result in unexpected costs and add significantly to time-to-value. Mistakes made during migrations can lead to costly downtime, diminished end-user productivity, and the loss of important data.
With proper planning and a thorough approach, you can avoid some of the most common data and application migration project mistakes. Here are 10 tips to keep your data migration project on track:
1. Assess the applications and data. Be certain to assess application functionality and analyze the app and its data from a business process standpoint. Consider the data types, transmission protocols supported, external dependencies, and input and output formats supported.
2. Clean up your data. Don't migrate your bad habits. Things may have started out perfect, but it's more than likely that data inconsistencies and data duplication were introduced over time with multiple users. So it's important to sanitize the data before the moving the data over, by de-duplicating data and fixing any data integrity problems.
3. Validate and redefine business rules. Your company's business and validation rules may have changed since the legacy system was first built. Now is the time to make sure that the businesses rules are current and any questions or disputed processes are ironed out. Determine if any data fields need to be changed or added and ensure the necessary validations are included in the new database.
4. Check cross-object dependencies. Cross-object dependencies often are not discovered until very late in the migration process. A complex project can have dozens of different data fields coming in from different applications. Discovering new sources of data or connectivity late in the process can throw off your migration timeline. It's better to identify external sources of data up front and make sure that the necessary connections are in place for the new system.
5. Leverage automation. Many projects suffer from costs overruns due to human error and system downtime. While traditional extract, transform, and load (ETL) tools may seem appealing at the outset, they won’t allow you to maximize uptime and maintain interconnected business processes, such as the automatic update of customer address data in your ERP system based on changes recorded in your CRM system. Instead, look for an integration platform that supports process-based integration and automation of stored procedures. In addition, an integration platform that has prebuilt connectors to the systems you're migrating can save a lot of time and help you avoid errors. It can also let you keep both new and old systems live and updated, without risk of downtime or transaction loss, until you are ready to do the full switch. System integration platforms can provide an effective solution that will not only accelerate your data migration project but can be useful for other system integration projects in the future, providing a high return on investment.
6. Rely on relevant expertise. Often there is a gross underestimation of just how complex and challenging data migrations can become. It is often perceived as a grunt exercise, tacked on to the end of the much-higher-visibility target implementation. But each data migration project requires specialists that can find the root cause of data discrepancies and then fix them, so it's essential that team members have the necessary professional qualifications and familiarity with the business data.
7. Don't wait for the target to be ready. Get started early. There is so much work to do on the data quality and business rules, so there is no reason to wait for the target to be ready or even chosen. You may have a lot of data to clean up front; it may be too late to hit the deadline if you wait too long.
8. Transform data systematically. Use business process automation tools and data mappers to transform data systematically. A well-designed data mapper will automatically handle differing database types, flat files, XML, flow variables, user-defined fields, operational data stores, constants, and string, arithmetic, and logical expressions.
9. Validate and test the data-migration process. Don't save this step for the end—you really need to make sure that you're validating and testing throughout the process. Having business users validate that the data is complete and accurate is important; the people who sign off on the data should be the ultimate consumers of the data.
10. Provide post-migration support. Following the migration and deployment of the new applications, fine-tuning, optimization, and other refinements may be needed. Be prepared to handle post-deployment issues quickly and with adequate resources. Failure to reserve the talent needed to handle post-deployment issues can result in problems with the adoption and use of the newly migrated application and its data.
Considering these factors before you start your data migration project will help increase the chances of success. Regardless of the complexity of your migration, make sure you have a robust plan that outlines the dependencies between various tasks and milestones before you begin any work. Remember, having a plan to work and working to plan can minimize the risk of a data migration disaster.
Glenn Johnson is a senior vice president at Magic Software Americas. Johnson is the author of the award-winning blog "Integrate My JDE" on ittoolbox.com and contributor to TheBusiness Week Guide to Multimedia Presentations (Osborne-McGraw Hill). He has presented at Collaborate, Interop, COMMON, CIO Logistics Forum, and dozens of other user groups and conferences. His interviews on software industry issues have been aired on NBC’s Today show, E! News, Discovery, and more.