How Robotic Process Automation Fits Data Migration

UPDATE: There is an updated version of this article available. Read "How RPA Transforms Data Migration" (published in 2019).

For the purpose of this article data migration is defined as the movement of data from an older legacy system to its new replacement system.  Typically, it is part of a larger business activity or decision, such as:

• Merger or acquisition in which overlapping or legacy IT systems or datasets are consolidated for cost savings and operational efficiencies.

• Enterprise standardization; again, a tactical decision to increase efficiencies by making employee skillsets more application across IT systems and user support less complex.

• Modernization which leads to strategic decisions to sunset older systems and gain business efficiencies from current technology.

Regardless of the reasons behind a data migration, it requires precision in planning, implementation and final transition to be successful. Risk is always significant and can be appreciated by all stakeholder by one key fact – when it ends the existing legacy system will be turned off and its screens will go dark. Given the inherent accuracy and consistency of process automation, it’s no surprise the use of robotic process automation (RPA) technology as a migration tool is growing.

RPA’s Role in Data Migration

Robotic automation is an excellent fit for this work because both the techn

ology and migration share a similar profile of highly structured, rules-based, activities.  For example, a typical methodology for migration has this high level extract, transform, load (ETL) approach:

• Extract Design: there must be a well-defined approach for the manner in which data will be extracted, held and verified.

• Transform: a migration design must guide how data is to be transformed for the targeted to-be data structure.

• Load: the details of how extracted, transformed, data is mapped into the target structure must be defined.

• Test and Recovery: unit and integration test procedures, reporting and exit criteria must be planned – along with roll-back and recovery procedures for every stage of the migration.

Because this methodology clearly spells out procedures, rules, sequential activities, and required outcomes for every step – from as-is current state to the to-be current state – there’s a very close fit with the automation requirements of RPA technology. Further, RPA’s loose integration protocols enable it to extract and load data from disparate sources quickly and without disruption.

A Use Case Example

A use case example from the insurance industry will illustrate the significant contribution robotic process automation can make to a migration project.

The need to migrate data in the insurance industry is commonly tied to the acquisition of a “book of business” – or block of policies – by one company from another.  It’s not unusual for these books to have been written years ago and administered on old legacy systems supplemented by spreadsheets. Acquired books of business are typically at least several hundred thousand policies and the practical difficulties of extracting and cleansing data of that magnitude from legacy systems and spreadsheets is not hard to imagine.  

Acquired policies are “live” and must be administered, so time spent in migration is of the essence. Absent automation, the acquiring company generally deploys a migration team of several small groups; consisting of a lead and analysts from the business unit which will service the acquired policies. These groups work through extraction, quality review and cleansing and may be relieved by new analysts for transformation, loading and testing with IT staff.  The data migration for even a small book of business can take between 10-12 weeks and cost the acquiring company well over $350,000.

 With proper planning and execution, robotic process automation can change both timelines and costs. The first step is designing and automating a rules-driven migration process for all three steps: extracting; transforming and loading. To be effective, this workflow must clearly define rule-driven steps appropriate for automation and logical exception points where human intervention is necessary. 

The second step is a reassignment of roles and responsibilities which leverages the different strengths of robotic automation and the business analysts. The third step is integrating the automated processes with unit and integration test scripts.

Because robots are as invisible as any other software, automating migration will likely go unnoticed by casual observers. The analyst groups will still be working – but in a different role. With error-free automation taking over the role of manual data movement, analysts will now be focused on documenting, archiving and resolving errors in data properties and activity outcomes.  

With effective process modeling, automation design, and training, the use of robotic technology in this example can be expected to cut the migration time by 50% and costs by 40%. Further benefits are complete accuracy in the extraction work and log files on all transactions for legal, tax and regulatory compliance.

RPA’s Value for Data Migration

As the use case example makes clear, the use of robotic process automation for migrating data provides compelling cost and performance benefits.  Significantly, the nature of this work is such that the use case example is the rule rather than the exception.  Whether migrating data is triggered from an acquisition, standardization, or modernization, it must always extract information from a system chosen for retirement because of limitations.  Therefore, moving data from such system often comes down to a choice between manual efforts or robotic software’s highly adaptive automation capabilities – and the technology's compelling benefits makes the choice an easy one to make.