Data migration & transformation

Do you have data held in a legacy system you’d like to move to a new application or consolidate from multiple different sources? We can assist with your migration, maintaining full data integrity.
Data Migration/Transformation

Migration planning

When moving between platforms or providers, it can be difficult to ensure your business data is correctly migrated. For example, a website built on a legacy content management system (CMS) may have hundreds or thousands of existing pages and assets, a bespoke customer relationship management (CRM) system or other internal data on thousands of customers – all of which might otherwise need to be manually moved. 

OWA can assist in creating a migration process, modelling the data formats used by both systems and the data transformations required, plus scripting the migration to automatically transfer all data between the systems.

Source & destination

At the outset, we review the source and destination systems and the data being migrated to confirm the best approach. Is the data held in a common database format such as MySQL, SQL Server, Oracle or MongoDB or a file format like CSV, XML or JSON? Does the source system have an API to extract the data or does it need to be taken directly from the database or via the existing user interface? Are there related assets, files or associated metadata that also need extracting?

Once data can be extracted from the source, it can be compared with the destination database, file format or API. A number of the migrations we undertake are to Software as a Service (SAAS) providers or cloud solutions, such as Amazon Web Service's S3 or RDS products, which have specific APIs and require data to be added in certain formats. OWA will review how the source data can be mapped to the destination and discuss any issues which arise and how best to work around these. For example, is the data field required by the destination missing from the source or does the source data exceed length requirements or other data constraints? Some issues can be resolved by automated data transformation, such as the addition of certain missing fields or truncating existing data to fit whereas other situations may require some manual processing. OWA will raise any issues with the data transformation process and discuss potential resolutions prior to the migration.

Our data migration process is customised to each application
OWA team reviewing a data migration project
Client reviewing the data migration plan designed by OWA

Quality assurance

After reviewing the source, destination and data mappings, OWA will create a migration process to transfer the data to the destination system. This is initially completed in a test environment so no existing data is changed and we can verify all the correct data has been imported or migrated to the correct fields in the destination system.

Once this process is complete, the test environment will then be released for quality assurance, allowing you to perform any further checks which are required. Should any issues arise, the migration plan is updated and re-run to ensure these are corrected.

Final migration

When all stakeholders are satisfied with the test migration, this can be applied to the live source and destination systems, migrating the data into the new application. We will advise on any necessary restrictions on the source and destination systems, and any time they will be unavailable while the migration is carried out. 

Depending on the nature of the migration, it may be possible to complete this process with minimal downtime or other impact on users of the system. We can also work with more complex requirements, for example where data needs to be synchronised between multiple systems over a period of time, fed back into the source system from the destination or where data needs to be migrated to or from multiple systems.

OWA team implementing new data migration tasks
OWA can assist in creating a data migration process
OWA creating a data migration process

I would recommend OWA for their friendly, dedicated service.

Amy Summers, communication and economic development manager, Hart District Council
Find out more about our data services

Find out more about our data services

Data migration can transform the utility of your applications and workflow.