Salesforce is the leading cloud-based CRM software solution that has grown to become the platform of choice for various IT organizations. I have been working with Salesforce for over a decade now, and in all of my Salesforce adventures, I've never contributed to a project that did not involve some form of data migration.
Now, say you have a medium to large organization, then the data migration becomes a quarterly thing that you just cannot get away from. In the mergers and acquisition world of today, it seems like technology teams are thrown a new system to integrate with and migrate from quite often. So, the more you master your migration and integration strategies, the better off you'll be. I wanted to look at how we could improve the quality of data migration and how it could have a positive impact on the overall lifecycle of a project.
Think Repetition
Focus must be given to make the process repetitive. There is little to be gained in just being able to somehow create a set of CSVs to get the data imported into Salesforce one time. Think about what you can do to make the process repetitive. It would be nice to be able to push a button and get the data migration into one sandbox on an as-needed basis and also have a way to repeat the process multiple times to prepare before the project goes live. How can you achieve this? The key is having the ability to delete the data out of the system so the process can be repeated. Structure your migration to allow you to load, test, delete, reload and retest the process. The better you get at this, the better your overall confidence in the live process will be, and you will have the ability to redo anything if business demands force you to change the initial migration.
Think Adaptability
Adaptability of your migration framework/code/platform is essential. Like any code -- whether it be in Apex, Java or something else -- the idea is to refactor and reuse as much as you can. The same principles apply to data migration processes. Manage the process just like you manage code. Version it, get it under source control, apply change management processes and make it configurable as much as possible. Strive for automation and push-button setup to eliminate manual misses. I would say this applies universally to all data migration strategies and not exclusively to Salesforce. When I plan my migration into Salesforce, I typically use the highly flexible data migration platforms supported by Salesforce. I love the flexibility that the command line data loader from Salesforce provides. But the same can be achieved with any migration/integration tool of your choice.
Think Configurability
Strive to make the code as simplistic as possible. Avoid having hard-coded data that would require a code change when a minor change is desired (like a new attribute needs to be migrated from a source or an additional transformation for an attribute is required). Make data do the data mapping and not your code. This allows you to focus on making the process more and more generic, and code also becomes easier to maintain. Whenever I guide my teams on development I like to emphasize on writing code from a product metaphor. When you have too many business rules hidden in the middleware, then you are pretty much reinventing the wheel every time a new project comes along.
Think Reporting
Do developers and architects really worry about reports? I have been bitten by missed reportable data points too many times after data migration efforts that required some heavy lifting. The lesson I learned is to not ignore the current data reporting needs of a business and have a good plan in place to not disrupt reporting. Keep the business stakeholders happy, strive to understand the business reporting patterns and ensure that you are not adversely impacting the data points that are needed for the reports to function properly. A simple example would be to not bring in legacy data with a created import date. It is extremely critical to maintain the source data points to avoid causing major reporting nightmares for everyone using your data. Migration needs to be seamless to the user as if the data were loaded originally into Salesforce. Data is king, and I always think data first on all my integration and migration planning. The better the data, the better it will be for everyone involved.
Think Integration
When we talk about migration and integration, the two are rarely connected. People say we want to migrate this data from this data source, so they create a migration team to do the migration. They also form a separate integration team for integration projects. Most of the same rules that apply to integration are also applicable for data migration, yet people make a mistake when they look at migration as a one-time activity and keep a very small team focused on migration. Migration needs to be given the same importance as integration. In fact, migration is the best test of your integration. I think that the business rules that govern data need to be applied at the organizational level and should not be dependent on what tool is being used to perform the activities.
So the next time you plan your migration, think repetition, adaptability, configurability, reporting and alignment with integration. I like to define integration as “continuous migration.” Tackle your migration strategies with an eye on continuous data integration.
