Nikki Pantland, Data Analyst at PBT Group.
Today's enterprise environments are increasingly moving toward cloud-based data platforms and data lakes. This migration comes with several benefits and drawbacks, both for the organization performing the migration and for the data teams in the field who must perform the migration.
One of the benefits of moving processes to a cloud-based platform or data lake is the scalability it provides.
It's also a way to bring all the data stored within your organization into one place for everyone to access, leveraging processes across different teams to ensure the same approach and sources are used. By doing so, you have the potential to create a single version of the truth across your organization.
Pushing processes to the cloud that have not passed at least one governance gate can lead to unnecessary and costly increases in cloud computing.
Another benefit is that you avoid unnecessary duplication of work and data between teams within your organization. Ultimately, duplication is stopped, reducing data storage requirements.
The nature of these cloud platforms increases the amount of governance required to ensure that the processes and data sources housed on the platforms are functional, relevant, and well-managed. Masu.
Pushing processes to the cloud that have not passed at least one governance gate can lead to unnecessary and costly increases in cloud computing.
Migrating on-premises data systems to the cloud is a large-scale undertaking that impacts everyone in your organization with a stake in the data. There are many tips to help organizations and field teams streamline processes and reduce frustration and roadblocks.
Communication and governance: The main tip for organizations is around providing information and guidance. At the group level, there needs to be clear direction and communication around the migration timeline and what the changes mean for the data workers in your organization.
All governance changes must also be delivered in a central location and in a simple, easy-to-understand format. This information should clearly outline the governance gates that need to be passed through in the process, the specific individuals responsible for each gate, and what is expected from the team at each step.
If your data team needs additional help or input, for example from an architect, to highlight how processes and data fit into the big picture, contact details for a dedicated person who can help with this. must be provided.
Documentation and accessibility: Changes in the tools available and required coding practices for new platforms should be clearly documented for the data team and made visible within the organization.
For data teams, the process of accessing data in the new environment must also be streamlined and simplified. Ideally, organizations should offer a similar access management approach. If individuals or teams have access to on-premises data or processes, it should be automatically mirrored to the new platform.
The way you connect to a new platform should also be as accessible and simple as possible to encourage use and exploration of the new platform throughout the process. If connecting to a new platform is too cumbersome, teams will cling to on-premises sources that are easier to access and use.
There is a little more work to be done for data teams who need to migrate their internal procedures and data construction to new data structures.
Planning and strategy: First, take inventory. What the portfolio of work is, how many processes there are, what sources are used, the history created, processes that may need to be considered as data exceptions and pushed directly to the platform How do they work together? The data sources you need are already available on the new platform.
Organize your data. As part of this, we do a housecleaning and ensure we only retain processes and builds that are still relevant and in use. Decommission any processes that are no longer contributing or that are obsolete. Archive data artifacts and functionality that will not be carried forward.
Perform the migration in stages. Be methodical when converting your code to the new standard and migrate one process at a time. Use the new governance structure to your advantage to ensure your processes are robust and optimized, and assess whether there is anything already on the platform that you can leverage to improve or extend your processes as part of your migration. Masu.
Moving to a cloud-based platform creates discomfort and confusion within your organization. By properly managing information delivery and ensuring a strong guidance base is in place, organizations and data teams can better navigate change.
The more support you provide and the more streamlined your access controls are, the less negative the impact will be on your users.
Ultimately, moving to a cloud-based data platform presents exciting opportunities for organizations and their data teams. However, managing the transition is the most important aspect of the transition because it defines the attitude toward the new structure within the organization.
