Data mesh: solving the business plea for data access

*Editor's note: This article was originally published on December 14, 2022, on TSYS.com.

Every day, 2.5 quintillion (250 trillion) bytes of data are created. While that's a staggering amount, it's likely to more than double by 2026! As the “dataverse" grows, many companies recognize the need for data intelligence, and it's leading them to take a data-first approach. They understand the increasing demand on data accessibility to drive growth, and with it, the complexity of managing that data.

Unfortunately, organizations struggle with enabling employees to find and consume the right data to make informed and timely decisions. In fact, more than 80% of data that companies create is not used because of challenges with latency and costs of moving data across environments.

Data mesh: 20 years in the making

Simply put, data mesh is a new strategic approach to data management. It brings order to an organization's data, helping deliver that data more efficiently to business leaders so they can make better decisions. In more technical terms, data mesh is a set of principles for designing a modern data architecture.

To better understand data mesh and help you decide whether it's a viable approach for your business, let's look back in time. Roughly 20 years ago, companies used Enterprise Data Warehouses (central repositories) to manage data as an asset. The idea was to create a single location for data to be stored that could be used to drive business decision-making. This required time and money up front. Plus, the warehouses could not meet storage or management needs and were unable to support AI or machine learning.

Fast-forward 10 years and we reach the Big Data era. The main concept was to quickly integrate large datasets into a company's data lake or data warehouse to be analyzed. The data lake developed from the need for more storage and better management of data. While data lakes removed the upfront costs and time-consuming development work to store and access the data, they were complex and time-consuming to maintain.

Today, it's common for companies to have multiple disparate data silos. This also presents challenges, however, when leveraging data as an enterprise asset including slow data-driven decision-making and poor collaboration across teams. Data mesh is a way companies could address this issue.

Why is data mesh important?

Think of data mesh as a kind of central nervous system for your organization's data. Just like your central nervous system is the body's processing center, data mesh for your company is like a network to exchange data about the business.

Data users can easily access and query data where it lives without first moving it to a data lake or data warehouse. Business teams are responsible for their data and resulting data products. As such, data mesh has a heavy emphasis on self service, allowing business teams to efficiently manage and utilize the data.

“Data mesh has solved the business plea for data access, something that organizations struggled with for many years," said Kenneth Viciana, vice president of Global Data & Analytics Products at TSYS, a Global Payments company. “The decentralized strategy delivers data to the right people in the organization in real time, reducing costs and providing faster analysis. The result is an increase in the validity of data-driven insights that allows businesses to make better data-based decisions."

A central nervous system for data

Putting data into the right hands

With data mesh, data is accessible to business users, which decentralizes the data and allows for a self-service infrastructure. This approach avoids potential bottlenecks when trying to integrate data into a data warehouse or data lake. Teams can now generate, collect and distribute data as needed in what amounts to a “data ownership" model.

No enterprise data team has ownership of the organization's data assets. Instead, these assignments land in business domains. For instance, customer relationship management data is often managed in Salesforce by an operations team, and the data mesh would assign operations as the business domain owner of this data set.

Opportunities also present challenges

As you have learned, data is being delivered more efficiently to the business and its users with data mesh, but does it provide the right insights? Despite the need for business access to data being met, data mesh comes with challenges. These challenges stand in the way of consistent, accurate insights that could lead to improper business decisions.

For one, there is a need for creating consistency across the business in how teams manage the data. To handle this situation, organizations generally have an Enterprise Data Governance Team that provides business units with frameworks and standards that promote alignment.

Secondly, data mesh requires a strong federated data governance approach. Cross-domain analytics and the potential to duplicate data across different domains could affect resource utilization and data management.

While data mesh may be the first step in solving the business data access need, there is more to solve to ensure the delivery of the right insights to make smart business decisions.

Learn more about our issuer solutions at TSYS.