How Data Warehousing And Governance Affect Connectivity

 A recent survey conducted by TDWI highlights the benefits of cloud data warehouses (CDW) for stronger performance and greater flexibility than traditional on-premise databases. The responses of around 200 IT managers identified “data governance” as the top challenge (50 per cent), closely followed by “integrating data across multiple sources” at over 40 per cent, and “getting data into the warehouse” at about 38 per cent.

The scope of the study was very much around cloud data integration, but the results were also telling from a network infrastructure perspective. For example, we know that while the cloud is being rapidly adopted for data warehousing, the private data centre is still very much part of the enterprise IT picture. This means businesses need to connect to both public and private cloud assets, one of the reasons why “integrating data across multiple sources” was the second greatest consideration. However, the main challenge, around data governance, also plays into this same issue.

Data governance is a global issue

In Europe we have the implementation of GDPR, while the US is demanding stronger data governance through state laws with similar intent, such as the California Consumer Privacy Act. What this means is that organisations are increasingly emphasising data privacy protections when they put big data applications into production use. In some cases it also affects the physical and geographic location that data can be stored. All of this has implications in terms of connectivity to the various sites where data is ingested, stored, manipulated and accessed.

But not all sites are equal, nor are all access requirements. Some sites will have multiple different connectivity requirements to support different applications and use cases. Consider a typical branch office that generates traffic from a wide variety of applications. Some traffic will be non-critical and non-sensitive and so can travel over the public internet, but mission critical connectivity to cloud-based applications or the data centre will need to travel over the more resilient MPLS network or other dedicated connection. When it comes to policy-based forwarding of traffic, the network needs to be designed so that that it can make forwarding decisions based on application type.

Different sites have different connectivity requirements

So, most enterprises will operate networks that use both public and private connections, and application traffic will drive these connectivity choices. For instance, it might be that businesses prefer to forward traffic from secure enterprise web applications (HTTPS) over the internet, but they prefer to support big data applications, storage and replication traffic, and enterprise resource planning (ERP) applications hosted in a public or private cloud with MPLS.