Data management has been a hot topic in the last years, topping even cloud computing. Here is a look at some of the trends and how they are going to impact data management professionals.
The Rise of ‘Datafication’
Today, businesses are ending up with more and more critical dependency on their data infrastructure. Before widespread electrification was implemented, most businesses were able to operate well without electricity but in a matter of a couple of decades, dependency on electricity became so strong and so broad that almost no business could continue to operate without electricity. Similarly, “datafication” is what’s happening right now. If underlying database systems are not available, manufacturing floors cannot operate, stock exchanges cannot trade, retail stores cannot sell, banks cannot serve customers, mobile phone users cannot place calls, stadiums cannot host sports games, gyms cannot verify their subscribers’ identity. The list keeps growing as more and more companies rely on data to run their core business.
Consolidation and Private Database Clouds
Database consolidation has been lagging behind application server consolidation. The latter has long moved to virtual platforms while the database posed unique challenges with host-based virtualization. However, with server virtualization improvements and database software innovations such as Oracle’s Multitenant, database consolidation moved to the next level and most recently reemerged as database as a service with SLA management, resource accounting and chargeback, self-service capabilities, and elastic capacity.
Commodity Hardware and Software
Hardware performance has been rising consistently for decades with Moore’s Law, high-speed networking, solid-state storage, and the abundance of memory. On the other hand, the cost of hardware has been consistently decreasing to the point where we now call it a commodity resource. Public cloud infrastructure as a service (IaaS) has dropped the last barriers of adoption.
On the software side, open source phenomena resulted in the availability of free or inexpensive database software that, combined with access to affordable hardware, allows practcally any company to build its own data management systems—no barriers for datafication.
The Future of Database Outsourcing
Datafication, consolidation, virtualization, Moore’s Law, engineered systems, cloud computing, big data, and software innovations will all result in more eggs (business applications) ending up in one basket (a single data management system). Consequently, the impact of an incident on such a system is significantly higher, affecting larger numbers of more critical business applications and functions—for example, a major U.S. retailer that has $1 billion of annual revenue dependent on a single engineered system or another single engineered system handling 2% of Japan’s retail transactions.
Operating such critical data systems becomes much more skills-intensive rather than labor-intensive, and, as companies follow the trend of moving from a zillion low importance systems to just a few highly critical systems, outsourcing vendors will have to adapt. The modern database outsourcing industry is broken because it’s designed to source an army of cheap but mediocre workers. The future of database outsourcing is with the vendors focused on enabling their clients to build an A-team to manage the critical data systems of today and tomorrow.
Breaking Enterprise IT Silos
The age of big data spells the end of enterprise IT silos. Big data projects are very difficult to tackle by orchestrating a number of very specialized teams such as storage administrators, system engineers, network specialists, DBAs, application developers, data analysts, etc.
It’s difficult to specialize due to the quickly changing scope of roles as well as rapid evolution of the software. Getting things done in a siloed environment takes a very long time—this is misaligned with the need to be more agile and adaptable to changing requirements and timelines. A single, well-jelled big data team is able to get work done quickly and in a more optimal way—big data systems are basically new commercial supercomputers in the age of datafication and—just like with traditional supercomputers—they require a team of professionals responsible for the management of the complete system end-to-end.