You might think the title of this article is somewhat controversial, but you should wait to render judgment until you’ve read to the end. There are several important shifts impacting data management and database administration that cause manual practices and procedures to be ineffective. Let’s examine several of these trends.
Data Growth but No DBA Growth
Data management continues to rise in importance as organizations amass and use more data every year. Along with storing more data, organizations are implementing and using a greater variety of data platforms and database management systems than ever before. At the same time, organizations are not adding more DBAs and data management professionals to their IT staff.
A recent update to the “IDC Global DataSphere” study projects that data growth will continue unabated through at least 2025, growing at a compound annual growth rate (CAGR) of 61%. At the same time, IDC says the DBMS market is expanding. The relational/ SQL database systems market is growing at a CAGR of more than 8% and the NoSQL market segment is expected to grow at a 5-year CAGR of nearly 31%.
You might think that as data volume grows—and the number of database systems we deploy also expands—that organizations would hire and train more DBAs and data management professionals to oversee the data. That is not the case.
In its “2020 Data Management Staffing Ratios” report, industry research firm Computer Economics found that data management staff represents about 5% of the typical IT staff. Although this is a growing amount—up a full percentage point from 2019—it still pales in comparison to the growth in the amount of data over the same time period.
These industry trends reveal an ever-growing, complex data environment that DBAs are expected to manage. This includes tasks as diverse as performance monitoring, SQL and database optimization and tuning, backup and recovery, change management, and handling data movement (both on-prem and to the cloud). Success increasingly requires intelligent automation of DBA tasks and procedures.
Speedier Dev: Agile and DevOps
Another important trend impacting DBAs is the increasing speed of application development, primarily due to the adoption of agile development techniques and DevOps. DevOps changes the application delivery lifecycle to a continuous loop, with continuous development, integration, and delivery of code. This continuous activity makes DBA requirements such as database design, change management, SQL tuning and quality, and data movement continuous, too.
As DevOps automates the process of software delivery and infrastructure changes, the objective is to deliver any change anywhere throughout the organization into production quickly, safely, predictably, and in a controlled manner. This means that software improvements are delivered in smaller chunks more frequently, instead of waiting until the end of the project. When implemented correctly, with everyone bought in and participating in the process, the result is that development moves faster than ever before.
But for DevOps to work for both development and operations (where the DBA is part of Ops) the entire development pipeline must be automated, not just for application development and delivery but also for operational implementation and support. A robust set of application development tools for orchestration and automation exists, but such a comprehensive set of tools for operational database administration must also be deployed and integrated into DevOps practices and procedures.
More Platforms: Heterogeneity and Cloud
For a long time, the DBMS space could be narrowed down to just a few leading vendors that dominated the market. Even today, the DBMS market is still dominated by SQL and relational, although the database landscape is more heterogeneous than ever before.
You can gauge just how heterogeneous things are by looking at the DB-Engines (www.db-engines.com) site, which tracks the popularity of more than 350 different database systems. As a result, DBAs must know traditional SQL database systems but also NoSQL databases, which behave and are managed quite differently. Automation is required to enable managing such a diverse set of database systems.
Then we must consider cloud adoption. According to a study from 451 Research, almost two-thirds of organizations are currently pursuing a hybrid IT strategy, with 65% adopting a hybrid cloud approach that leverages both on- and off-prem resources.
This hybrid future impacts DBA processes and procedures. DBAs still must know how to manage on-premise databases, but they also must understand the various ways that cloud database implementation impacts data management and availability.
The Bottom Line
DBAs are being asked to do more. There are larger amounts, and more types, of data that need to be accessed more rapidly and from more sources. Everything must be done without any prolonged downtime, while using and supporting new database types and capabilities, and with fewer DBAs as a percentage of IT staff than ever before.
If you take this all into consideration, I think the title of this column makes absolute sense. Don’t you?