Organizations have directed a lot of attention recently to consolidation, automation, and cloud efforts in their data management environments. This will purportedly result in decreased demand for data managers and the need for fewer DBAs per groups of databases. However, the opposite seems to be occurring. In actuality, there is a growing need for more talent, as well as expertise to manage through growing complexity. A new survey among data executives, managers, and professionals finds that a more challenging data environment is arising due to a confluence of factors.
The research, conducted by Unisphere Research, a division of Information Today, Inc., and sponsored by Idera, included the responses of more than 300 DBTA readers who represent a range of industries and company sizes.
Half of the data shops covered in the survey have grown in size over the course of the last 5 years—some dramatically. Close to one in four respondents reports growth exceeding 25% of their organization’s original staff size. By contrast, only 10% of respondents say their staffs have shrunk in size.
How has the size of your data management team changed over the past 5 years
Has grown more than 50% - 11% of respondents
Has grown 26%-50% - 12% of respondents
Has grown 11%-25% - 15% of respondents
Has grown up to 10% - 12% of respondents
Has not changed in size - 31% of respondents
Has shrunk - 10% of respondents
Don’t know/unsure - 6% of respondents
What’s driving the continuing growth in database staffing? For the most part, companies have been expanding—adding more lines, more services, and increasing transaction volumes. Sixty-one percent of sites experiencing staff growth say the growing volume of business necessitates adding more data managers to their teams. The growth of data itself—exacerbated by big data—is also a contributing factor, cited by half of this group. The rise of new data frameworks, such as Hadoop or data warehouse expansions, is yet another driver among 44% of sites.
If your data management team has expanded over the past 5 years, what has driven this growth?
Business growth—more customers, more transactions, more products/services - 61% of respondents
Data growth - 50% of respondents
Expanded data functions/data environment (such as adding Hadoop, data warehouse) - 44% of respondents
Need for greater security or compliance management - 25% of respondents
Movement to cloud databases - 23% of respondents
Don’t know/unsure - 14% of respondents
Other - 6% of respondents
In addition, the vast majority, 89%, agree that the complexity of their database environments has increased over the past 5 years. Close to half, 46%, state that their database environments have grown “significantly” or “extremely” more complex during this time. As with the challenges that caused the ongoing growth in database staff sizes, both business growth and data growth are also adding complexity to data environments.
The move to cloud computing, at least for mission-critical enterprise data, will be a slow one. Only 19% of data managers indicate that they intend to move a significant portion of their enterprise data (defined as more than 25% of their total data stores) to a public cloud, while 26% intend to move a significant portion of their data to private or hybrid cloud arrangements.
Database lifecycle management methodologies—which involve coordinated processes, tools, and people—are emerging to address growing complexity within data environments.
Are any proactive measures being taken to address this complexity? Virtualization and automation are the top options being adopted by data managers to provide some much-needed simplicity to increasingly heterogeneous environments. The use of management and configuration tools is seen as a way forward for 38% of respondents. About one-third of respondents report they are adopting database lifecycle management (DLM) methodologies to address growing complexity within their data environments. (DLM involves coordinated processes, tools, and people to optimize all aspects of the lifecycle of data, including data architecture and modeling, database design, monitoring, administration, security, storage, and archiving.)
Data managers report a range of tangible business advantages that their organizations are gaining as a result of their DLM efforts. More uptime of data systems is the leading benefit being realized. A majority, 57%, say they have experienced reduced system downtime as a direct result of their DLM engagements. Another 55% of respondents report that their efforts have made data more available to their end users. Confidence in the data itself is also up at 38% of sites.
Yet, data managers have also encountered obstacles in their efforts to implement DLM. Half, for example, report that their efforts have been stymied by the need for greater funding or staff time to pursue DLM. At least one-third of respondents to the survey also indicate that their DLM programs do not have as high a priority at other similar initiatives, such as application lifecycle management. A similar percentage of respondents see other impediments, such as a lack of visibility to the issues that may be affecting database performance.