Building a successful, effective data architecture, regardless of industry or company size, is easier said than done. Becoming data- and analytics-driven requires a specialized, unique plan that fits an enterprise just right; a one-size-fits-all modern data architecture trend cannot comprehensively and adequately support a company and its data.
John O’Brien, principal advisor and industry analyst in data strategy and architecture at Radiant Advisors, led Data Summit’s pre-conference workshop, “Build The Modern Data Architecture & Plan That Transforms Your Company,” to demystify the steps toward building a data architecture that fits an enterprise’s unique needs while addressing the challenges that might arise.
The annual Data Summit conference returned to Boston, May 10-11, 2023, with pre-conference workshops on May 9.
O’Brien emphasized business strategy alignment, which will ultimately optimize the modern analytics lifecycle for enterprise scalability. He broke it down into four steps:
- Identify the business outcomes to achieve
- Translate to data and analytics capabilities
- Prioritize the cloud roadmap for analytics
- Implement technologies for optimal ecosystem in the cloud
Further, data-driven companies represent a variety of principles that mark its success. O’Brien explained that by adopting intuitive tools, empowering people through self-service, optimizing the analytics lifecycle, and setting analytics as a strategic priority, any organization can amplify decision making with relevant data. These principles enable enterprises to overcome uncertainty, volatility, and disruption to harbor resilient, empowered, distributed, and scalable data architectures.
O’Brien then transitioned into the importance of an enterprise analytics framework that ultimately supports how people work with data. To support people working with data, a company must understand how people work with data; O’Brien outlined the steps of a modern analytics lifecycle:
- State business need to understand, explore, or make a decision with data
- Discover and ascertain what data is available by context and upload new data
- Connect, explore, and profile data for applicability to business problem
- Prep, enrich, integrate, and transform data for analytics
- Analyze data and build models (statistical, predictive, spatial)
- Visualize data and analytics to validate and govern outputs
- Deploy and operationalize monitoring of new data and analytics assets
In terms of data architectures, O’Brien has encountered several questions raised surrounding business capabilities within enterprise analytics frameworks. Questions include:
- “What data do we have?”
- “How do I get data access?”
- “Am I using the data correctly?”
- “I can do it quicker myself.”
- “This data available is too old…”
These questions concern a few areas that should be front-of-mind while building and planning a modern data architecture: data simplification, access simplification, data governance and security, business self-service data, and the value of near real-time data.
O’Brien explained that building a roadmap with business impact is critical in cultivating a successful data architecture. He further argued that starting with pain relief—or areas that need improvements because they cause significant issues—will guide an effective roadmap to success.
Starting small, O’Brien emphasized that focusing on how to deliver business value, and particularly, on on-prem applications that may benefit from a cloud platform, from within a small scope can aid in determining where an architecture can be transformed into a more efficient one. As projects increase in size and complexity, it's crucial to identify what series of projects can be migrated to the cloud to alleviate intricacy pains and drive positive business outcomes.
. Patience is specifically important for foundational cloud Ops configurations and hybrid configurations, new data architecture standards and definitions to be set, and the development of proficiency in setting up configuring, optimizing, and billing cloud services.
O’Brien divided this roadmap planning into several phases:
- Map current state into framework
- Make initial improvements
- Adopt a governed data lake
- Adopt an enterprise data warehouse and analytics sandbox to developing new models, improving ETL, and replacing cubes with in-memory RDBMS
- Adopt data engineering and DataOps
When developing this data architecture framework, O’Brien urged attendees to do things in a way that benefit from the cloud instead of simply running on the cloud. This may lead to benefits such as improved quality through agile iterations, increased operating efficiency and cost, scalability, and flexibility among development teams.
Once this stage is complete,, the next significant step is ensuring that the data strategy proposed can be properly executed through resource planning and setting expectations.
O’Brien offered three ways to work with the business better to encourage successful data strategy execution: Think like a business stakeholder by sharing in business stakeholder goals and frustrations; balance architecture with delivery, as enterprise data architecture development is a journey of delivering business analytics projects and learning; and adopt an agile architecture mindset that acknowledges that data architectures evolve through lean, iterative, refactoring projects from the edge and delivery.
Architecture strategy is delivery, O’Brien explained; thinking of strategies as puzzles where the picture on the box is the architecture reference will enable enterprises to develop and adhere to the vision of an architecture that is both holistic and high-level. Pure data architecture projects should be able to demonstrate ROI, where value can be visualized and easily consumed.
O’Brien continued by highlighting that an open architecture will be optimal for driving interoperability, adherence to industry standards, and adopting valuable technology However, if a specialized tool will contribute significant value to a team, adopting that tool is certainly justifiable.
Ultimately, successful and effective data architectures are always evolving to enable better, faster, and confident business decisions. Additionally, adopting an “MVP” mindset—or minimum viable projects—will allow an enterprise to establish end-to-end delivery of a reusable architecture pattern. O’Brien concluded by explaining that new architecture patterns should be lean, initially, to save cost, as well as very scalable for ROI purposes.
Many Data Summit 2023 presentations are available for review at https://www.dbta.com/DataSummit/2023/Presentations.aspx.