Semantic layers stand out as a key approach to solving business problems for organizations grappling with the complexities of managing and understanding the meaning of their data.
At Data Summit 2024, Joseph Hilger, COO, Enterprise Knowledge and Sara Nash, principal consultant, Enterprise Knowledge LLC, discussed “Building the Semantic Layer of Your Data Platform,” during their workshop session.
The annual Data Summit conference returned to Boston, May 8-9, 2024, with pre-conference workshops on May 7.
A semantic layer, also called a context layer, is a business representation of data that allows organizations to quickly map various data definitions, from multiple data sources to familiar business terms, offering a consistent and consolidated view of data.
“It’s new, people are talking about it but that doesn’t mean your organization is ready to fund it,” Hilger said. “There is no single tool that builds your semantic layer. The good news is that it’s made of products you probably already own.”
A semantic layer enables the ability to ask questions and receive answers back in context, Hilger explained. It can also increase access and trust in data, breaking down silos so more people within the organization can grab the information they need that may be further hidden in areas believed inaccessible.
“It can help solve the challenges around data lineage, accuracy, and integrity,” Nash said.
The underlying drivers of a semantic layer include content (knowledge, data, and information) is managed and accessible; data is connected across repositories, databases, and applications; and context and meaning are embedded with source data, making common understanding of data machine-readable. These key components are the foundations of the semantic layer, Nash explained.
“A semantic layer is a standardized framework that organizes and abstracts organizational knowledge and data (structured, unstructured, semi-structured) and serves as a connector for all organizational assets,” Nash said. “Another way to think about a semantic layer is that it’s your ‘Rosetta Stone’. It is really a central utility that can serve downstream use cases.”
A semantic layer solves several problems:
- Inefficient data analysis
- Non-intuitive user interactions
- Inconsistent metadata
- Poor data quality and governance
- Vendor lock
- AI hallucinations
“Ultimately what we want to create is a system where it’s easier for people to get the information they want,” Hilger said.
The components within a semantic layer include metadata, a business glossary, taxonomy/information architecture, ontology, and a knowledge graph, said Hilger and Nash.
“Coming up with definitions so people can look up information and know what it is, that is important,” Hilger said. “When you have consistent categories, it makes reporting easier.”
A graph database is better at relations than a relational database, Hilger said. This becomes the layer that pulls the data fabric, or semantic layer, together.
AI comes into play during the data source and semantic layer stage, said Nash. It can extract entities and relationships; summarize unstructured data; and recognize patterns and curate facts.
“If we want to find evidence of certain products being offered in certain regions, we could use AI to recognize patterns,” Nash said.
A semantic layer is not just for data; not a single product; not just a graph; not all your content; and not an automated/AI-generated solution.
“It’s not just for data, it’s for all of your knowledge assets,” Nash said. “It’s not a one size fits all solution either. There are many different ways to approach this framework using existing tools and processes.”
Top enterprise use cases include:
- Customer/Enterprise 360
- Supply chain and Environmental Social Governance (ESG)
- Data as a Product
- Context and reasoning for AI and ML
- Mergers and acquisitions
- Content personalization
- Data quality and governance
- Semantic search
Many Data Summit 2024 presentations are available for review at https://www.dbta.com/DataSummit/2024/Presentations.aspx.