At the Strata Data conference in New York, Attunity, a provider of data integration and big data management software solutions, showcased the new release of its data integration platform designed to address the changing needs of companies with advanced analytics and data management initiatives.
The new release includes Attunity Replicate 6.0 along with new versions of Attunity Compose for Hive and Attunity Enterprise Manager (AEM), to offer expanded capabilities that enable large-scale and accelerated data pipeline automation across cloud, data lakes, and streaming enterprise architectures.
“We have a platform, with multiple products within the platform, and more and more points of integration,” said Kevin Petrie, senior director and technology evangelist at Attunity. For example, Attunity Replicate integrates with Attunity Compose for Hive so that when you have data landing in the data lake, Compose for Hive will then take that and create schemas and data stores within the Hive architecture, and load data into that from staging and also do ACID merge—loading any updates in an ACID-compliant way into those data stores. This enables organizations to effectively have SQL data warehouses on top of Hadoop, said Petrie. They get the SQL structure, the familiar ACID compliance of the data warehouse but they are still leveraging the processing power and the economic storage underneath. “That is just one example of how we are stitching our products into a meaningful platform to enable data pipelines,” he said.
“We also have Attunity Enterprise Manager (AEM) available as an add-on extension to Attunity Replicate and that will enable people to look across their organization and manage thousands of replicate tasks across of hundreds and potentially thousands of endpoints through one pane of glass,” said Petrie, noting that the AEM layer is going to become the primary portal for managing all the products within the platform with more capabilities being added to the AEM in the future.
“There are three primary directions for where data is flowing,” said Petrie. It is going to cloud, data lakes and streaming architectures. They obviously overlap quite a bit and it is coming from deeply established systems that have the lion’s share of company data—SQL Server, Oracle, and mainframe systems. “Those types of systems represent an incredibly rich repository that companies want to feed into their data lakes and their streaming architectures, ideally on the cloud in a lot of cases because it is the most cost effective and elastic option to address new analytics use cases. That is the theme and the direction of where we are headed and we will be filling out the platforms that we support in order to executive on that vision,” said Petrie.
According to Petrie, many legacy data integration tools are not able to handle the necessary volumes of data feeding to the cloud at the required performance levels, and doing that without compromising security is a challenge. To address this, Attunity provides encrypted multi-pathing, enabling it to send data in one or more AES 256 encrypted channels across the wide area network to have it securely reach its target. Attunity also provides support for the full range of major cloud platforms, including Azure, AWS, and Google. “Organizations want to be able to move to those platforms quickly and then move between them as needed,” said Petrie.
By enabling people to move data securely and with high performance over the wide area network, Attunity is helping organizations address two main challenges, said Petrie. Another primary challenge is copying source production data that is on-premises and supporting thousands of users with online transaction procession. “How do you copy that data without interfering with production operations? We have the most polite, non-intrusive method of copying real time changes to those platforms because we simply remotely scan and scrape change logs to identify real-time changes, make live replicas and send them to the target without needing any source agents, or triggers to take away processing power from the source.” The combination of “politely copying” real time data from the production sources, moving it securely over the wide area network, and moving it at high speed are three ways that Attunity is helping to overcome the challenges of typical legacy tools and helping organizations move more easily to the cloud, said Petrie.
The EU’s upcoming GDPR was also a big topic of conversation at Strata. Attunity products will contribute to GDPR compliance, said Petrie, by enabling organizations to mask selected datasets to provide an extensive audit trail of exactly what is happening to data throughout its integration path. “We also have a product called Visibility which will identify within in a data warehouse or Hadoop datastore what queries are being run by whom on what datasets.” If an organization has a customer who has explicitly opted out of any activities except for basic transactions related to things they buy, then that organization needs to make sure that the data warehouse only reflects those types of activities. “Attunity Visibility can provide the transparency to further assist GDPR,” he said.
GDPR really affects all multinational organizations because they all have European customers, he noted, pointing out that some U.S.-based companies and U.S.-based operations of multinational companies may be coming a little late to the game since GDPR goes into effect in May 2018. “We hear from our UK office which serves all of EMEA that GDPR comes up in almost every conversation with customers there.”
For more information, go to www.attunity.com.