SnapLogic, a provider of application integration software, has introduced a solution aimed at enabling easy connection and reliable large data integration between business applications, cloud services, social media and Hadoop. The product, called SnapReduce, transforms SnapLogic data integration pipelines directly into MapReduce tasks, making Hadoop processing more accessible and resulting in optimal Hadoop cluster utilization.
SnapReduce harnesses the power, performance and reliability of Hadoop with the ease of connection provided by SnapLogic's services oriented (REST-based) platform and language-neutral data containers - called "Snaps" - to enable scalable, very large data integration for cloud and on-premise business applications.
According to the vendor, unlike other Hadoop integration solutions that rely on higher-level languages such as Pig and Hive, SnapReduce matches SnapLogic dataflow pipelines directly to equivalent MapReduce jobs resulting in more efficient data processing and pipeline monitoring on Hadoop.
"This is Hadoop for humans," says Gaurav Dhillon, CEO of SnapLogic. Part of the company's offering is its Hadoop Distributed File System (HDFS) Snap, designed "to let people get big data in and out of Hadoop, which is kind of a janitorial task that is often ignored by the super-bright computer science crew that built Apache Hadoop," he explains. "A lot of these things are just hard for the average practitioner. They don't have Ph.D.s in computer science. It lets mere humans, the administrator of a database or salesforce or storage network handle large computational problems by using SnapLogic and the power of Apache Hadoop."
SnapReduce is also engineered to work side by side with applications and tools that process data within Hadoop, such as Pig, Hive, and Flume. This allows SnapReduce to work with existing Hadoop workflows and computations.
SnapReduce will be commercially available in the second half of 2011, the vendor says.
More details can be found at the SnapLogic website.