Hortonworks Ramps Up Hadoop Security

Hortonworks this week announced a series of enterprise security efforts to bolster performance and data safety with its Hortonworks Data Platform.

The company announced Tuesday thatPivotal Software will standardize on Hortonworks’ Hadoop distribution. Hortonworks also will resell extract, transform and load tools developed bySyncsort.

The thrust of the Hortonworks’ product announcements, which were made in conjunction with its Hadoop Summit, concerned updates on applying security policies and maintaining data governance to simplify the provisioning of clusters in hybrid clouds. Those procedures were designed to make it easier for customers to interactively explore data in Hadoop.

Hadoop is an open source framework for storing and processing big data in a distributed environment across clusters of computers using simple programming models. It was designed to scale up from a single server to thousands of machines, each offering local computation and storage.

“Security is not just about preventing malware; it also includes integrity of the data,” said Scott Petry, CEO ofAuthentic8.

“Some of the capabilities announced that focus on data governance will have a broader impact on the overall security posture of their Hadoop installations,” he told LinuxInsider.

Integrated Outlook

Apache Ranger for security and Apache Atlas for data governance have been integrated to define and implement dynamic classification-based security policies.

A technical preview lets enterprises use Atlas to classify and assign metadata tags. Ranger then enforces various access policies. Atlas also provides cross-component lineage to provide a better view of data movement across multiple components.

A major improvement is the integration with Apache Metron, an open source system for identifying malicious activity, Hortonworks said. The project improves the security of Hadoop’s internal deployment and provides a real-time threat-detection engine based on another open source component called Apache Storm.

Storm scans the activity records from an analytics cluster for signs of known malware. If it finds evidence of malware, Metron tries to block it and then issues alerts.

“We are very excited about the progress Metron has made in the past few months and believe it will be revolutionary technology. Comprehensive security in real time will become necessary, and Metron has the ability to combat the cybersecurity threats of today, as well as the new types of threats we will see in the future,” a company spokesperson said in comments provided to LinuxInsider by company rep Taylor Hassman.

Integrating Apache Ranger and Apache Atlas into Hortonworks’ platform gives customers the necessary classification-based security tools to keep their data safe. One of the reasons customers choose Hortonworks is its commitment to open source, the spokesperson said.

Adding Ranger and Atlas to HDP lets the company stay true to its mission and gives customers a superior experience while allowing them to avoid vendor-lock in, the spokesperson said.

Hortonworks also announced the release of Cloudbreak 1.2 during the Hadoop Summit. Cloudbreak is part of the Hortonworks Data Platform. It is a unifying system for provisioning HDP workloads across cloud infrastructure.

No Overt Concerns

Security with Hadoop distributions is not a problematic area, said Charles King, principal analyst at Pund-IT. The bigger picture is Hortonworks’ intention to build security and related management functions into Hadoop that satisfy the requirements of the company’s enterprise customers.

“This is more about making Hadoop fit conventional enterprise use cases than it is about inherent security issues or weaknesses,” he told LinuxInsider.

When users build out large-scale hosted applications, they have to ensure a security perspective that spans everything from physical access to resources to integral data controls, noted Authentic8’s Petry.

“This continuum of risk is consistent whether it is a Hadoop-based application or not. There is nothing inherently unique in the cloud-Hadoop equation that would suggest any additional concern in cloud-based deployments,” he said.

A Cloudy Picture

Still, some uncertainty exists about the impact Hortonworks’ latest measures may have on open source Hadoop enterprise security, according to King.

Two of the initiatives — the integration of Apache Atlas and Apache Ranger — focus on the role of management and data governance. Apache Metron is still in incubation, but it works at application, system and packet levels (and reads feeds from tools) to find anomalies indicating that an attack may be occurring, the company said.

“If the efforts all succeed, they could provide the basis for larger Hadoop-related security solutions, including offerings from traditional security vendors. But that’s a ways ahead of these announcements,” King said.

Any efforts to beef up data security benefits the entire big data workspace, noted Petry.

“Looking at the governance capabilities, these seem to address the needs of customers in regulated or pseudo-regulated industries. Adding this type of capability on an already-accepted open source solution can only improve prospects for broader enterprise adoption,” he said.

Sunny Outlook

Hortonworks’ latest moves are likely to have a positive effect on its competitive position. The company is “certainly progressing quickly with these moves,” said King.

That, combined with Pivotal’s announced plans to align itself more closely with Hortonworks, bodes well for the company’s prospects. It is a show of strength and is important given the recent moves of Hortonworks’ biggest competitor,Cloudera, he added.

“These new Hortonworks solutions and alliances could not have come at a better time,” King said.

The demand for big data solutions is increasing. With that comes a trend for customers who want fully managed solutions that include Hadoop with Metapod, said Niki Acosta, director of evangelism at Metacloud, which is owned by Cisco.

“I think it is great to see Hortonworks releasing products to address the concerns of enterprise buyers,” she told LinuxInsider. “Security seems to lag with open source projects until there is enterprise adoption. This is a validation that the market is finding value in big data solutions.”