The Linux Foundation on Thursday announced that it’s forming theOpenHPC Collaborative Project to push the software supply to support high-powered computing.
The project will provide a new open source framework to meet HPC’s unique application demands and parallel runtime requirements, the organization said.
The framework will provide upstream project components, tools and interconnections to enable the software stack. The HPC components will contribute to a full-featured reference software stack for developers, system administrators and users.
The open source software stack will be framework-agnostic. It will provide flexibility for multiple configurations and scalability to meet a variety of user needs, according to the Linux Foundation.
A key aspect of the development work is the creation of a stable environment for software testing and validation. The project will include a continuous integration environment for building infrastructure and maintaining source control.
“Open source is prevalent in HPC environments,” said Jim Zemlin, executive director at the Linux Foundation.
“Now is the natural time to standardize on a framework that can accelerate new technologies for some of the most sophisticated computing environments in the world,” he told LinuxInsider.
Who’s Included
Industry leaders on several continents have signed onto the initiative, including Allinea Software, Altair, Ansys, Atos, Cray, Dell, Fujitsu Systems Europe, Hewlett-Packard, Intel, Lenovo, MSC Software, NEC, Senai Cimantec and SUSE.
Founding members also include Argonne, Barcelona Supercomputing Center, the Center for Research in Extreme Scale Technologies at Indiana University, Jlich Supercomputing Centre, Lawrence Berkeley National Laboratory, Lawrence Livermore National Laboratory, Leibniz Supercomputing Center, Los Alamos National Laboratory, Pittsburgh Supercomputing Center, Sandia National Laboratories and the Texas Advanced Computing Center.
What’s Ahead
The core components of the stack have been proposed. The full framework will be determined by the membership once a board of directors and technical steering group are elected, Zemlin said.
While the use of open source software is central to HPC, the industry lacks a unified community across key stakeholders. These include academic institutions, workload management companies, software vendors and computing leaders. That has caused duplication of effort and increased the barrier to entry, he noted.
“Neutral governance in open source projects is extremely important to keep the community united around a common purpose and push code forward,” Zemlin said.
“By hosting a project at the Linux Foundation, member companies involved in a project can focus on code development and technical milestones, rather than legal, marketing and other efforts that are essential to collaborative development but can serve as a distraction for the technologists,” he added.
Business Needs
Intel’s x86 architecture has become the platform of choice for high-end HPC and supercomputing. Linux has followed closely in popularity and prominence, according to Charles King, principal analyst at Pund-IT.
“It makes great sense that the Linux Foundation is stepping in with the open source collaborative project. That said, the foundation will face some challenges, particularly as HPC continues to drive into the mainstream among a growing range of business use cases and workloads,” he told LinuxInsider.
If the foundation can move the project successfully and keep up with business needs, then all will remain good. If not, enterprises leveraging Linux and HPC may look for other alternatives, King warned.
Keeping Linux Validated
The prevalence and influence of Linux in the HPC market has validated the importance of Linux, according to Jay Lyman, research manager for cloud platforms at 451 Research. Linux and related open source software are a significant part of HPC.
The Linux Foundation is a proven leader in this effort. The OpenHPC project is right in line with the organization’s expertise and experience in promoting open source community and industry standards in tandem, he told LinuxInsider.
“The impact of these types of projects, communities and consortia are largely tied to involvement and participation from developers and end users, who ultimately determine their success,” Lyman said.
Alternatives Exist
Software remains one of the biggest challenges for HPC user adoption, according to IDC’s “Worldwide HPC Server 2015-2019 Forecast.”
Open source and Linux-based software components have become a standard way to reliably test and maintain stable operating conditions while providing a cost-effective means for scaling with data growth, according to the Linux Foundation.
Other open source software efforts exist should OpenHPC’s efforts fail. These include projects and products in database, networking, storage and other areas that represent alternatives to proprietary software, noted Lyman.
“In addition, with the breadth and variety of open source software tools and components, there are a number of options for developers and teams building and working on HPC environments. The OpenHPC Collaborative Project should help bring some of these different pieces together and help the industry coalesce around best practices and standards,” he said.
Get the Bugs Out
One thing the Linux teams will need to keep in mind is how to do dynamic analysis and debugging of the applications that run on top of the open source framework, according to Marty Bakal, principal product manager atRogue Wave Software.
“The difficulty behind analysis and debugging of HPC applications is the fact that everything runs in parallel, and hence isolating failures, memory leaks and crashes are very time-consuming,” he told LinuxInsider.
That situation gets worse when applications use complex data types that have to be viewed as memory dumps in debuggers, he said. Finding and fixing these failures quickly is what can keep development efforts on time and budget.
Loading ...
