The Linux Foundation is doubling down on open-source collaboration to drive more innovation in technology that uses artificial intelligence (AI), machine learning (ML), and deep learning (DL). The LF AI Foundation and ODPi merged on Oct. 25 to form the LF AI & Data Foundation.
This new organization, under the auspices of the LF, will build and support an open community and a growing ecosystem of open-source AI, data, and analytics projects.
The LF AI Foundation built an ecosystem to enable and sustain open-source innovation in AI, ML, and DL. ODPi was a nonprofit organization accelerating the open ecosystem of big data solutions. The merger is significant as AI and data are inseparable and codependent on each other.
Combining efforts in both spaces brings developers and projects under a single roof, orchestrated by a single Technical Advisory Council and several committees (Trusted AI, BI & AI), to work together towards building the open-source AI and data ecosystem and accelerating development and innovation.
The combined entities provide unified guidance for end users on tools, interoperability, integration, standards, and the future of AI, data, and analytics as its use continues to grow in every industry. Another advantage sought in the merger allows greater efficiency for members across the various services LF offers to its hosted projects.
Hosting projects under a single umbrella enables closer collaboration, integration, and interoperability across projects. It is a proven recipe for building strong open ecosystems, according to LF officials.
LF AI has added 13 projects so far this year, noted Ibrahim Haddad, LF AI executive director, including several data projects.
LF AI & Data timeline of project developments in 2020
“What is making AI so attractive or viable across so many industries? Saving time and money, providing better personalization to users, generating business insights, automating processes, improving productivity, etc.,” he told LinuxInsider. Adding, “Open source offers unique attraction to AI in terms of fairness, lineage, explainability, robustness, and open data.”
Well-Designed Plan
The merger is a natural move to bring together the open AI and data communities. Some of the reasoning that supports the move needs some explanation, Haddad suggested.
Fairness is assured with methods to detect and mitigate bias in datasets and models. Fairness counters bias against known protected populations.
Robustness applies to detecting alterations or tampering with datasets and models. This also takes into account alterations from known adversarial attacks.
Explainability methods enhance understandability and interpretability by persona/roles in the process of AI model outcomes/decision recommendations. It includes ranking and debating results and decision options.
Lineage methods ensure the provenance of datasets and AI models. This takes into account the reproducibility of generated datasets and AI models.
Open Data requires consistent methods to clean, sort, tag, and track provenance and a governance structure for doing these things.
“The open-source development methodology is ideal to ensure openness, transparency, and collaborative approach to building AI systems or AI-infused services that are fair, robust, and explainable,” said Haddad.
Integration Essential
Over the past five years, ODPi has been a part of driving standardization and consolidation in the Big Data and Hadoop ecosystem. That organization has become a focal point for the data challenges of today’s enterprises, such as metadata, governance, and data science, said John Mertic, ODPi’s director.
“AI complements the existing data strategies an organization leverages, enabling them to serve customers better, reduce costs, and have more consistent outcomes,” he told LinuxInsider. “Combining AI with standards-based data management tools such as Egeria ensures the data policies an organization has will be adhered to, letting this scale.”
Coming together to form LF AI and Data is the next natural step in this mission, enabling greater interoperability to drive both innovation and sustainable growth for key projects, he offered.
ODPi and its projects Egeria and OpenDS4All will become hosted projects under the LF AI and Data Foundation. BI and AI become a committee within the foundation.
They will maintain their current open technical governance model, establish collaboration with other hosted projects in LF AI & Data via the efforts of its Technical Advisory Council, and benefit from a host of services offered to facilitate collaboration and increased adoption.
“This joint foundation enables the key open source projects the industry depends on to have a sustainable home, which will drive further innovation and collaboration,” said Craig Rubendall, ODPi’s chairperson.
Greater Opportunities
The move under LF’s umbrella also provides a higher degree of opportunities to contribute for those committed to advancing open-source AI and data as an industry-standard infrastructure and democratizing access to AI, according to Jim Spohrer, director, IBM cognitive open technologies chair, Technical Advisory Council, LF AI & Data Foundation. Two avenues for doing so are joining the bi-weekly TAC meetings and connecting on the LF AI slack channel.
The pandemic continues to accelerate digital transformation. This extends from simply doing more online to growing bigger datasets to developing more complex processes that can integrate AI/Robotics where time to value makes sense, he said.
“The result has been an explosion in open-source AI and Data projects from companies across industries globally with some of the most popular projects captured in the ever-growing interactive landscape,” Spohrer told LinuxInsider.
As AI finds more viable applications across industries, there is also a growing need to ensure the technical reality of trusted AI systems that are robust, fair, and explainable,” he continued. “Like a gold rush, more developers are being attracted to popular projects with millions of lines of new code being added each month.”
Loading ...
