Canonical is pushing the limits on its MLOps platform to automate the full lifecycle of feature engineering, training, and release workflows for machine learning (ML) models.
The Canonical Data Platform team on Tuesday announced the release of its MLOps platform Charmed Kubeflow 1.4. The new free release enables data science teams to securely collaborate on AI/ML innovation on any cloud, from concept to production.
Charmed Kubeflow is an open source MLOps platform released under the Apache License 2.0. The platform helps data scientists automate the workflow from ideation to production.
This latest release includes upstream Kubeflow 1.4 with many improvements over previous versions. It now includes support for MLFlow integration.
Charmed Kubeflow deploys in any environment without constraints, paywall, or restricted features. Data labs and MLOps teams only need to train their data scientists and engineers once to work consistently and efficiently on any cloud or on-premises installation.
The platform’s main benefit is a centralized, browser-based system that runs on any conformant Kubernetes. Other benefits include enhanced productivity, improved governance, and reduced risks associated with shadow IT.
The latest release adds several features for advanced model lifecycle management, including upstream Kubeflow 1.4. Future releases will continue to focus on empowering data scientists and data engineers, according to Rob Gibbon, product manager at Canonical.
“One area of focus for the product is composability and extensibility via a component ecosystem,” he told LinuxInsider.
“Additionally, we will be continually improving solution enterprise readiness, and of course tracking upstream Kubeflow to ensure data scientists continue to get access to the very latest features in a fully supported manner,” said Gibbon.
Getting Started
Kubeflow is available now. Data scientists can get started with it using Juju, the unified operator framework for hyper-automated management of applications running on both virtual machines and Kubernetes.
The new release is in the CharmHub stable channel now. It can be deployed to any conformant Kubernetes cluster using a single Juju command:
juju deploy kubeflow
The full installation guide is available here for free. The software is open source with 24/7 support or fully managed service options available from Canonical.
Engineers and data scientists can rapidly set up an evaluation environment with or without GPU acceleration using just a single system running MicroK8s. Evaluators can read the getting started guide. It takes less than 30 minutes to start improving AI automation.
Under the Hood
This release provides better model lifecycle management with Kubeflow 1.4 and MLFlow integration. Kubeflow 1.4 comes with major usability improvements over previous releases, including a unified training operator.
The new training operator supports the popular AI/ML frameworks TensorFlow, MXNet, XGBoost, and PyTorch. This greatly simplifies the solution, improving future extensibility and consumes fewer resources on the Kubernetes cluster.
Kubeflow 1.4 has support for MLFlow integration, enabling true automated model lifecycle management using MLFlow metrics and the MLFlow model registry.
MLFlow is an open-source platform for AI/ML model lifecycle management. It includes features for experimentation, reproducibility, and deployment. MLFlow also offers a centralized model registry.
Using Integration
Data scientists and data engineers can use the MLFlow integration capability to build automatic model drift detection and trigger a Kubeflow model retraining pipeline.
Model drift occurs as model accuracy starts to decline over time due to changes in the live prediction dataset versus the training dataset.
Enabling MLFlow on a Kubernetes cluster and integrating it with a Charmed Kubeflow deployment using the Juju unified operator framework is straightforward, and the MLFlow Juju operator is available in CharmHub for immediate deployment.
Charmed Kubeflow 1.4 fully supports multi-user deployment scenarios out of the box for all Kubeflow components, including Kubeflow notebooks, pipelines, and experiments.
This update simplifies using Charmed Kubeflow to improve governance and reduce the occurrence of shadow-IT environments. It also helps to combat organizational data leakage.
The authentication provider integration guide provides more information on setting up multi-user access controls for the Charmed Kubeflow 1.4 MLOps platform.
Loading ...
