Blog / What's New in v0.12.0: Serverless Inferencing on Kubernetes with KServe
August 02, 2022 - Dickson Neoh Tze How - 4 mins read
Last updated: October 17, 2022.
The 0.12.0 release contains our KServe integration. KServe opens the door to highly scalable, simple, pluggable production ML serving.
We’ve been working on minor bug fixes and improving our docs page to improve your experience with ZenML.
We also added functionality to specify hardware resources on a step level to control the amount of memory, CPUs, and GPUs that each ZenML step has access to.
We’ve also added functionality to determine if secrets are shared with other ZenML Secrets Managers using the same backend.
As always, we’ve also included various bug fixes and lots of improvements to the documentation and our examples.
For a detailed look at what’s changed, give our full release notes a glance.
The spotlight in this release is the ZenML Kserve integration. This integration lets you define pipeline steps to scale up your model by serving your ML models on Kubernetes using any of your favorite ML frameworks like TensorFlow, XGBoost, Scikit-Learn, PyTorch from within ZenML!
Some steps of your machine learning pipeline might be more resource-intensive and require special hardware to execute. In this release, you can now specify the resources (such as the amount of memory, CPU, and GPU) to allocate on a step level in your pipeline.
To allocate resources to a specific step in your pipeline, simply specify
resource_configuration in the
from zenml.steps import step, ResourceConfiguration @step(resource_configuration=ResourceConfiguration(cpu_count=8, gpu_count=2)) def training_step(...) -> ...: # train a model
This currently works on
VertexOrchestrator, but will be extended in the upcoming weeks to support the
The majority of Secrets Managers use a single global scope in the back-end they are connected to. This means all secrets that are configured and visible through a Secrets Manager instance are also visible in all other instances. Furthermore, secrets configured directly in the back-end through means other than ZenML are also visible and can be managed in all ZenML Secrets Managers connected to that same back-end.
This has a number of disadvantages:
In this release, we’ve added support for scoped secrets in our AWS, GCP, and Vault Secrets Managers. These updated Secrets Managers allow you to configure a scope that determines if secrets are shared with other ZenML Secrets Managers using the same backend.
The latest release includes several smaller features and updates to existing functionality:
--decouple_storeserror message by @strickvl in https://github.com/zenml-io/zenml/pull/814
Join our Slack to let us know if you have an idea for a feature or something you’d like to contribute to the framework.
We have a new home for our roadmap where you can vote on your favorite upcoming feature or propose new ideas for what the core team should work on. You can vote without needing to log in, so please do let us know what you want us to build!