What’s New in v0.10.0: A Kubernetes Native Orchestrator! | ZenML Blog

Last updated: October 17, 2022.

The 0.10.0 release continues our streak of extending ZenML with support for new orchestrators, this time by adding the Kubernetes Native Orchestrator. Also included are: a Data Validator stack component and Great Expectations implementation and a community-contributed Vault secret manager among a host of other things! ✨

Release GIF

Beyond this, as usual we included a number of smaller bugfixes and documentation changes to cumulatively improve experience of using ZenML as a user. For a detailed look at what’s changed, give our full release notes a glance.

🛳️ Kubernetes Native Orchestrator

We’ve heard you! With this new addition to our increasing list of orchestrators, you can now run your ZenML pipelines natively on your Kubernetes Cluster.

This orchestrator is a lightweight alternative to other distributed orchestrators like Airflow or Kubeflow that gives our users the ability to run pipelines in any Kubernetes cluster without having to install and manage additional tools or components.

It’s amazing but don’t take my word for it; try it on your own or wait for the dedicated blog post that we’ve planned, which, by the way, also features a little surprise to make it easier for you to follow along 😉

Kubernetes Orchestration pods

There’s a lot to love about this integration.

We can’t wait to hear your thoughts on this 🙂

🎉 Great Expectations as the new Data Validator

Want to run data quality checks as part of a ZenML pipeline? We’ve got you covered!

We introduce Data Validators and The Great Expectations integration which eliminates the complexity associated with configuring the store backends for Great Expectations by reusing our Artifact Store concept for that purpose and gives ZenML users immediate access to Great Expectations in both local and cloud settings.

Great Expectation Validation Result

In addition, there are two new standard steps:

A ZenML visualizer that is tied to the generated Great Expectations data docs is also included and can be used to visualize the expectation suites and checkpoint results created by pipeline steps 😍

Got feedback or just want to let us know how you feel? Connect with us or join us for a Community Meetup 👋 which happens every Wednesday!

🔐 A Vault Secret Manager

To add to our growing list of secret managers, we now have a Vault integration, courtesy of one of our community members, Karim Habouch! We are grateful for their contribution ⭐

⌨️ CLI Improvements

A new release means new improvements to the CLI. We made changes to make handling stacks a bit easier 🥰:

📖 Documentation & User Guides

As usual, user-facing documentation is really important for us and we made a bunch of fixes and additions towards that end.

➕ Other Updates, Additions and Fixes

The latest release include several smaller features and updates to existing functionality:

🙌 Community Contributions

We received several new community contributions during this release cycle. We mentioned Karim’s Vault Secret Manager above already, but here’s everybody who contributed towards this release:

👩‍💻 Contribute to ZenML!

Join our Slack to let us know if you have an idea for a feature or something you’d like to contribute to the framework.

We have a new home for our roadmap where you can vote on your favorite upcoming feature or propose new ideas for what the core team should work on. You can vote without needing to log in, so please do let us know what you want us to build!

[Photo by Roberto Arias on Unsplash]

More from us: