Last updated: November 3, 2022.
ZenML 0.8.0 – and a number of changes around the edges including our messaging
and website – is all about extensibility. We’ve been working hard to make it
clear and comprehensible how you can add custom components to the core
framework. You’ll love how we’ve upgraded and beautified the CLI, even giving it
a speed boost while we were there!
Beyond this, a host of smaller bugfixes and documentation changes combine to
make ZenML 0.8.0 a more pleasant user experience overall. For a detailed look at
what’s changed, give our full release
notes a glance.
⏲️ EVENT: Join the ZenML team for an MLOps Day
We are hosting a MLOps day on Wednesday, May 25, where we’ll be building a
vendor-agnostic MLOps pipeline from scratch.
to join the entire ZenML team in showcasing the latest release, answering the
community’s questions, and live-coding vendor-agnostic MLOps features with the
🧘♀️ Extensibility is our middle name
From talking to many of you in our Slack community, we know that custom stack
components and integrations is often at the top of your collective minds. ZenML
0.8.0 makes it much easier to extend the core ZenML framework. To achieve this,
we changed the underlying code and wrote better documentation and guides on
exactly how to do this.
- We added the ability to register custom stack component flavors, and
was renamed to
- We added the ability to easily extend orchestrators along with extended
documentation on how you can do this yourself.
- Our documentation for stacks, stack components and flavors has been fully
fleshed out. We know you’re going to love it!
- We added in new MySQL metadata stores along with the ability to use SSL to
connect to those MySQL clients (e.g. in the cloud)
🥞 Manage your Stacks
We added some extra functionality that’s going to improve your quality of life
considerably when interacting with your stacks!
- You can now update your stacks and stack components directly! No need to
delete and re-register things from scratch. Check out the CLI
docs to learn how this works.
- You can also remove custom (optional) attributes from stack components
- When deleting stacks with
stack delete..., we’ll be sure to confirm with you
before we go ahead and delete the stack now.
- You can export the stack to
a shareable YAML file with
zenml stack export and import from a compatible
YAML file with
zenml stack import. For the full collaborative experience
you’ll want to use the
ZenServer, but this will
still be a useful feature for some of you!
- The ZenServer now enables shared projects along with the foundations of a
shared and collaborative user experience. We added some functionality to
interacting with your shared ZenServer.
- All of the new collaboration features have been fully documented
🖥 CLI Improvements
- We saw how it was sometimes difficult to see the full details of stack
components and their configuration when using the CLI tool, so we made some
fixes and tweaks to make sure text doesn’t get shortened. Instead, it just
wraps onto the next line.
- We added a
zenml go command which launches an easy tutorial on how to get
started with ZenML and some of its features.
- We’ve noticed that the CLI has started to become a little sluggish and took
the first steps towards adding some 🚄 speed back in.
- We’ve been adding more and more stack components to handle the full MLOps
story, so the list of options in the CLI has been growing and growing. With
this release you’ll see that the CLI options are split into menu groups which
should make everything a lot more comprehensible and navigable! Check it out!
Doesn’t it look beautiful now?
🚀 New Integrations
- We saw we didn’t have a PyTorch
example on the
books yet, so we added that in.
- We added a GCP ☁️ Secrets Manager to sit alongside our AWS Secrets Manager
📖 Documentation & User Guides
- We completely overhauled the ZenBytes introductory guide. You not only learn
about ZenML but we teach some MLOps basics from the ground up. Check out the
notebook-based lessons here! (Shout
out to Felix for making this happen!)
- Like our CLI commands, our examples directory was starting to get a bit
unwieldy with all the additions so we updated how that all looks and is
presented. The main
README subdivides the
examples by use case.
➕ Other Updates, Additions and Fixes
- We improved how secrets are passed around in a deployed or cloud setting.
- We added additional metadata tracking for pipeline runs which adds to what you
can see as part of the post-execution workflow. (This includes tracking the
Git SHA, the Docker SHA as well as docstrings.)
- Model deployer logs can now be streamed through the CLI and not just accessed
post-facto. Viewing the logs is as simple as calling
zenml model-deployer models
logs from the CLI.
- After popular request, the
--force flag has now been replaced with
--yes. Don’t worry, though, this won’t break your workflows just
yet. We’ve deprecated the
-f flags and you’ll see an error message for a
while before we remove it completely.
- You can now pass in-line pip requirements into your pipeline decorator instead
of only being able to pass in a
requirements.txt file. We also renamed this
requirements_file to just
requirements, but don’t worry we
just deprecated the old parameter to give you time to adjust your code.
- We fixed the Evidently drift detection
so that it now works on Google Colab notebooks again. (Hamza is doing an AMA
with the Evidently community on May 24; click
We received a new community contribution during this release cycle, from Avram
Ðorđević in Belgrade. It’s great to have the community returning to contribute
to the codebase! Avram added support for scipy sparse matrices in his
👩💻 Contribute to ZenML!
Join our Slack to let us know if you have an
idea for a feature or something you’d like to contribute to the framework.
We’re super excited to be able to share the new home for our
roadmap where you can vote on your favorite upcoming
feature or propose new ideas for what the core team should work on. You can vote
without needing to log in, so please do let us know what you want us to build!
[Photo by Audrey Martin on Unsplash]