Best Tools for ML Pipelines, Serving, and model graph visualization | #03
Do you want to know how to get your ML models up and running quickly?
You can do it 2x faster using the tools I’m going to talk about today. I know it might sound too good for you, but it’s not.
Using the tools I will later describe you can get a significant speed up in the following 3 key areas of your MLOps efforts:
- Pipelines
- Model Development and Serialization
- And Production Deployment.
Let’s dive in…
Netron — visualizing models
Netron is an interactive viewer for ML and DL modes.
It allows you to visually inspect the model graph and see how tensors flow through each node. You can see things like input size, output size, activations, etc.
Netron comes in two flavors: hosted and on-perm versions.
Netron supports
- ONNX,
- TensorFlow Lite
- TensorFlow JS
- Caffe 1 and 2
- Keras,
- Darknet,
- PaddlePaddle,
- Core ML,
- MXNet.
Netron has experimental support for:
- PyTorch
- TensorFlow
- TorchScript
- Scikit-learn
BentoML — serving models
According to their docs, it’s a framework that makes it easy to create ML services that are ready to deploy.
They state on their docs that it’s “The Unified Model Serving Framework”.
At a high level it’s like fastAPI on steroids, imagine if fastAPI did model management and inference scheduling as well.
Like my friend Ariel (MLOps meme guy) posted, it’s another MLOps tool that uses YAML to glue things together and enable heavy customization, it’s not really a bad thing but it’s good you know.
Key Features
- Simply API
- Easy to get up and running quickly
- Allows for heavy customization (i.e., multi-model deployment. In both sync, async, or parallel)
- Ability to containerize the service
- Ready for production (reasonable scale simple HTTP API to hyper-scale with distributed systems like k8s)
- Standardizes the saved model format, service API definition, and the Bento build process.
- Integrations
- MLFlow
MLflow Pipelines
MLflow Pipelines was recently introduced and will be a part of the MLflow v2.0 release. It is an experimental and opinionated framework for structuring MLOps workflows that simplifies and standardizes machine learning application development and productionization. MLflow Pipelines make it easy for data scientists to follow best practices for creating production-ready ML deliverables, allowing them to focus on developing excellent models. MLflow Pipelines also enable ML engineers and DevOps teams to seamlessly deploy these models to production and incorporate them into applications.
MLflow Pipelines provides production-quality Pipeline Templates for common ML problem types, such as regression & classification, and MLOps tasks, such as batch scoring. Pipelines are structured as git repositories with YAML-based configuration files and Python code, offering developers a declarative approach to ML application development that reduces boilerplate.
MLflow Pipelines also implements a cache-aware executor for Pipeline Steps, ensuring that steps are only executed when associated code or configurations have changed. This enables data scientists, ML engineers, and DevOps teams to iterate very quickly within their domains of expertise. MLflow offers APIs and a CLI for executing pipelines and inspecting their results.
Key Features
- Get started quickly: Predefined templates for common ML tasks, such as regression modeling, enable data scientists to get started quickly and focus on building great models.
- Iterate faster: The intelligent pipeline execution engine accelerates model development by caching results from each step of the process and re-running the minimal set of steps as changes are made.
- Easily ship to production: The modular, git-integrated pipeline structure dramatically simplifies the handoff from development to production by ensuring that all model code, data, and configurations are easily reviewable and deployable by ML engineers.
- Well integrated with DataBricks
Thank you for reading! I will be bringing new tools every week so follow me to get notified of my next release.
You can also follow me on Twitter.
I appreciate you!
If you want to read more of my ML and MLOps takes, consider becoming a member. Your membership fee directly supports the writers you read. In addition, I’ll earn a small commission if you sign up using my link.
Below you can find many references for you to check out and devour. Have fun!
