BentoML

BentoML is an open source framework for packaging machine learning models and serving them as applications. It is designed to turn a model into a service that other systems can call.

BentoML lets you package a model or a custom inference pipeline together with its code and dependencies. The package can be served as an API and deployed on your own cloud, on premises or on Kubernetes. A commercial inference platform sits on top, adding deployment automation, monitoring, access control and resource tracking. The vendor's catalog includes popular open models that can be launched with a few clicks.

• Package models and custom inference pipelines
• Serve models as production APIs
• Deploy on your own cloud, on premises or Kubernetes
• Platform layer for deployments, monitoring and access control
• Open model catalog for popular models

BentoML suits ML engineers and platform teams who need to serve models to other services. Someone who only trains models in a notebook may not need a serving framework.

The open source framework is licensed under Apache-2.0 and is free to use. The managed inference platform and cloud compute are priced separately, and the site asks for a demo for those. Serving models adds ongoing work, including scaling, monitoring and security updates.