Metaflow

Metaflow is an open source Python framework for building and managing machine learning, AI and data science projects. It is designed so that a project can start on a laptop and scale to the cloud without a rewrite.

Metaflow lets you write workflows in plain Python. Each step's variables and data are tracked and versioned automatically, which makes experiments easier to compare and debug. You can run steps locally and then send the same code to the cloud, using GPUs, many cores or large memory when needed. Workflows can be deployed to production with a single command and can react to new data or events.

• Write workflows in plain Python
• Run steps locally, then scale to the cloud
• Track and version data between steps automatically
• Use GPUs, many cores and large memory when needed
• Deploy to AWS, Azure, Google Cloud or Kubernetes

Metaflow suits data scientists and ML engineers who want a clear way from notebook to production. Teams that only run a single script once a month may not need a workflow framework.

Metaflow is open source under the Apache-2.0 license and is free to use. Your cloud costs are the main expense, since the framework runs on your own accounts. A hosted sandbox is available in the browser for trying it out before you set up a stack.