Kubeflow

Kubeflow is an open source AI platform built for Kubernetes. It is made of modular projects that help teams run data, machine learning and high-performance computing workloads on a cluster.

Kubeflow brings several subprojects together. Kubeflow Notebooks runs interactive development environments on Kubernetes. Kubeflow Trainer supports distributed training and fine-tuning across frameworks such as PyTorch, JAX and Hugging Face tools. Katib handles automated machine learning, including hyperparameter tuning. The Spark Operator runs Spark applications the same way other workloads run on Kubernetes. The project is a CNCF graduated project under the Apache-2.0 license.

• Interactive notebooks on Kubernetes
• Distributed training across frameworks such as PyTorch and JAX
• Automated tuning of model hyperparameters
• Spark applications run through a Kubernetes operator
• Modular subprojects you can mix and match

Kubeflow suits platform engineers, ML engineers and data teams that already run Kubernetes. A team with no cluster experience may find the setup steep.

Kubeflow is free and open source. Your costs are the cluster, the GPUs and the staff needed to run them. Because the project is modular, you can adopt only the parts you need, but each part adds something to maintain.