MLflow

MLflow is an open source AI engineering platform for agents, LLM applications and machine learning models. It helps teams debug, evaluate, monitor and optimize their work, and it is backed by the Linux Foundation.

MLflow captures traces of LLM applications and agents, built on OpenTelemetry, so you can see how each request was handled. It runs evaluations with more than 50 built-in metrics and LLM judges, and it can flag issues in traces for review. Prompts can be versioned, tested and deployed with lineage tracking. An AI gateway provides one OpenAI-compatible interface for many providers. Agents can be deployed as FastAPI endpoints with a single command.

• Traces of LLM apps and agents built on OpenTelemetry
• Evaluations with 50+ built-in metrics and LLM judges
• Prompt versioning and testing with lineage tracking
• AI gateway with one OpenAI-compatible interface
• Deploy agents as FastAPI endpoints with one command

MLflow suits ML engineers, data scientists and AI developers who need one tool across models and agents. A team with a single notebook and no deployment plans may find the setup more than it needs.

MLflow is open source under the Apache-2.0 license and free to self-host. The vendor also offers a managed option through partners. Running the server, storage and databases is your responsibility when you self-host, so plan for those as the project grows.