LocalAI

LocalAI is an open source server that runs AI models on your own hardware. It gives you an API that follows the OpenAI format, so existing tools can point at a local address instead of a cloud service.

LocalAI runs text, voice, vision, image and video models through one runtime. It exposes compatible endpoints for the OpenAI, Anthropic and Ollama APIs, so many clients need only a URL change. Each model can use a different engine, such as llama.cpp, vLLM or MLX, and the client does not need to know which one is in use. The project also includes support for agents, MCP tools and retrieval.

• OpenAI-compatible API served from your own machine
• Language, voice, image and video models in one server
• Runs on CPU only, without a GPU
• Switch the inference engine per model with a config change
• Agent, MCP and retrieval support in the same runtime

The tool suits developers and teams that want to keep data on their own hardware. It is also useful for anyone who needs an API for testing without paying per request.

LocalAI is released under the MIT license and is free to use. Your costs are the hardware and electricity needed to run it. Larger models need more memory, and a GPU helps with heavier workloads even though CPU-only setups are supported.