Open source
SGLang
Open-source framework for serving large language models
Local RunnersFast LLM serving framework
Alternatives to vLLM
12 tools like vLLM, chosen by what they do. About vLLM
Open source
LocalAI
Self-hosted server with an OpenAI-style API for local models
Local RunnersSelf hosted OpenAI style API
Open source
GPT4All
Desktop app for private chat with local AI models
Local RunnersPrivate local chat on your PCInactive
Curated
Msty
Private AI workspace for chatting with and organizing models
Local RunnersSimple desktop LLM client
Curated
Open WebUI
Self-hosted interface for chatting with AI models
Local RunnersSelf hosted AI chat interface
Open source
AnythingLLM
Private AI assistant app that runs on your own computer
Local RunnersAll in one local AI app
Open source
Jan
Open source app for running AI models on your own computer
Local RunnersOffline ChatGPT alternative
Open source
LibreChat
Open source chat interface for many AI model providers
Local RunnersOpen source multi model chat
Curated
Baseten
Hosting platform for deploying and scaling AI models
Inference PlatformsProduction model serving
Curated
Open source
