Alternatives to

llama.cpp

The C++ engine that made local LLMs practical

7 approved alternatives Suggest an alternative

The most recently approved AI tools people use as an alternative to llama.cpp (The C++ engine that made local LLMs practical), newest first.

Modular

MAX and Mojo for fast portable AI inference

Report

LocalAI

Drop-in OpenAI-compatible API for local models

Report

TensorRT-LLM

NVIDIA's library for optimized LLM inference

Report

SGLang

Fast serving framework for LLMs and multimodal models

Report

vLLM

The open-source inference engine most self-hosted LLM deployments run on

Report

Ollama

Run open-source LLMs locally with one command

Report

LM Studio

Desktop app for downloading and running local LLMs

Report

Sign in to continue

It's easier when you're signed in — Altern helps you get more out of AI.

By continuing you agree to our Terms and Privacy Policy.