Ollama

Ollama downloads and runs open-source models — Llama, Mistral, Gemma, Qwen, and dozens more — entirely on your own hardware with a single command, and exposes a local API compatible with the OpenAI format so anything built for GPT can point at a local model instead. Privacy-conscious developers and anyone without a cloud budget have made it one of the fastest-growing tools in the whole open AI ecosystem.