Skip to content
CodeLifter

What runs on your GPU?

Run a model on your own hardware and your prompts and code stay on your machine. Set your memory to see which open-weight models are likely to fit, how they scored on Arena on 15 October 2026, and which runner can load them.

A black memory module beside an orange-red paper tag on string

Fit calculator

Quantisation

Gemma 4 31B IT is the highest Arena-scored model on this list that fits, needing about 19 GB by our estimate.

This is CodeLifter’s own estimate, not a vendor figure: parameters × bits ÷ 8, plus 20% for context and runtime. For scale, llama.cpp’s documentation lists an 8B model at 4.58 GiB in its Q4_K_M format (4.89 bits per weight) and 7.95 GiB at Q8_0. Long contexts need more memory than this. Use the result as a first filter, then check the file size of the exact build you download.

Open-weight models by size

  1. Granite 4.2 3BIBM · Apache 2.0 · 3.7B parameters 3 GB needed Arena 1290
  2. Gemma 3 4B ITGoogle · Gemma Terms of Use · 4.3B parameters 3 GB needed Arena 1303
  3. Gemma 3 12B ITGoogle · Gemma Terms of Use · 12.2B parameters 8 GB needed Arena 1342
  4. Mistral Small 3.2 24BMistral AI · Apache 2.0 · 24B parameters 15 GB needed Arena 1356
  5. Qwen3.8 27BAlibaba Qwen · Apache 2.0 · 27.8B parameters 17 GB needed Arena 1438
  6. Gemma 4 31B ITGoogle · Apache 2.0 · 31.3B parameters 19 GB needed Arena 1452
  7. Hy3Tencent · Apache 2.0 · 299B parameters 180 GB needed Arena 1453
  8. GLM 5.3 FlashZ.ai · MIT · 321B parameters 193 GB needed Arena 1474

Bar shows memory needed against your setting; dimmed rows do not fit. Parameter counts are the totals in each model’s Hugging Face safetensors metadata, rounded, and the licence is the one on the same model card. Score is the model’s Arena text score read from arena.ai on 15 October 2026. Hy3 (21B active parameters) and GLM 5.3 Flash (18B active) are mixture-of-experts models, but the estimate counts every parameter, because llama.cpp’s documentation notes that models are fully loaded into memory.

Runners

  • Pulls and runs a model with one command, then serves it on localhost:11434, including a /v1 chat-completions endpoint that existing client libraries can point at.

    ollama run gemma4:31b
  • LM StudioProprietary

    Desktop app for finding, downloading and chatting with models, used under LM Studio’s own app terms. Its separate lms command-line tool is MIT-licensed and starts the local server.

    lms server start

  • Inference engine in C and C++ that loads GGUF files. llama-server exposes an HTTP API, and llama-quantize gives direct control over the quantisation format.

    llama-server -m model.gguf

  • Runs models on Apple silicon with the MLX array framework. mlx_lm.server serves a model from a local path or a Hugging Face repo over an HTTP API.

    mlx_lm.server --model <repo>
  • vLLMApache-2.0

    A high-throughput, memory-efficient inference and serving engine, in the project’s own words. vllm serve starts a server with list-models and chat-completion endpoints.

    vllm serve <model>

Licences checked against each project’s repository or terms, and commands against its own documentation, on 15 October 2026. Each runner name links to the page we checked.