Runtimes
Which local inference runtimes B3IQ supports, which one a given machine should use, and how one-click install verifies each build.
A runtime is the local engine that actually runs a model on your machine. B3IQ routes work to your node, but the runtime is what loads the weights and produces tokens. Picking the right one is mostly about your hardware, and the setup wizard does the picking for you in most cases.
You do not need to memorize any of this. The wizard detects your hardware, pre-selects a sensible default, and labels every option in plain language. This page exists so you understand what it chose and why.
Supported runtimes
| Runtime | Best for | Requirements | Install |
|---|---|---|---|
| llama.cpp | The recommended default. Squeezes the most out of a gaming or workstation GPU, and still runs on any computer. Downloads models on demand, so trying a new one needs no restart. | Works on any computer | One click |
| Ollama | The simplest start. Pick a model from a curated list and it just runs. | Works on any computer | One click |
| FastFlowLM | Experimental serving on an AMD Ryzen AI NPU. | Needs an AMD Ryzen AI XDNA 2 NPU with a current driver | You install it |
| LocalAI | Images, speech, embeddings, and rerank alongside chat, from one install. | Works on any computer | One click |
| Xinference | Video generation and the long tail of models. | Needs an NVIDIA graphics card | You install it |
| vLLM | High-throughput serving for an NVIDIA GPU server answering many requests at once. | Needs an NVIDIA graphics card | You install it |
If you are not sure, take the wizard's pre-selected default. llama.cpp is the default on every machine; Ollama is the easiest alternative if you would rather pick from a curated model list. Both run on any computer, and B3IQ installs and supervises both for you.
B3IQ cannot one-click FastFlowLM, Xinference, or vLLM, so their cards carry a copy-paste install command and a link to upstream docs instead of an Install button. Any OpenAI-compatible server you run yourself can also be pointed at as a runtime endpoint, and B3IQ will serve through it.
What the wizard shows you
Rather than asking you to reason about drivers and architectures, the wizard labels each runtime with one plain-language eligibility line based on the hardware it detected:
| Label | What it means |
|---|---|
| Works on any computer | Runs on CPU, and uses a GPU if you have one. Safe choice anywhere. |
| Needs an NVIDIA graphics card | Only offered when an NVIDIA GPU is present on this machine. |
| Needs an AMD Ryzen AI XDNA 2 NPU with a current driver | Only offered when the machine has that NPU and its driver passes the version floor. |
| Not supported on this machine | This runtime cannot run on your hardware, so the wizard will not let you start a doomed install. |
One-click install
The setup wizard can install a supported runtime for you. Each runtime in the catalog is pinned to a known build, so you get a tested version rather than whatever happens to be latest.
Before B3IQ installs a pinned runtime build it verifies the download against a known checksum, and stops and reports the failure rather than running an unverified binary. This applies to host-agent provisioning and to one-click binary runtime installs.
Where a one-click install genuinely isn't available for a runtime on your machine, the wizard shows a copy-paste command and a docs link instead of an Install button.
Picking a runtime
llama.cpp is the recommended default and works on any computer. B3IQ picks the accelerated build that matches the hardware it found, so a machine with a GPU gets a GPU build without you choosing one. Ollama is the easiest alternative if you would rather pick from a curated model list. Advanced operators can point B3IQ at a self-hosted vLLM or Xinference endpoint instead.
Hardware detection
B3IQ reads your machine's hardware so it can offer the right runtimes and the right accelerated builds. It detects:
| Hardware | How B3IQ uses it |
|---|---|
| NVIDIA GPU | Detected; unlocks the GPU-accelerated llama.cpp build and the NVIDIA-only runtime cards. |
| AMD GPU | Detected; served by the GPU-accelerated llama.cpp build. |
| Intel GPU | Detected; served by the same GPU-accelerated llama.cpp build. |
| AMD Ryzen AI NPU | Detected, with its driver version checked; recommends FastFlowLM, which you install yourself. |
B3IQ detects an AMD Ryzen AI XDNA 2 NPU and, when its driver is current, points you at FastFlowLM as the runtime to evaluate. It cannot install or supervise that runtime, and NPU serving is experimental. Every runtime B3IQ installs and watches serves on CPU or GPU.
Self-healing vs self-hosted
Not every runtime gets the same crash recovery. A runtime the host agent launched and is watching restarts automatically if it crashes. A runtime the agent could restart but is not currently watching carries a won't self-heal badge on the machine's Runtimes tab, so you learn about the gap before an outage rather than during one; use the Restart action on that card to bring it back. An endpoint B3IQ only proxies, such as a server you run yourself, gets no supervision at all: B3IQ shows it and routes to it, but restarting it is on you.
Listing models for a runtime
Once a runtime is installed, list the models actually available on your node rather than guessing at artifact names. The host agent exposes an OpenAI-compatible list:
bashcurl --fail --silent --show-error http://127.0.0.1:8831/v1/models
Use the ids returned here when configuring a model profile. The available set changes as you install or remove models, so always list rather than hard-code.
The same installed set is visible remotely, without SSHing into the box. The b3iq CLI can list it:
bashb3iq models list --machine <machine>b3iq runtimes list --machine <machine>
The machine's Models and Runtimes tabs in the account dashboard show the same statuses, too.