> ## Documentation Index
> Fetch the complete documentation index at: https://docs.b3iq.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Mixture of Experts

Fan one prompt out to several models on your own machines and get one combined OpenAI-compatible answer, with strategies, quorum, and a metadata-only trace.

Mixture of Experts (MoE) lets one `POST /v1/api/chat/completions` request run a
prompt through **several models in parallel** and return **one combined
OpenAI-compatible answer**. You attach an optional `b3iq.mixture` object to the
request you already send; a plain request with a single `model` and no
`b3iq.mixture` is completely unchanged.

The `b3iq` block is consumed by B3IQ routing and stripped before any node sees
the request, so nodes serve each expert exactly like a normal single-model call.

## How it works

When a request carries `b3iq.mixture`, B3IQ plans one sub-call per expert (plus
one for the aggregator, if the strategy uses one) and dispatches the experts in
parallel. Each sub-call is routed **independently** through the same selection
policy as a single-model request. Supply is picked by health, readiness, and
policy, and privacy class and your key's model policy apply to every expert and
the aggregator alike.

Every sub-call stays inside the machines on your account, the same
own-nodes-strict scope any gateway key already dispatches to. Ownership is
verified server-side on every sub-call, so the fan-out can never leave your
fleet. Each expert model (and the aggregator model, for `synthesize`) must be
installed and serving on at least one machine you own, and the request must be
**unpinned**: a mixture combined with a node pin returns a 400, because a
mixture always spreads across the fleet by policy. A failed mixture is never
silently retried elsewhere. There is no fallback outside the machines you
own, and a quorum failure is returned as-is. Mixtures are not available on the
encrypted [private-jobs](/developers/private-jobs) path or direct node
dispatch.

Once the experts return, a quorum check decides whether the mixture succeeded,
and the strategy combines the surviving answers into the final response. The
response is a standard `chat.completion` whose `usage` aggregates every
sub-call, plus a metadata-only `b3iq.mixture` trace.

## Strategies

| Strategy | How it combines | Aggregator |
| --- | --- | --- |
| ` synthesize ` | An aggregator model reads every expert answer and writes the final response. If the aggregator fails, the mixture degrades to the top-weighted expert answer (degraded: true) instead of erroring. | Required |
| ` vote ` | Weighted plurality (self-consistency): matching expert answers pool their weights and the winning answer is returned verbatim. | Not used |
| ` weighted ` | The highest-weight successful expert answer is returned verbatim: a quality-ordered failover across models. | Not used |

A `debate` strategy is reserved but not yet supported. Sending it returns a
  400 today.

## Configuration

All fields live under `b3iq.mixture` on the chat-completion request body.

| Field | Required | Description |
| --- | --- | --- |
| ` strategy ` | Yes | One of synthesize, vote, weighted |
| ` experts[] ` | Yes | Non-empty array of expert sub-calls. Up to 4 experts by default (the cap is operator-configurable). |
| ` experts[].model ` | Yes | Model id for this expert, from GET /v1/api/models (with your key, the list is already scoped to what your machines serve) |
| ` experts[].weight ` | No | Positive number, default 1. Drives vote pooling, weighted selection, and degraded fallback order. |
| ` aggregator.model ` | For synthesize | Model that writes the final answer. Ignored by vote and weighted. |
| ` min_experts ` | No | Quorum: minimum experts that must succeed. Defaults to half the experts, rounded up, for vote; otherwise 1. |
| ` max_price_wei ` | No | Deprecated: inference is free, so the cap is a no-op. Accepted for back-compat and ignored. |

**Mixtures never pin nodes**

An expert entry carries a model only. Any `node_id`, `route`, or hostname on
  an expert is rejected with a 400. A single-model hosted request may still pin
  a machine you own (`?node_id=` or `b3iq.node_id`), but combining a node pin
  with `b3iq.mixture` on the same request is rejected with a 400. Every
  sub-call in a mixture is placed by B3IQ route policy, never pinned.

## Send a mixture

A `synthesize` mixture: two experts answer in parallel, then an aggregator
model writes the final response. Model ids come from `GET /v1/api/models`.
Called with your key, that list is already scoped to the models the machines
on your account serve right now.

```bash curl
: "${B3IQ_GATEWAY_BASE_URL:?set B3IQ_GATEWAY_BASE_URL}"
: "${B3IQ_GATEWAY_KEY_FILE:?set B3IQ_GATEWAY_KEY_FILE}"

cfg="$(mktemp)"
chmod 600 "$cfg"
trap 'rm -f "$cfg"' EXIT
{
  printf 'url = "%s/chat/completions"\n' "${B3IQ_GATEWAY_BASE_URL%/}"
  printf 'request = "POST"\n'
  printf 'header = "Authorization: Bearer %s"\n' "$(sed -n '1p' "$B3IQ_GATEWAY_KEY_FILE")"
  printf 'header = "Content-Type: application/json"\n'
} > "$cfg"

curl --fail --silent --show-error --config "$cfg" --data-binary @- <<'JSON'
{
  "model": "qwen3-8b",
  "messages": [{ "role": "user", "content": "Prove that sqrt(2) is irrational." }],
  "b3iq": {
    "mixture": {
      "strategy": "synthesize",
      "experts": [
        { "model": "qwen3-8b", "weight": 2 },
        { "model": "llama3.1-8b" }
      ],
      "aggregator": { "model": "qwen3-8b" }
    }
  }
}
JSON
```

```javascript fetch.js
import { readFileSync } from "node:fs";

const baseURL = process.env.B3IQ_GATEWAY_BASE_URL;
const token = readFileSync(process.env.B3IQ_GATEWAY_KEY_FILE, "utf8").trim();

const response = await fetch(`${baseURL}/chat/completions`, {
  method: "POST",
  headers: {
    authorization: `Bearer ${token}`,
    "content-type": "application/json"
  },
  body: JSON.stringify({
    model: "qwen3-8b",
    messages: [{ role: "user", content: "Prove that sqrt(2) is irrational." }],
    b3iq: {
      mixture: {
        strategy: "synthesize",
        experts: [
          { model: "qwen3-8b", weight: 2 },
          { model: "llama3.1-8b" }
        ],
        aggregator: { model: "qwen3-8b" }
      }
    }
  })
});

console.log(await response.json());
```

A `vote` mixture needs no aggregator. Matching answers pool their weights and
the winner is returned verbatim:

```json
{
  "model": "qwen3-8b",
  "messages": [{ "role": "user", "content": "What is 17 * 24? Answer with the number only." }],
  "b3iq": {
    "mixture": {
      "strategy": "vote",
      "experts": [
        { "model": "qwen3-8b" },
        { "model": "llama3.1-8b" },
        { "model": "mistral-small" }
      ],
      "min_experts": 2
    }
  }
}
```

## Response and the mixture trace

The response is a standard `chat.completion`. `usage` aggregates tokens across
**every** sub-call (experts plus aggregator), and `b3iq.mixture` carries a
trace of what ran. For `synthesize`, the response `model` is the aggregator
model; otherwise it is your advisory top-level `model`.

```json
{
  "id": "chatcmpl-…",
  "object": "chat.completion",
  "model": "qwen3-8b",
  "choices": [
    { "index": 0, "message": { "role": "assistant", "content": "…" }, "finish_reason": "stop" }
  ],
  "usage": { "prompt_tokens": 96, "completion_tokens": 210, "total_tokens": 306 },
  "b3iq": {
    "request_id": "…",
    "mixture": {
      "strategy": "synthesize",
      "degraded": false,
      "experts": [
        { "role": "expert", "model": "qwen3-8b", "status": "ok", "node_id": "b3iq_…", "usage": { "total_tokens": 84 } },
        { "role": "expert", "model": "llama3.1-8b", "status": "ok", "node_id": "b3iq_…", "usage": { "total_tokens": 91 } },
        { "role": "aggregator", "model": "qwen3-8b", "status": "ok", "node_id": "b3iq_…", "usage": { "total_tokens": 131 } }
      ]
    }
  }
}
```

The trace is **metadata only**: strategy, per-sub-call role, model, node id,
status, and token counts. It never contains expert or aggregator text, and
intermediate answers are never persisted; they exist only in memory while the
mixture combines.

`degraded: true` means the mixture completed with a partial failure: one or
more sub-calls did not succeed, or the synthesize aggregator failed and the
answer fell back to the top-weighted surviving expert. The request still
returns 200. Check the trace's per-sub-call `status` fields for what happened.

## Streaming

`stream: true` is supported. The fan-out is fully buffered and settled first,
then the final combined answer is delivered as SSE, with `usage` and the
`b3iq.mixture` trace on the terminal chunk. Because every expert must finish
before the stream opens, time-to-first-byte for a streamed mixture equals the
full fan-out latency. A mixture is inherently slower than a single-model call.

## Cost and budgets

A mixture is **free**, like every request served by the machines on your
account: no per-sub-call charge and no aggregate quote to afford. What is
budgeted is capacity: the estimated **token total across every sub-call** is
checked against the mixture token budget up front, each sub-call is metered
like a normal request, and everything counts toward your account's daily token
quota (default uncapped). See
[Usage & Quotas](/developers/usage-and-quotas).

## Errors and limits

Mixture rejections use the same envelopes as other hosted errors. See
[Errors](/developers/errors). The common cases:

| Status | When |
| --- | --- |
| ` 400 ` | Malformed mixture config (missing strategy or experts, bad weight or quorum), a pinned expert, a mixture combined with a node pin, an unsupported path, or the estimated token total exceeding the mixture token budget |
| ` 403 ` | An expert or aggregator model is outside your key's model policy |
| ` 503 ` | Quorum not met: fewer than min_experts experts produced an answer |

Deployment-configurable limits apply per request: expert count (default 4), an
aggregate token budget across all sub-calls, and an aggregate wall-clock
deadline for the whole mixture. Keep `max_tokens` and the expert count modest.
Every expert multiplies the token and time budget the request consumes.

## Related

**[Hosted gateway](/developers/hosted-gateway)**

The endpoint mixtures ride on: keys, headers, streaming, and routing gates.
  **[Run a private fleet](/guides/private-fleet)**

Own-nodes-strict routing and the machines a mixture can fan out across.
  **[Usage & quotas](/developers/usage-and-quotas)**

Free inference, per-key metering, and the token quota mixtures count toward.
  **[Errors](/developers/errors)**

The stable error envelopes mixture rejections use.