Global – – October 12, 2026 -- Fuzzball v4.3 adds a vLLM catalog entry and eleven validated model presets that serve a large language model from a pool of replicas on NVIDIA or AMD GPUs, behind an OpenAI-compatible endpoint.
Eleven presets come with GPU count, context size and vLLM arguments already set
The generic vLLM entry serves a HuggingFace model from a single catalog command. The eleven presets cover GPT-OSS 20B and 120B, Ministral 3 8B and 14B, Qwen3-Coder 30B and Next 80B, Nemotron 3 Nano and Super, Mistral Medium 3.5, Llama 4 Scout and Gemma 4 31B. Each has its GPU fit and serving arguments validated per model.
Users can accept the preset values or override them. In one example, a persistent volume keeps the downloaded checkpoint between starts, and MaxReplicas is raised to eight so the pool can grow under load.
Built-in LiteLLM gateway gives clients and agents a single endpoint
Each model ships with a LiteLLM Model Gateway in front of the workflow, so the workflow scales with demand behind one endpoint. Operators can disable the per-workflow gateway and run one shared gateway that discovers every model-serving workflow a user can access.
The catalog also includes two coding agents, OpenCode and hermes-agent, which run in the cluster and detect model gateways automatically. A separate rag entry runs a document retrieval service built on haiku.rag. It ingests PDF, DOCX, PPTX, HTML and plain text, and exposes hybrid vector and full-text search as MCP tools.
Autoscaled pool endpoints can idle at zero replicas and wake on authenticated requests
Service endpoints now work against autoscaled pools. A pool keeps one URL for the life of the workflow and routes each request to a ready replica. New replicas join once they pass readiness, and departing replicas leave when their drain period starts.
A pool with replicas.min set to 0 keeps its endpoint while idle. A request that arrives gets a 503 with Retry-After: 120 and triggers one replica start, even during a burst. Only authenticated requests wake a pool, so a public-scope endpoint never will, which prevents outsiders from switching on GPUs. Every wake appears in the workflow events.
A per-replica option gives each live replica its own endpoint for teams that run their own load balancing. Endpoints can carry key-value annotations, which is how the LiteLLM gateway finds the pools it fronts. A service can also combine multinode with autoscaler replicas, so each replica is a gang-scheduled group of ranks serving from rank 0.
Persistent node-local volumes arrive, but cannot support autoscaled serving
Earlier releases supported only ephemeral node-local volumes. Version 4.3 adds persistent local volumes and improves ephemeral handling. Each local volume has exactly one holding node, set at creation or via the --node flag, and every stage that mounts it is placed there. Placement is capacity-aware: nodes without enough space are skipped, and manually targeting a node with insufficient space fails.
Local volumes do not support concurrent multi-node stages. MPI jobs, concurrent task arrays and multi-node scale-up services need a shared volume. Autoscaled serving and node-local volumes are therefore mutually exclusive; a persistent volume for the vLLM entry needs a shared provisioner, and MaxReplicas defaults to 4.
Three upgrade changes affect catalog repositories, entry IDs and ingress TLS
Catalog repository URIs are now restricted by default to HTTPS and publicly routable hosts. Non-compliant repositories stop syncing and must be re-added over HTTPS. Operator-managed deployments can relax the rule with spec.fuzzball.workflowCatalog.restrictions.allowPrivateAddresses.
Catalog entry IDs change on the first sync after upgrade because they now derive from the catalog source as well as the entry's own ID. Saved links and scripts that reference entries by UUID need updating. Ingress routes on the kong and nginx classes now require TLS and redirect plaintext callers with a 308; other classes are unchanged.
Version 4.3.0 includes all fixes released through v4.2.3.