Models

Serverless › Models is where you decide what to run:

• Official — verified, 1-click catalog curated by Meshive.
• Custom — any Hugging Face repo you register yourself.

Deploying either is the same Serving flow.

Official catalog

• Each card shows: display name, modality, context length, quantization, parameter count (B), and the measured VRAM requirement (what determines which GPUs can serve it).
• No price on the card, by design — cost only exists as the hourly GPU price you see and cap in the deploy panel.

Note

A catalog model can be scheduled for retirement. Deployments based on one show a warning — see Model retiring.

Registering a custom model

The only required input is a Hugging Face repo (plus a token if the repo is gated or private). Everything else — modality, framework, file size, VRAM estimate — is derived from HF metadata, and every derived value can be overridden:

Field	Required?	Notes
Hugging Face repo	Yes	The one mandatory input
Model name	No	Defaults to the repo’s last segment
Min VRAM (GB)	No¹	Overrides the automatic estimate
Context length, Quantization, Parameter count	No	Derived from HF when possible
vLLM args (JSON)	No	LLMs only, API only — see allowed engine args
Modality / input modalities	Usually derived	See below
Supported sizes	Image models	e.g. 1024x1024 — the API/Playground validate against this list
Supported resolutions	Video models	e.g. 720x1280
Tensor parallel / USP size, Safety filter	No	Advanced

¹ Diffusion models with no VRAM info at all are rejected (see error table) — a deployment that can’t be placed helps no one.

• Fine-tuned repo without a parameter count? The VRAM estimate falls back to the base model’s config.
• HF lookup failed? Registration still proceeds with what you entered — HF is best-effort. Modality-required fields are always validated.
• Same repo registered twice? You get the existing registration back, not a duplicate. If you pass a token, it replaces the stored one.

Gated repos

A gated or private repo needs a token from a Hugging Face account that has access — accept the license on the model page first. Your deployment downloads the weights with this token. Public repos need no token and are downloaded anonymously, which Hugging Face rate-limits more tightly, so a token can still help with large models.

• One-off — paste an HF token directly into the form.
• Saved — store tokens once in the workspace token list, pick by label. Deleting a saved token later doesn’t affect already-registered models.

Allowed engine args

Custom models accept a short list of engine flags, because they become part of the replica’s start command:

Engine	Flags
vLLM	max-model-len, max-num-seqs, max-num-batched-tokens, tool-call-parser, reasoning-parser, tokenizer-mode, enable-auto-tool-choice, async-scheduling, dtype, kv-cache-dtype, chat-template-content-format
SGLang	context-length, tool-call-parser, reasoning-parser

Anything else, including trust-remote-code, is refused at registration.

Image & video specifics

Custom image & video registration is temporarily paused

Registering your own image or video (diffusion) models is temporarily unavailable while the generation engine migrates to ComfyUI. In the meantime:

• Official image and video models in the catalog deploy normally.
• Existing custom deployments keep serving — nothing already running is affected.
• To run your own workflows today, deploy ComfyUI on a GPU pod — see Quick Deploy - ComfyUI.

Custom LLM and embedding registration is unaffected. Custom diffusion registration is on our roadmap to return.

For official models (and for custom registrations, once they return):

• Input modalities: ["text"] → LLM / T2I / T2V · ["text","image"] → I2I (image edit) / I2V.
• Image models must list supported sizes; video models supported resolutions.
• Official video models pin a native (frames, fps) — mismatched requests are rejected with a 422 naming the supported values.
• Generation endpoints expose what each deployment actually accepts via supported_parameters on GET /v1/models.

GGUF and quantized models

• GGUF repos register like any other LLM. The platform picks the best single .gguf variant in the repo automatically (sharded variants are grouped; a multimodal projector file is included when present), and derives context length and architecture from the GGUF metadata.
• Quantization is detected from the repo and determines which GPUs can serve the model:

Quantization	GPU requirement
FP16 / BF16 / FP32	Any fleet GPU
Weight-only (AWQ, GPTQ, GGUF, INT8/4)	Any fleet GPU
FP8	FP8-native GPU (Ada / Hopper / Blackwell)
FP4 (NVFP4 / MXFP4)	FP4-native GPU (Blackwell)

GPUs that can’t run the model’s quantization are excluded from the deploy panel’s candidates, with the reason shown.

Registration errors

Situation	Response	What it means
Unsupported modality (HF pipeline_tag)	400 Unsupported Model	The repo’s task type isn’t supported; supported modalities listed
Modality can’t be determined	422 Modality Required	Pick the modality yourself in the form
Image model without supported sizes (etc.)	422 Missing Required Field	The message names the exact field
LLM incompatible with the serving engine (vLLM)	400	The architecture isn’t servable; the message explains the check
Diffusion model without any VRAM info	400 VRAM info required	Provide Min VRAM (the model card usually states it)
Gated or private repo without a token	400 HF Token Required	Add a token from an account that has access
Engine flag outside the allowed list	400 Invalid runtime args	The message lists the flags you can use
Repo id isn’t owner/model	422 Invalid Repository	Check the id; full Hugging Face URLs are accepted

Workspace isolation

• Registrations are owned by your workspace — the same HF repo in two workspaces is two independent registrations. Platform-wide sharing is what the Official catalog is for.
• Delete a custom model from the Models page once no active deployment uses it (stop the deployment first).

Playground

Six modes — Chat / Completion / Image / Image Edit / Video / Image-to-Video — auto-selected by the model’s input modalities.

• No key juggling — entering issues a temporary session key (30-minute lifetime, auto-renewed); your real keys are untouched.
• Real constraints — size/resolution pickers only offer what the model supports, the same validation the API enforces.
• Same path as the API — identical gateway and deployment; the panel generates the matching curl/Python snippet for your endpoint.

API keys

The API Keys tab manages the workspace’s inference keys (workspace admins only):

Action	Behavior
Create	Instant. The plaintext mk-... is shown once — only a SHA-256 hash is stored, it can never be shown again
Rename	Label only; the key value is untouched
Revoke	Invalidates the key immediately
Regenerate	Revokes the old value, issues a fresh plaintext in place — the recommended path for a lost/leaked key

• Per-key limits: RPM and TPD, shown in the list. Unset = platform defaults, currently 60 RPM / 1,000,000 tokens-day.
• Exceeding either returns 429 with Retry-After.
• Limits apply to text endpoints; image/video are governed by deployment capacity instead (details).

Caution

Playground session keys expire after 30 minutes — don’t copy them into scripts. Create a real key.