Run a vLLM Server on HF Jobs in One Command
02:00 · June 26, 2026 · Hugging Face Blog
Summary
Hugging Face Jobs provides a lightweight way to run a private vLLM server that exposes the OpenAI chat completions API. A single hf jobs run command pulls the official vllm/vllm-openai image, selects GPU hardware with the --flavor flag, and routes port 8000 through the platform’s public proxy, returning a job-specific URL such as <job_id>--8000.hf.jobs. Every request must include a Hugging Face token with read access to the job’s namespace, so the endpoint remains gated even though it is reachable from the public internet.
Once the container reports “Application startup complete,” the server accepts both curl requests and the standard OpenAI Python client pointed at the jobs URL. The same pattern scales to larger models by choosing multi-GPU flavors and adding --tensor-parallel-size together with memory-tuning flags such as --max-model-len and --max-num-seqs; these limits prevent out-of-memory errors when models like the 122 B Qwen3.5 mixture-of-experts exceed default context and batch settings.
Additional workflows are supported without changing the core deployment. A short Gradio script can be run locally to present a chat interface that streams reasoning traces. Launching the job with --ssh allows an interactive shell inside the container for nvidia-smi checks or direct inspection. The resulting endpoint can also serve as a backend for the Pi agent harness once tool-calling flags are enabled in vLLM.
Jobs are billed per second and terminate automatically after the requested timeout, making them suitable for short-lived experiments, evaluations, or batch generation. For long-lived services that require public or fine-grained access control and scale-to-zero billing, Inference Endpoints remain the managed alternative.
Why it matters
Directly actionable for ML engineers needing quick, production-adjacent model serving setups with explicit handling of VRAM constraints, distributed GPU configs, and pay-per-second costs; relevant for Dutch teams using HF tooling.
