Run AI workloads on any cloud, store on Hugging Face: zero-egress storage with SkyPilot
02:00 · July 7, 2026 · Hugging Face Blog

Summary
Hugging Face and SkyPilot have integrated Hugging Face Storage as a native backend in SkyPilot, allowing teams to mount repositories or Buckets directly into jobs via the hf:// scheme. The mount relies on the hf-mount FUSE driver, which presents remote content as a local filesystem path. Reads occur on demand: when application code requests bytes, only the needed portions are fetched from the Xet backend, and an on-disk cache retains them for subsequent accesses. This approach eliminates the need for a full upfront download, so training or inference processes can begin immediately rather than waiting for complete dataset or checkpoint copies.
The integration addresses a common constraint in multi-cloud AI operations. GPU capacity is often distributed across providers, yet object storage remains regional and egress fees quickly accumulate when data must cross vendor boundaries. Because Hugging Face charges no egress or CDN fees for reads, the same bucket remains accessible from any SkyPilot target—whether AWS, GCP, Azure, Lambda, Nebius, or on-prem Kubernetes—using only the existing HF_TOKEN. Writes still incur the usual egress charges of the compute cloud, but the dominant traffic pattern in most workloads (repeated dataset streaming and model loading) occurs without cost.
Xet-backed storage further reduces expenses through content-defined chunking that splits files into roughly 64 KB segments and stores each unique chunk once. Subsequent checkpoints or model variants therefore transmit and retain only the modified portions. SkyPilot jobs can use either MOUNT or MOUNT_CACHED semantics; for Hugging Face both modes maintain the local cache. The feature supports the full model lifecycle—reading base artifacts, writing training outputs, publishing final models—without requiring separate storage accounts or per-cloud credentials.
Why it matters
This article provides ML Engineers with a practical, hands-on solution to a major MLOps pain point: high egress costs in multi-cloud GPU environments. It offers actionable code snippets and benchmarks that AI teams can immediately implement to optimize their cloud compute budgets and avoid vendor lock-in.








