[Jobs] Decouple Job hardware from Spaces, auto-sync enums with Hub API by Wauplin · Pull Request #4266 · huggingface/huggingface_hub

Wauplin · 2026-05-26T13:59:01Z

Summary

Jobs hardware was previously tied to SpaceHardware — the same enum, same values, same code paths. As the two diverge (e.g. Jobs don't support ZeroGPU, Spaces don't have the same GPU catalog), this coupling causes friction in the UX with badly documented commands.

In addition, the hardcoded enum means every infra change requires a manual PR like #4259.

This PR decouples them and makes future hardware updates fully automated:

Own enum for Jobs — JobHardware(str, Enum) in _jobs_api.py, independent of SpaceHardware. The existing JobHardware dataclass (return type of list_jobs_hardware()) is renamed to JobHardwareInfo to free the name. ⚠️ this is a breaking change but perfectly fine IMO
Future-proof CLI — new SoftChoice(click.Choice) type that shows known values for docs/autocomplete but accepts any string, so older CLI versions don't reject new server-side flavors. Used for --flavor in jobs, spaces, and repos CLIs.
Automated sync — utils/check_hardware_flavors.py fetches the live hardware catalog from the Hub API and patches both enums. New flavors are appended; removed ones get a # legacy comment (kept for backward compat). A daily CI workflow (.github/workflows/update-hardware-flavors.yaml) runs the script and opens a PR via huggingface-hub-bot when something changes.
Docs — the hardcoded hardware list in docs/source/en/guides/jobs.md is replaced with a pointer to hf jobs hardware / list_jobs_hardware() so it stays current automatically.

$ hf jobs run --flavor future-gpu-x99 python:3.12 echo "works with unknown flavors"
Job started with ID: ...

>>> from huggingface_hub import JobHardware, JobHardwareInfo
>>> JobHardware.CPU_BASIC
<JobHardware.CPU_BASIC: 'cpu-basic'>
>>> from huggingface_hub import list_jobs_hardware
>>> list_jobs_hardware()[0]
JobHardwareInfo(name='cpu-basic', pretty_name='CPU Basic', ...)

$ python utils/check_hardware_flavors.py
Fetching hardware flavors from the Hub API...
  Spaces: 29 flavors
  Jobs:   28 flavors

✅ All good! Hardware enums are up to date.

🤖 Generated with Claude Code

Note

Medium Risk
Breaking rename of JobHardware dataclass to JobHardwareInfo and enum/catalog splits affect public API and CLI flavor validation; automated enum edits need human review before merge.

Overview
Jobs hardware is decoupled from Spaces: a dedicated JobHardware enum drives job APIs and types, while SpaceHardware is trimmed to the Spaces catalog. The old JobHardware dataclass from list_jobs_hardware() is renamed to JobHardwareInfo (breaking rename for consumers who imported the dataclass).

CLI and docs use new SoftChoice for --flavor / --hardware so help lists known values but unknown Hub flavors still pass through. Jobs guide hardware is an auto-generated table plus pointers to hf jobs hardware / list_jobs_hardware().

Automation: utils/check_hardware_flavors.py syncs both enums (and the jobs doc table) from the live Hub API—new flavors appended, removed ones kept with # legacy. A daily GitHub Actions workflow runs --update, make style, and opens a bot PR when anything drifts.

^{Reviewed by Cursor Bugbot for commit 26fc3b2. Bugbot is set up for automated code reviews on this repo. Configure here.}

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

bot-ci-comment · 2026-05-26T14:05:30Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Wauplin · 2026-05-26T14:32:37Z

for this one I reviewed the code quickly
I mainly focused on testing it locally with all sorts of situation (adding 1, removing 1, adding multiples while removing multiple, etc.). Always worked correctly so I think "it's fine"

hanouticelina

Thank you!

hanouticelina · 2026-05-27T13:02:13Z

    env: dict[str, Any] | None,
    secrets: dict[str, Any] | None,
-    flavor: SpaceHardware | None,
+    flavor: JobHardware | str | None,


str is redundant I think, JobHardware already inherits from str

Suggested change

flavor: JobHardware | str | None,

flavor: JobHardware | None,

Here we can do flavor="something" where "something" is not a JobHardware so type should reflect that. We could do flavor: str | None, instead since as you said JobHardware is a str but not the other way around. Though I still prefer to be explicit on JobHardware for self-documentation purposes

hanouticelina · 2026-05-27T13:04:45Z

        "arguments": [],
        "environment": env or {},
-        "flavor": flavor or SpaceHardware.CPU_BASIC,
+        "flavor": flavor or JobHardware.CPU_BASIC,


should we delegate the default to the server instead?

hanouticelina · 2026-05-27T13:10:43Z

+on:
+  workflow_dispatch:
+  schedule:
+    - cron: "0 4 * * *" # Every day at 4am


maybe once a week is enough no? no strong opinion

github-actions

Pinact found unpinned actions in this repo.

3 inline suggestion(s) below — click Apply suggestion on each.

julien-c · 2026-05-27T13:27:09Z

- CPU: `cpu-basic`, `cpu-upgrade`
- GPU: `t4-small`, `t4-medium`, `l4x1`, `l4x4`, `a10g-small`, `a10g-large`, `a10g-largex2`, `a10g-largex4`,`a100-large`
- TPU: `v5e-1x1`, `v5e-2x2`, `v5e-2x4`


only drawback is we're losing a bit of SEO/built-in info for agents (w/o invocation of a script)

maybe we can maintain a list in the same way we maintain the docstring? (in addition to documenting the python code to get it)

good idea
pushed 26fc3b2 to address it. Now the output of hf jobs hardware is listed in the docs with hardware + cost

Co-authored-by: célina <hanouticelina@gmail.com>

cursor

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 26fc3b2. Configure here.}

cursor · 2026-05-27T14:23:54Z

    CPU_UPGRADE = "cpu-upgrade"
-    CPU_PERFORMANCE = "cpu-performance"
-    CPU_XL = "cpu-xl"
-    SPRX8 = "sprx8"


SpaceHardware members deleted instead of marked legacy

Medium Severity

Twelve SpaceHardware enum members (including CPU_PERFORMANCE, CPU_XL, SPRX8, H200, H200X2, H200X4, H200X8, RTX_PRO_6000 variants, and INF2X6) are outright deleted. The companion script check_hardware_flavors.py is designed to keep removed flavors with a # legacy comment for backward compatibility, but the manual changes in this PR don't follow that pattern. Any downstream code referencing e.g. SpaceHardware.H200 will raise AttributeError.

^{Reviewed by Cursor Bugbot for commit 26fc3b2. Configure here.}

that's on purpose yes

hanouticelina

all good, thank you!

hanouticelina · 2026-05-27T14:35:32Z

 > For comprehensive guidance on running model training jobs with TRL on Hugging Face infrastructure, check out the [TRL Jobs Training documentation](https://huggingface.co/docs/trl/main/en/jobs_training). It covers fine-tuning recipes, hardware selection, and best practices for training models efficiently.

-Available `flavor` options:
+Here is the full list of available hardware to run Jobs:


huggingface-hub-bot · 2026-05-28T15:12:52Z

This PR has been shipped as part of the v1.17.0 release.

[Jobs] Decouple Job hardware from Spaces, auto-sync enums with Hub API

36d3b52

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

cursor Bot reviewed May 26, 2026

View reviewed changes

Comment thread utils/check_hardware_flavors.py Outdated

github-advanced-security AI found potential problems May 26, 2026

View reviewed changes

Comment thread .github/workflows/update-hardware-flavors.yaml Fixed

Wauplin marked this pull request as draft May 26, 2026 14:13

Wauplin and others added 6 commits May 26, 2026 16:17

correct list of hardware

c027dd0

cleanupg

fab433e

Add minimal workflow permissions

d2007c5

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

beter

539675d

some other reveiwers

5503205

nit

89d33ae

Wauplin commented May 26, 2026

View reviewed changes

Wauplin requested a review from hanouticelina May 26, 2026 14:32

Wauplin marked this pull request as ready for review May 26, 2026 14:33

Wauplin added 2 commits May 26, 2026 16:33

Merge branch 'main' into decouple-job-hardware-from-spaces

c3656a5

arg

afc2574

This comment was marked as low quality.

Sign in to view

cursor Bot reviewed May 26, 2026

View reviewed changes

Comment thread src/huggingface_hub/hf_api.py

hanouticelina reviewed May 27, 2026

View reviewed changes

Merge branch 'main' into decouple-job-hardware-from-spaces

0291e8e

github-actions Bot reviewed May 27, 2026

View reviewed changes

Comment thread .github/workflows/update-hardware-flavors.yaml Outdated

Comment thread .github/workflows/update-hardware-flavors.yaml Outdated

Comment thread .github/workflows/update-hardware-flavors.yaml Outdated

cursor Bot reviewed May 27, 2026

View reviewed changes

Comment thread utils/check_hardware_flavors.py

pinact

64739dc

julien-c reviewed May 27, 2026

View reviewed changes

Update utils/check_hardware_flavors.py

2b80baa

Co-authored-by: célina <hanouticelina@gmail.com>

cursor Bot reviewed May 27, 2026

View reviewed changes

Comment thread utils/check_hardware_flavors.py

fix imports

440fc4a

cursor Bot reviewed May 27, 2026

View reviewed changes

Comment thread utils/check_hardware_flavors.py

auto updated hardware list in docs

26fc3b2

Wauplin requested a review from hanouticelina May 27, 2026 14:16

Wauplin requested a review from julien-c May 27, 2026 14:18

cursor Bot reviewed May 27, 2026

View reviewed changes

hanouticelina approved these changes May 27, 2026

View reviewed changes

Wauplin merged commit ffc07b6 into main May 27, 2026
26 checks passed

Wauplin deleted the decouple-job-hardware-from-spaces branch May 27, 2026 14:48

	flavor: JobHardware \| str \| None,
	flavor: JobHardware \| None,

Conversation

Wauplin commented May 26, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

bot-ci-comment Bot commented May 26, 2026

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment was marked as low quality.

Uh oh!

Uh oh!

hanouticelina left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Wauplin May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot May 27, 2026

Choose a reason for hiding this comment

SpaceHardware members deleted instead of marked legacy

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hanouticelina left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

huggingface-hub-bot Bot commented May 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Wauplin commented May 26, 2026 •

edited by cursor Bot

Loading

Wauplin May 27, 2026 •

edited

Loading