Use CDI for GPU injection for AMD devices for --gpus by shiv-tyagi · Pull Request #52048 · moby/moby

shiv-tyagi · 2026-02-16T09:07:07Z

This PR enhances the functionality of the --gpus option for AMD GPUs by utilizing CDI (Container Device Interface) specs for device injection when available. It falls back to the existing vendor runtime-based injection if AMD CDI specs are not detected on the machine.

Related PR: containerd/containerd#12839 (Similar implementation for containerd/ctr)

- What I did
Added support for CDI-based GPU device injection through --gpus option for AMD devices.

- How I did it
Created a similar composite device driver like NVIDIA's which discovers if AMD's CDI specs are there on the system during registration and registers itself with appropriate updaters to handle the device request.

- How to verify it

Built the binaries using make binary.
Started the newly built dockerd instance via ./bundles/binary/dockerd.
Used amd-ctk cdi generate to install the CDI specs on the host.
Test Injection: Ran docker run --rm --gpus all rocm/rocm-terminal rocm-smi.
- Verified that CDI-based GPU injection works as expected.
Test (Fallback Path):
- Deleted the CDI specs and restarted the dockerd process.
- Retried using the runtime flag: docker run --rm --runtime="amd" --gpus all rocm/rocm-terminal rocm-smi.
- Verified that the vendor runtime still works as a fallback when CDI specs are absent.
- Checked the environment variable inside the container has AMD_VISIBLE_DEVICES set when CDI specs are not there to verify that the fallback is working correctly.

I have also added unit tests for vendor discovery function.

- Human readable description for the release notes

The `--gpus` option now uses CDI-based injection for AMD GPUs.

- A picture of a cute animal (not mandatory but encouraged)

elezar · 2026-02-16T09:39:48Z

daemon/devices_amd_linux.go

+
+	// Try to detect AMD GPU vendor via CDI cache if cdiCache is available
+	if cdiCache != nil {
+		vendor, err := discoverGPUVendorFromCDI(cdiCache)


One thing about this approach ... this only checks whether the cache includes AMD cdi devices at the point where the daemon is reloaded. In contrast to the other projects where we have added this functionlity, the cache here is started with AutoRefresh enabled meaning that the CDI spec directories are watched for changes to ensure that specs for new devices are detected.

With that in mind, the drivers that one wants to register would have to determine the vendor from the cache for every --gpus request and not only once at startup.

Yes, makes sense.

I have updated the logic to always discover vendor from CDI registry on the fly since the registry is auto refreshed. I have also verified it is working as expected by deleting the CDI files while the daemon is running and verifying that the vendor discovery fails in that case.

Thanks for the suggestion.

daemon/devices_amd_linux.go

daemon/devices.go

shiv-tyagi · 2026-03-04T03:44:12Z

Hey @vvoland I have tried answering your comments. Can you please review again and let me know if you still see any issues? I will be more than happy to improve. Thanks!

vvoland · 2026-03-04T10:12:35Z

Code looks good, but the CI is failing on linting issues

vvoland · 2026-03-04T11:01:57Z

Failure on Windows:

0.750 cmd/dockerd/main_windows.go
0.773 Building static /tmp/bundles/binary-daemon/dockerd.exe (windows/amd64)...
79.23 # github.com/moby/moby/v2/daemon/command
79.23 daemon/command/daemon.go:288:9: undefined: daemon.RegisterGPUDeviceDrivers

shiv-tyagi · 2026-03-04T11:06:22Z

Looks like this is because RegisterGPUDeviceDrivers() is defined only inside devices_nvidia_linux.go?

Would you recommend to create a corresponding _windows.go (or _unsupported.go) file and add a dummy implementation there?

shiv-tyagi · 2026-03-04T11:15:57Z

@vvoland I have added a no-op implementation of that function in devices_unsupported.go. make win runs fine for me locally now. Please check and let me know if that is not the correct way of doing it. Thanks.

vvoland · 2026-03-04T12:24:57Z

daemon/devices_unsupported.go

+import "tags.cncf.io/container-device-interface/pkg/cdi"
+
+// RegisterGPUDeviceDrivers is a no-op on non-Linux platforms.
+func RegisterGPUDeviceDrivers(_ *cdi.Cache) {}


nit: It would be a bit cleaner to have the devices_linux.go file with the RegisterGPUDeviceDrivers implementation so we avoid the empty stub.

I have created device_linux.go which overrides RegisterGPUDeviceDrivers with registerGPUDeviceDrivers which is linux specific implementation. This avoids the empty stub.

Oh, sorry I missed that RegisterGPUDeviceDrivers is actually called from a non "linux" file, so your previous was actually correct 🤦🏻

I can fix that in a follow up PR, so we don't need to block this one.

Sure. I was also a bit confused... I assumed you are trying to avoid an _unsupported.go file and looking for a _linux.go somehow.

I'm going to wait for Evan to give a look too so I pushed 13a8626 to address this.

Feel free to squash it with your PR if you prefer!

daemon/devices_amd_linux.go

Signed-off-by: Shiv Tyagi <Shiv.Tyagi@amd.com>

shiv-tyagi · 2026-03-04T15:57:35Z

I don't think the test failure is related. Please let me know if it is.

Thank you so much @vvoland for all your time and efforts in reviewing this.

vvoland

Thanks, LGTM!

vvoland · 2026-03-04T16:04:03Z

@elezar want to give this one a last look?

elezar · 2026-03-04T16:49:40Z

@elezar want to give this one a last look?

I just finished off for the day. I'll have a look tomorrow first thing!

Change GPU device driver registration from init-time function assignment to direct function exports. This removes the init() function and global function variable pattern in favor of a more explicit approach. Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

vvoland · 2026-03-05T12:06:48Z

@elezar I'll get this in to unblock 29.3.0. Feel free to still give this one a look though!

elezar · 2026-03-05T12:41:24Z

@elezar I'll get this in to unblock 29.3.0. Feel free to still give this one a look though!

Thanks. I did have some thoughts, but was working on formulating them more clearly. I'll either comment here, or create a PR with some minor changes proposed.

elezar · 2026-03-05T13:22:36Z

daemon/command/daemon.go

 		cdiCache = daemon.RegisterCDIDriver(cli.Config.CDISpecDirs...)
 	}

+	daemon.RegisterGPUDeviceDrivers(cdiCache)


As a general note, I think doing this explicitly instead of relying on init makes things cleaner.

Hey @elezar! This was needed to ensure we use the same cdiCache which is initialized above before this call.

Sorry I am not following. We are not using init() here. Can you explain more?

Yes. I was just saying that I think this change is for the better. Thanks for this.

elezar · 2026-03-05T13:27:56Z

daemon/devices_amd_linux.go

+			return fmt.Errorf("failed to discover GPU vendor from CDI: %w", err)
+		}
+
+		if vendor != "amd.com" {


Thinking as one f the maintainers of the CDI tooling, adding a ListKinds function to a cdi.Cache will allow us to be more accurate with this check. There may be OTHER kinds (e.g. amd.com/not.gpu) that would cause the vendor detection to pass, but the amd.com/gpu kind could still be unknown.

Since we fall throug to the runtime-based update in this case anyway, I don't think it's too critical to address though.

I agree. We can add that function and use that is all CLIs for --gpus.

elezar · 2026-03-05T14:30:31Z

I was a bit delayed in getting to this. I have no blocking comments, but did create #52145 as a follow-up.

github-actions bot added the area/daemon Core Engine label Feb 16, 2026

elezar reviewed Feb 16, 2026

View reviewed changes

vvoland added kind/enhancement Enhancements are not bugs or new features but can improve usability or performance. area/cdi labels Feb 16, 2026

vvoland added this to the 29.3.0 milestone Feb 16, 2026

shiv-tyagi force-pushed the vendor-detection branch from f0f8245 to faba468 Compare February 16, 2026 13:12

github-actions bot added the area/testing label Feb 16, 2026

shiv-tyagi mentioned this pull request Feb 16, 2026

Detect vendor in cdi specs to generate deviceIDs for --gpus containerd/containerd#12839

Merged

vvoland reviewed Feb 26, 2026

View reviewed changes

daemon/devices_amd_linux.go Outdated Show resolved Hide resolved

vvoland reviewed Feb 26, 2026

View reviewed changes

daemon/devices_amd_linux.go Show resolved Hide resolved

vvoland reviewed Feb 26, 2026

View reviewed changes

daemon/devices.go Outdated Show resolved Hide resolved

shiv-tyagi force-pushed the vendor-detection branch from faba468 to 5ec493c Compare February 27, 2026 09:27

shiv-tyagi requested a review from vvoland March 4, 2026 03:42

shiv-tyagi force-pushed the vendor-detection branch from 5ec493c to 735a141 Compare March 4, 2026 10:55

vvoland added the impact/changelog label Mar 4, 2026

shiv-tyagi force-pushed the vendor-detection branch from 735a141 to bc4c3f2 Compare March 4, 2026 11:14

vvoland reviewed Mar 4, 2026

View reviewed changes

Use CDI for GPU injection for AMD devices for --gpus

561a5a9

Signed-off-by: Shiv Tyagi <Shiv.Tyagi@amd.com>

shiv-tyagi force-pushed the vendor-detection branch from bc4c3f2 to 561a5a9 Compare March 4, 2026 13:15

shiv-tyagi requested a review from vvoland March 4, 2026 13:25

vvoland removed the area/testing label Mar 4, 2026

vvoland approved these changes Mar 4, 2026

View reviewed changes

github-actions bot added the area/testing label Mar 4, 2026

vvoland merged commit 4c19a01 into moby:master Mar 5, 2026
252 of 253 checks passed

elezar reviewed Mar 5, 2026

View reviewed changes

elezar mentioned this pull request Mar 5, 2026

Update NVIDIA --gpu requests using CDI to match AMD logic #52145

Open

Conversation

shiv-tyagi commented Feb 16, 2026 • edited by vvoland Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shiv-tyagi commented Mar 4, 2026

Uh oh!

vvoland commented Mar 4, 2026

Uh oh!

vvoland commented Mar 4, 2026

Uh oh!

shiv-tyagi commented Mar 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shiv-tyagi commented Mar 4, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shiv-tyagi Mar 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

shiv-tyagi commented Mar 4, 2026

Uh oh!

vvoland left a comment

Choose a reason for hiding this comment

Uh oh!

vvoland commented Mar 4, 2026

Uh oh!

elezar commented Mar 4, 2026

Uh oh!

vvoland commented Mar 5, 2026

Uh oh!

Uh oh!

elezar commented Mar 5, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shiv-tyagi Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elezar Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elezar commented Mar 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

shiv-tyagi commented Feb 16, 2026 •

edited by vvoland

Loading

shiv-tyagi commented Mar 4, 2026 •

edited

Loading

shiv-tyagi Mar 4, 2026 •

edited

Loading

shiv-tyagi Mar 5, 2026 •

edited

Loading

elezar Mar 5, 2026 •

edited

Loading