Add support for flexible device hierarchy model by jandres742 · Pull Request #169 · oneapi-src/level-zero-spec

jandres742 · 2023-06-27T06:51:41Z

Resolves: #1

Resolves: oneapi-src#1 Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>

jandres742 · 2023-06-27T06:52:09Z

@wdamon-intel , @MichalMrozek: please review

wdamon-intel

LGTM, mostly, just a few minor comments to address.

wdamon-intel · 2023-06-27T08:59:07Z

scripts/core/PROG.rst

+
+With a value of `0`, ${x}DeviceGet returns all the devices that do not have a root-device. Traversing the device hierarchy is possible by querying sub-devices with ${x}DeviceGetSubDevices and root-devices with ${x}DeviceGetRootDevice. Driver implementation may perform implicit optimizations to submissions and allocations done in the root-devices.
+
+With a value of `1`, ${x}DeviceGet returns all the devices that do not have sub-devices. Traversing the device hierarchy is **not** possible, with ${x}DeviceGetSubDevices returning always a count of 0 device handles and and ${x}DeviceGetRootDevice returning nullptr. This mode allows Level Zero driver implementations to optimize execution and memory allocations by removing any overhead required to account for simultaneous use of root-devices and sub-devices in the same application.


[...] 0 device handles and and [...]
"and ~~and~~"

scripts/core/PROG.rst

MichalMrozek · 2023-06-27T09:18:00Z

scripts/core/PROG.rst

+
+${X}_FLAT_DEVICE_HIERARCHY allows users to select the device hierarchy model with which the underlying hardware is exposed and the types of devices returned with ${x}DeviceGet.
+
+With a value of `0`, ${x}DeviceGet returns all the devices that do not have a root-device. Traversing the device hierarchy is possible by querying sub-devices with ${x}DeviceGetSubDevices and root-devices with ${x}DeviceGetRootDevice. Driver implementation may perform implicit optimizations to submissions and allocations done in the root-devices.


in mode 0 DeviceGetRootDevice is not allowed.

@MichalMrozek

You mean in mode 0, which is what we have currently, the application cannot query for the root device? could you elaborate on why would that be? this has been a request from some customers some times, to query for the parent device for a sub-device. From point of view of implementation, this doesnt change too much, as the Level Zero Driver implementation already has that info, it would be just a matter of returning it.

The API was only meant for mode 2, when you start to flatten the hierarchy.
If hierarchy is not flattened (as in mode 0), then getroot is not supported.

That's the whole difference between mode 2 and other modes, it allows this new API.

@MichalMrozek : thanks. yes, the API was originally meant for mode 2, but there's no functional or performance reason to not have it available in mode 0, would that be correct? By having it in mode 0 as well, we allow users to have more flexibility and broader set of APIs to use.

let's say you are in mode 0
app did ZE_AFFINITY_MASK=0.0
app calls getRoot on such device, what is returned?

@MichalMrozek it is return nullptr, but if you are in mode 0, and use ZE_AFFINITY_MASK=0, then you can call getRootDevice on any handle obtained with getSubDevices and get the parent device.

It is the same for getSubDevices. With MASK=0, getSubDevices for a root device returns a list of sub-devices, and with MASK=0.0 getSubDevices returns a count of 0. Spec doesnt forbid the use of getSubDevices with MASK=0.0, it just returns the appropriate value depending on the situation.

but in mode 1 , we will return all sub devices as root device and getRoot will return nullptr?

@MichalMrozek Correct. in mode 1, getRootDevice returns nullptr, and also getSubDevices will return count of 0.

MichalMrozek · 2023-06-27T09:19:28Z

scripts/core/PROG.rst

+same system with two devices and four sub-devices.
+
+- `0, 1, 2, 4`: all sub-devices are reported by ${x}DeviceGet (same as default)
+- `0`: only sub-device 0 in the first device is reported


there are no sub devices when FLAT_DEVICE_HIERACHY is "1"

thanks @MichalMrozek . I have expanded this section to make it clearer and added some tables showing what is being exposed. It would look something like this when rendered:

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>

wdamon-intel

LGTM

jandres742 · 2023-07-03T05:51:38Z

@MichalMrozek : do you have further comments or think that PR is good to go?

MichalMrozek · 2023-07-03T06:27:35Z

The initial plan was that new api getRoot is only available in mode 2.
Right now it will be enabled in mode 0, so this would be addition to existing model.
We need clear explanation that resources created for sub devices will not work on extracted root devices, unless root device is a part of the context, same with programs, kernels etc.
Right now we allow that resources created on root devices are available in sub devices , even if they are not in the context, so we need to be careful to not have the same level of expectations with new getRoot API.

jandres742 · 2023-07-04T05:13:17Z

The initial plan was that new api getRoot is only available in mode 2. Right now it will be enabled in mode 0, so this would be addition to existing model. We need clear explanation that resources created for sub devices will not work on extracted root devices, unless root device is a part of the context, same with programs, kernels etc. Right now we allow that resources created on root devices are available in sub devices , even if they are not in the context, so we need to be careful to not have the same level of expectations with new getRoot API.

@MichalMrozek : currently, we allow for sub-devices to share the resources from the root device, that's correct, and you are correct, the same cannot be assumed when you have created a resource in a sub-device, and then getRootDevice is called.

So we could add something like this to getRootDevice:

"To allow the device handle returned by getRootDevice access the resources created by the sub-device handle, these resources need to be created with a context containing explicitly both handles.".

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>

jandres742 · 2023-07-06T20:50:21Z

The initial plan was that new api getRoot is only available in mode 2. Right now it will be enabled in mode 0, so this would be addition to existing model. We need clear explanation that resources created for sub devices will not work on extracted root devices, unless root device is a part of the context, same with programs, kernels etc. Right now we allow that resources created on root devices are available in sub devices , even if they are not in the context, so we need to be careful to not have the same level of expectations with new getRoot API.

@MichalMrozek : currently, we allow for sub-devices to share the resources from the root device, that's correct, and you are correct, the same cannot be assumed when you have created a resource in a sub-device, and then getRootDevice is called.

So we could add something like this to getRootDevice:

"To allow the device handle returned by getRootDevice access the resources created by the sub-device handle, these resources need to be created with a context containing explicitly both handles.".

Added clarification.

Add support for flexible device hierarchy model

cb72954

Resolves: oneapi-src#1 Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>

wdamon-intel reviewed Jun 27, 2023

View reviewed changes

MichalMrozek reviewed Jun 27, 2023

View reviewed changes

Jaime Arteaga added 2 commits June 27, 2023 10:10

Expand section for affinity mask

86a9858

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>

Define names for HIERARCHY environment variable

d49a7fa

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>

wdamon-intel approved these changes Jun 29, 2023

View reviewed changes

Add clarification on getRootDevice

0bf17d4

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>

MichalMrozek approved these changes Jul 7, 2023

View reviewed changes

wdamon-intel merged commit 05e8e15 into oneapi-src:master Jul 7, 2023


		With a value of `0`, ${x}DeviceGet returns all the devices that do not have a root-device. Traversing the device hierarchy is possible by querying sub-devices with ${x}DeviceGetSubDevices and root-devices with ${x}DeviceGetRootDevice. Driver implementation may perform implicit optimizations to submissions and allocations done in the root-devices.

		With a value of `1`, ${x}DeviceGet returns all the devices that do not have sub-devices. Traversing the device hierarchy is not possible, with ${x}DeviceGetSubDevices returning always a count of 0 device handles and and ${x}DeviceGetRootDevice returning nullptr. This mode allows Level Zero driver implementations to optimize execution and memory allocations by removing any overhead required to account for simultaneous use of root-devices and sub-devices in the same application.


		${X}_FLAT_DEVICE_HIERARCHY allows users to select the device hierarchy model with which the underlying hardware is exposed and the types of devices returned with ${x}DeviceGet.

		With a value of `0`, ${x}DeviceGet returns all the devices that do not have a root-device. Traversing the device hierarchy is possible by querying sub-devices with ${x}DeviceGetSubDevices and root-devices with ${x}DeviceGetRootDevice. Driver implementation may perform implicit optimizations to submissions and allocations done in the root-devices.

Conversation

jandres742 commented Jun 27, 2023

Uh oh!

jandres742 commented Jun 27, 2023

Uh oh!

wdamon-intel left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jandres742 Jun 28, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wdamon-intel left a comment

Choose a reason for hiding this comment

Uh oh!

jandres742 commented Jul 3, 2023

Uh oh!

MichalMrozek commented Jul 3, 2023

Uh oh!

jandres742 commented Jul 4, 2023

Uh oh!

jandres742 commented Jul 6, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jandres742 Jun 28, 2023 •

edited

Loading