[QNN EP] Passthrough EP Parameters in Node #23468
Conversation
Update forked repository to latest
|
/azp run Big Models,Linux Android Emulator QNN CI Pipeline,Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline |
|
/azp run Windows ARM64 QNN CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,Windows x64 QNN CI Pipeline,onnxruntime-binary-size-checks-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed |
|
/azp run ONNX Runtime Web CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows GPU CUDA CI Pipeline,Windows GPU DML CI Pipeline,Windows GPU Doc Gen CI Pipeline |
|
Azure Pipelines successfully started running 5 pipeline(s). |
|
Azure Pipelines successfully started running 9 pipeline(s). |
|
Azure Pipelines successfully started running 5 pipeline(s). |
|
Sorry for the late review. I think we should better first define the JavaScript API for QNN here, then in this PR parse/read them accordingly. |
…ault elsewhere. No need to set it here.
Please let me know if you are good with the change (naming and typing). Please approve the PR if you are OK with it and I will go merge it. |
### Description As a pre-requisite of #23468
|
/azp run Windows ARM64 QNN CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,Windows x64 QNN CI Pipeline,onnxruntime-binary-size-checks-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed |
|
/azp run ONNX Runtime Web CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows GPU CUDA CI Pipeline,Windows GPU DML CI Pipeline,Windows GPU Doc Gen CI Pipeline |
|
/azp run Big Models,Linux Android Emulator QNN CI Pipeline,Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline |
|
Azure Pipelines successfully started running 5 pipeline(s). |
1 similar comment
|
Azure Pipelines successfully started running 5 pipeline(s). |
|
Azure Pipelines successfully started running 9 pipeline(s). |
### Description As a pre-requisite of microsoft#23468
### Description As a pre-requisite of microsoft#23468
|
I checked the implementation and I found that the existing implementation has a few errors. Since the errors are not only for QNN ( there are also places to be fixed for CoreML and WebGPU), so I made this change. Please refer to 4ce51f3 as a patch to this PR. |
### Description As a pre-requisite of #23468
@joncamp would you be able to apply @fs-eire 's patch to this PR? let's try to merge it into main asap (for upcoming ORT 1.21 release, code freeze end of week) |
fix session option parse
|
/azp run Windows ARM64 QNN CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline,Windows x64 QNN CI Pipeline,onnxruntime-binary-size-checks-ci-pipeline,orttraining-linux-ci-pipeline,orttraining-linux-gpu-ci-pipeline,orttraining-ortmodule-distributed |
|
/azp run Big Models,Linux Android Emulator QNN CI Pipeline,Linux CPU CI Pipeline,Linux CPU Minimal Build E2E CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,Linux OpenVINO CI Pipeline,Linux QNN CI Pipeline,MacOS CI Pipeline |
|
Azure Pipelines successfully started running 5 pipeline(s). |
|
/azp run ONNX Runtime Web CI Pipeline,Win_TRT_Minimal_CUDA_Test_CI,Windows GPU CUDA CI Pipeline,Windows GPU DML CI Pipeline,Windows GPU Doc Gen CI Pipeline |
|
Azure Pipelines successfully started running 5 pipeline(s). |
|
Azure Pipelines successfully started running 9 pipeline(s). |
fs-eire
left a comment
There was a problem hiding this comment.
LTGM.
Please test if the code works before merge.
|
/azp run Linux GPU CI Pipeline |
|
Azure Pipelines successfully started running 1 pipeline(s). |
### Description As a pre-requisite of #23468
### Description The existing implementation of session options for the QNN EP does not honor the various bindings available. As such, even if set at runtime they are ignored. Fix is to follow the pattern of the `webgpu` provider and parse/populate the options accordingly. Existing defaults are preserved, such that if options are not set the prior behavior will persist. ### Motivation and Context During debugging and development of Node implementations using the QNN EP the need to set various parameters became apparent. Currently the parameters can only be set via changes to the ORT dll code itself, which is inflexible and slows development. --------- Co-authored-by: Yulong Wang <7679871+fs-eire@users.noreply.github.com>
### Description As a pre-requisite of #23468
### Description The existing implementation of session options for the QNN EP does not honor the various bindings available. As such, even if set at runtime they are ignored. Fix is to follow the pattern of the `webgpu` provider and parse/populate the options accordingly. Existing defaults are preserved, such that if options are not set the prior behavior will persist. ### Motivation and Context During debugging and development of Node implementations using the QNN EP the need to set various parameters became apparent. Currently the parameters can only be set via changes to the ORT dll code itself, which is inflexible and slows development. --------- Co-authored-by: Yulong Wang <7679871+fs-eire@users.noreply.github.com>
Description
The existing implementation of session options for the QNN EP does not honor the various bindings available. As such, even if set at runtime they are ignored. Fix is to follow the pattern of the
webgpuprovider and parse/populate the options accordingly.Existing defaults are preserved, such that if options are not set the prior behavior will persist.
Motivation and Context
During debugging and development of Node implementations using the QNN EP the need to set various parameters became apparent. Currently the parameters can only be set via changes to the ORT dll code itself, which is inflexible and slows development.