Skip to content

Fix the INFERENCE_PRECISION_HINT in load_config not work #777

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
MayureshV1 merged 3 commits into from
Aug 13, 2025
Merged

Fix the INFERENCE_PRECISION_HINT in load_config not work #777

MayureshV1 merged 3 commits into from
Aug 13, 2025

Conversation

@sgbihu
Copy link

@sgbihu sgbihu commented Aug 13, 2025

Description

Override the default value with the value in load_config.

  Ort::KeyValuePairs provider_options;
  provider_options.Add("load_config", "{\"GPU\":{\"INFERENCE_PRECISION_HINT\": \"ACCURACY\"}}");

      const std::string option_prefix = OrtSessionOptions::GetProviderOptionPrefix(ep_to_select.c_str());
      std::vector<const char*> keys, values;
      provider_options.GetKeyValuePairs(keys, values);
      for (size_t i = 0, end = keys.size(); i < end; ++i) {
        // add the default value with prefix
        session_options.AddConfigEntry((option_prefix + keys[i]).c_str(), values[i]);
      }

Motivation and Context

The default configure will passed to OpenVINO and will override the configure in load_config.

@mingmingtasd
Copy link

mingmingtasd commented Aug 13, 2025

Thanks very much! I verified this PR can fix my issue! @sgbihu

@MayureshV1 MayureshV1 requested review from ankitm3k, Copilot and preetha-intel and removed request for Copilot August 13, 2025 07:09
Copilot AI reviewed Aug 13, 2025
View reviewed changes
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes an issue where INFERENCE_PRECISION_HINT configurations from load_config were being overridden by default values set by the OpenVINO execution provider. The change ensures that load_config values take precedence over default configurations.

  • Adds logic to update existing device_config entries with values from target_config before adding new entries
  • Refactors the condition check to reuse the iterator for better performance

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@MayureshV1 MayureshV1 self-requested a review August 13, 2025 07:17
MayureshV1
MayureshV1 approved these changes Aug 13, 2025
View reviewed changes
Copy link

@MayureshV1 MayureshV1 left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sgbihu .. Changes look good to me. Can you please port it to ovep::develop

@sgbihu sgbihu changed the base branch from master to ovep-develop August 13, 2025 07:20
@MayureshV1 MayureshV1 merged commit 609dfbf into intel:ovep-develop Aug 13, 2025
3 of 5 checks passed
javier-intel pushed a commit that referenced this pull request Aug 14, 2025
@sgbihu @javier-intel
Fix the load_config not work when set INFERENCE_PRECISION_HINT (#777)
999ea8f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@MayureshV1 MayureshV1 MayureshV1 approved these changes

@ankitm3k ankitm3k Awaiting requested review from ankitm3k

@preetha-intel preetha-intel Awaiting requested review from preetha-intel

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@sgbihu @mingmingtasd @MayureshV1