Skip to content

Add numeric_limits for MLFloat16 and BFloat16#22197

Merged
tianleiwu merged 5 commits intomainfrom
tlwu/fp16_bf16_limits
Sep 26, 2024
Merged

Add numeric_limits for MLFloat16 and BFloat16#22197
tianleiwu merged 5 commits intomainfrom
tlwu/fp16_bf16_limits

Conversation

@tianleiwu
Copy link
Copy Markdown
Contributor

@tianleiwu tianleiwu commented Sep 24, 2024 •
edited
Loading

Description

  • Add std::numeric_limits for MLFloat16 and BFloat16.
  • Update some comments in csharp ORTFloat16.shared.cs.
  • Add unit tests (including Clip)

Note that the canonical NaN is not consistent in C++ and C#. C# uses negative quiet NaN as canonical NaN, while C++ uses positive quiet NaN. The choice of CSharp Float16.NaN is to be consistent with System.Half.NaN.

FP16 data returns from CUDA might have 7FFF as NaN; FP16 data from CPU provider might have 0x7E00 as NaN. Anyway there is no consistent canonical NaN in ORT right now. Because all these NaNs are aligned with IEEE spec, there shall not an issue in downstream.

Motivation and Context

std::numeric_limits is used in codebase but not defined for MLFloat16 and BFloat16. It causes some bugs like #21957 introduced by #21493.

@tianleiwu tianleiwu force-pushed the tlwu/fp16_bf16_limits branch from 13e2aa7 to 718e05b Compare September 24, 2024 08:26
snnn
snnn previously approved these changes Sep 24, 2024
View reviewed changes
yuslepukhin
yuslepukhin reviewed Sep 24, 2024
View reviewed changes
Comment thread csharp/src/Microsoft.ML.OnnxRuntime/OrtFloat16.shared.cs Outdated
@yuslepukhin
Copy link
Copy Markdown
Member

yuslepukhin commented Sep 25, 2024

Do we want to add a test for Clip or this is separate?

yuslepukhin
yuslepukhin previously approved these changes Sep 25, 2024
View reviewed changes
Copy link
Copy Markdown
Member

@yuslepukhin yuslepukhin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

yuslepukhin
yuslepukhin previously approved these changes Sep 25, 2024
View reviewed changes
Copy link
Copy Markdown
Member

@yuslepukhin yuslepukhin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

yuslepukhin
yuslepukhin approved these changes Sep 25, 2024
View reviewed changes
Copy link
Copy Markdown
Member

@yuslepukhin yuslepukhin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

github-advanced-security[bot]
github-advanced-security AI found potential problems Sep 25, 2024
View reviewed changes
Comment thread onnxruntime/core/providers/cuda/math/clip.cc
void operator()(cudaStream_t stream, const Tensor* X, const Tensor* min, const Tensor* max, Tensor* Y) const {
auto min_default = clip_internal::LowMax<T>::low();
auto max_default = clip_internal::LowMax<T>::max();
auto min_default = std::numeric_limits<T>::lowest();

Check warning

Code scanning / PREfast

The function 'std::numeric_limits<unsigned __int64>::lowest' is constexpr, mark variable 'min_default' constexpr if compile-time evaluation is desired (con.5).

The function 'std::numeric_limits<unsigned __int64>::lowest' is constexpr, mark variable 'min_default' constexpr if compile-time evaluation is desired (con.5).
Comment thread onnxruntime/core/providers/cuda/math/clip.cc
void operator()(cudaStream_t stream, const Tensor* X, const Tensor* min, const Tensor* max, Tensor* Y) const {
auto min_default = clip_internal::LowMax<T>::low();
auto max_default = clip_internal::LowMax<T>::max();
auto min_default = std::numeric_limits<T>::lowest();

Check warning

Code scanning / PREfast

The function 'std::numeric_limits<__int64>::lowest' is constexpr, mark variable 'min_default' constexpr if compile-time evaluation is desired (con.5).

The function 'std::numeric_limits<__int64>::lowest' is constexpr, mark variable 'min_default' constexpr if compile-time evaluation is desired (con.5).
Comment thread onnxruntime/core/providers/cuda/math/clip.cc
void operator()(cudaStream_t stream, const Tensor* X, const Tensor* min, const Tensor* max, Tensor* Y) const {
auto min_default = clip_internal::LowMax<T>::low();
auto max_default = clip_internal::LowMax<T>::max();
auto min_default = std::numeric_limits<T>::lowest();

Check warning

Code scanning / PREfast

The function 'std::numeric_limits<unsigned char>::lowest' is constexpr, mark variable 'min_default' constexpr if compile-time evaluation is desired (con.5).

The function 'std::numeric_limits<unsigned char>::lowest' is constexpr, mark variable 'min_default' constexpr if compile-time evaluation is desired (con.5).
Comment thread onnxruntime/core/providers/cuda/math/clip.cc
void operator()(cudaStream_t stream, const Tensor* X, const Tensor* min, const Tensor* max, Tensor* Y) const {
auto min_default = clip_internal::LowMax<T>::low();
auto max_default = clip_internal::LowMax<T>::max();
auto min_default = std::numeric_limits<T>::lowest();

Check warning

Code scanning / PREfast

The function 'std::numeric_limits<signed char>::lowest' is constexpr, mark variable 'min_default' constexpr if compile-time evaluation is desired (con.5).

The function 'std::numeric_limits<signed char>::lowest' is constexpr, mark variable 'min_default' constexpr if compile-time evaluation is desired (con.5).
Comment thread onnxruntime/core/providers/cuda/math/clip.cc
void operator()(cudaStream_t stream, const Tensor* X, const Tensor* min, const Tensor* max, Tensor* Y) const {
auto min_default = clip_internal::LowMax<T>::low();
auto max_default = clip_internal::LowMax<T>::max();
auto min_default = std::numeric_limits<T>::lowest();

Check warning

Code scanning / PREfast

The function 'std::numeric_limits<double>::lowest' is constexpr, mark variable 'min_default' constexpr if compile-time evaluation is desired (con.5).

The function 'std::numeric_limits<double>::lowest' is constexpr, mark variable 'min_default' constexpr if compile-time evaluation is desired (con.5).
Comment thread onnxruntime/core/providers/cuda/math/clip.cc
auto min_default = clip_internal::LowMax<T>::low();
auto max_default = clip_internal::LowMax<T>::max();
auto min_default = std::numeric_limits<T>::lowest();
auto max_default = std::numeric_limits<T>::max();

Check warning

Code scanning / PREfast

The function 'std::numeric_limits<__int64>::max' is constexpr, mark variable 'max_default' constexpr if compile-time evaluation is desired (con.5).

The function 'std::numeric_limits<__int64>::max' is constexpr, mark variable 'max_default' constexpr if compile-time evaluation is desired (con.5).
Comment thread onnxruntime/core/providers/cuda/math/clip.cc
auto min_default = clip_internal::LowMax<T>::low();
auto max_default = clip_internal::LowMax<T>::max();
auto min_default = std::numeric_limits<T>::lowest();
auto max_default = std::numeric_limits<T>::max();

Check warning

Code scanning / PREfast

The function 'std::numeric_limits<unsigned char>::max' is constexpr, mark variable 'max_default' constexpr if compile-time evaluation is desired (con.5).

The function 'std::numeric_limits<unsigned char>::max' is constexpr, mark variable 'max_default' constexpr if compile-time evaluation is desired (con.5).
Comment thread onnxruntime/core/providers/cuda/math/clip.cc
auto min_default = clip_internal::LowMax<T>::low();
auto max_default = clip_internal::LowMax<T>::max();
auto min_default = std::numeric_limits<T>::lowest();
auto max_default = std::numeric_limits<T>::max();

Check warning

Code scanning / PREfast

The function 'std::numeric_limits<signed char>::max' is constexpr, mark variable 'max_default' constexpr if compile-time evaluation is desired (con.5).

The function 'std::numeric_limits<signed char>::max' is constexpr, mark variable 'max_default' constexpr if compile-time evaluation is desired (con.5).
Comment thread onnxruntime/core/providers/cuda/math/clip.cc
auto min_default = clip_internal::LowMax<T>::low();
auto max_default = clip_internal::LowMax<T>::max();
auto min_default = std::numeric_limits<T>::lowest();
auto max_default = std::numeric_limits<T>::max();

Check warning

Code scanning / PREfast

The function 'std::numeric_limits<double>::max' is constexpr, mark variable 'max_default' constexpr if compile-time evaluation is desired (con.5).

The function 'std::numeric_limits<double>::max' is constexpr, mark variable 'max_default' constexpr if compile-time evaluation is desired (con.5).
Comment thread onnxruntime/core/providers/cuda/math/clip.cc
auto min_default = clip_internal::LowMax<T>::low();
auto max_default = clip_internal::LowMax<T>::max();
auto min_default = std::numeric_limits<T>::lowest();
auto max_default = std::numeric_limits<T>::max();

Check warning

Code scanning / PREfast

The function 'std::numeric_limits<float>::max' is constexpr, mark variable 'max_default' constexpr if compile-time evaluation is desired (con.5).

The function 'std::numeric_limits<float>::max' is constexpr, mark variable 'max_default' constexpr if compile-time evaluation is desired (con.5).
@tianleiwu tianleiwu merged commit 7880342 into main Sep 26, 2024
@tianleiwu tianleiwu deleted the tlwu/fp16_bf16_limits branch September 26, 2024 00:10
@yihonglyu yihonglyu mentioned this pull request Oct 5, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yuslepukhin yuslepukhin yuslepukhin approved these changes

@snnn snnn Awaiting requested review from snnn

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@tianleiwu @yuslepukhin @snnn @github-advanced-security