Skip to content

BUG: Highly degraded precision for tanh for arrays of float16 on Sapphire Rapids - Regression in 2.3.0 #30821

Description

@Flamefire

Describe the issue:

I ran into an issue I traced to numpy, that was introduced in 2.3.0, 2.2.6 works fine.
I only observed it on Intel Sapphire Rapids, but not AMD Zen3 CPUs

Calculating tanh of Float16 values has a large error when processing arrays. Most notably it saturates at 0.9985 instead of 1

Reproduce the code example:

$ python -c 'import numpy as np; print(np.tanh(np.float16(1000)), np.tanh(np.array([1000], dtype=np.float16)), np.tanh(np.array([1000], dtype=np.float32)))'
1.0 [0.9985] [1.]

Python and NumPy Versions:

2.4.2, but observed since 2.3.0, NOT with 2.2.6

Python 3.11.5 (main, Dec 17 2024, 11:35:51) [GCC 13.2.0], but doesn't matter

Runtime Environment:

[{'numpy_version': '2.4.2',
'python': '3.11.5 (main, Dec 17 2024, 11:35:51) [GCC 13.2.0]',
'uname': uname_result(system='Linux', node='login1', release='5.14.0-570.49.1.el9_6.x86_64', version='#1 SMP PREEMPT_DYNAMIC Fri Oct 3 15:42:32 UTC 2025', machine='x86_64')},
{'simd_extensions': {'baseline': ['X86_V2'],
'found': ['X86_V3', 'X86_V4', 'AVX512_ICL', 'AVX512_SPR'],
'not_found': []}},
{'ignore_floating_point_errors_in_matmul': False}]

How does this issue affect you or how did you find it:

E.g. the mish operation for ML/AI relies on tanh, specifically: x * tanh(softmax(x)). With this issue results are wildly off (due to multiplication with x amplifying the error)

Found in the PyTorch test suite where numpy is used as the reference

Possibly related: #25934 1fa958d

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions