-
Notifications
You must be signed in to change notification settings - Fork 126
Relax the required accuracy of tan(half)? #1303
Description
Since the working group has recently reduced the required accuracy of 1 / half, I'd like to request relaxing the required accuracy of tan(half).
My implementation fails bruteforce on tan(half) like this:
ERROR: tan: 2.248892 ulp error at 0x1.2e8p+14 (half 0x74ba)
Expected: 0x1.edcp+3 (half 0x4bb7)
Actual: 0x1.ee4p+3 (half 0x4bb9)
This 2.25ULP error is the largest of my implementation for all half inputs.
Consider these steps (0x1.2e8p+14 == 19360)
// Reduce argument
19360 - 12325 * pi / 2 = -0.064727747100832...
// Compute result
tan(-0.064727747100832...) = -0.064818295060042395...
-1 / -0.064818295060042395... = 15.4277430326373342225222...
// Exact result
tan(19360) = 15.4277430326373342225222...
// Rounding to half
round(-0.064727747100832...) = -0x1.09p-4
tan(-0x1.09p-4) = -0.0647876855794585216773... which rounds to -0x1.094p-4
-1 / -0x1.094p-4 = 15.4420358152686145146... which rounds to 0x1.ee4p+3 (see actual)
So, even with a correctly rounded reduced argument, correctly rounded tan(), and correctly rounded reciprocal, we end up with a relative error of 2.25ULP.
Since we would like to carry out as much of the tan(half) implementation in half precision as possible, we're requesting lifting the required accuracy of tan(half) to 2.25 ULP.