Skip to content

BUG: left bit shift undefined behavior#29539

Merged
ngoldbaum merged 2 commits intonumpy:mainfrom
m-clare:fix/bit-shift-ubsan
Aug 11, 2025
Merged

BUG: left bit shift undefined behavior#29539
ngoldbaum merged 2 commits intonumpy:mainfrom
m-clare:fix/bit-shift-ubsan

Conversation

@m-clare
Copy link
Copy Markdown
Contributor

@m-clare m-clare commented Aug 11, 2025

  • Addresses UBSAN warnings for sse/arithmetic.h and avx2/arithmetic.h
  • adjusts variable names for operations (sigb to sbit) to match those in sse/operations.h avx2/operations.h for the same operation with a vector argument (see snippet below).
  • Removes UBSAN suppressions for these files, compiler_sanitizers.yml workflow should succeed without errors.
  • partially closes BUG: resolve undefined behavior issues currently suppressed by UBSAN CI #29524

From sse/operations.h L34:

NPY_FINLINE __m128i npyv_shr_s64(__m128i a, int c)
{
    const __m128i sbit = npyv_setall_s64(0x8000000000000000);
    const __m128i cv   = _mm_cvtsi32_si128(c);
    __m128i r = _mm_srl_epi64(_mm_add_epi64(a, sbit), cv);
    return _mm_sub_epi64(r, _mm_srl_epi64(sbit, cv));
}

The pattern throughout these implementations is to use the hex representation of the value, which I've done here. Not sure if it would be a better option to use the limit library instead.

@m-clare m-clare changed the title fix: left bit shift undefined behavior BUG: left bit shift undefined behavior Aug 11, 2025
@ngoldbaum
Copy link
Copy Markdown
Member

I re-triggered the failing CI job, which was unrelated to this PR.

@ngoldbaum
Copy link
Copy Markdown
Member

Ping @seiko2plus - this PR touches code you originally wrote back in 2021.

Copy link
Copy Markdown
Member

@seberg seberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am fine with just putting it in, I do think the << is clearer to read, but this is so targeted, that I guess it just doesn't matter.

const __m256i sigb = npyv_setall_s64(1LL << 63);
q = _mm256_srl_epi64(_mm256_add_epi64(q, sigb), shf1);
q = _mm256_sub_epi64(q, _mm256_srl_epi64(sigb, shf1));
const __m256i sbit = npyv_setall_s64(0x8000000000000000);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose it is relatively clear from the sbit context, but I wonder if we can at least keep a

Suggested change
const __m256i sbit = npyv_setall_s64(0x8000000000000000);
const __m256i sbit = npyv_setall_s64(0x8000000000000000 /* 1 << 63 */);

(since I don't have a better idea, I assume 1ULL << 63 doesn't warn, but then we would also need to cast back to long long probably, so dunno that is better)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did try it out with 1ULL << 63 and it removed the warning, but was trying to be consistent with other bit shift operations.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sebastian's suggestion just adds a comment, I think it's fine to add. Probably also worth adding in sse/operators.h as well:

sse/operators.h
36:    const __m128i sbit = npyv_setall_s64(0x8000000000000000);
220:        const __m128i sbit = npyv_setall_s64(0x8000000000000000);

There's also this in sse/arithmetic.h:

sse/arithmetic.h
254:    const __m128i sigb = npyv_setall_s64(1LL << 63);

Although maybe this one is fine because it's a signed long long?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, that one also gets flagged by UBSAN. I also have a fix in for that in this PR.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah of course, got a little tunnel-vision there.

Let's merge this as-is, we can clean up the constants to be formatted nicely later. Fixing the UB is way more important.

@ngoldbaum ngoldbaum merged commit 7c34da7 into numpy:main Aug 11, 2025
84 of 85 checks passed
@ngoldbaum
Copy link
Copy Markdown
Member

Thanks @m-clare!

IndifferentArea pushed a commit to IndifferentArea/numpy that referenced this pull request Dec 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: resolve undefined behavior issues currently suppressed by UBSAN CI

3 participants