add simdutf constexpr more thoroughly #864

lemire · 2025-11-21T23:52:11Z

This is an extension of PR #859 by @anonrig

I picked up a comment by @pauldreik and ran with it.

This PR, just like #859 is not expected to affect the code's performance or functionality.

However, progressively extending the range of code that is constexpr is probably worthwhile.

lemire · 2025-11-22T03:27:30Z

@pauldreik Please tell me what you think.

pauldreik

I like this improvement! just some minor changes requested - constexpr on function declarations only needs C++11 constexpr (which we demand by requiring C++11). the new simdutf_constexpr macro only needs to be used on if statements.

pauldreik · 2025-12-03T14:45:16Z

src/scalar/swap_bytes.h


+namespace utf32 {
+template <endianness big_endian>
+simdutf_constexpr uint32_t swap_if_needed(uint32_t c) {


I think this should be constexpr, not simdutf_constexpr

pauldreik · 2025-12-03T14:45:28Z

src/scalar/swap_bytes.h

+
+namespace utf16 {
+template <endianness big_endian>
+simdutf_constexpr uint16_t swap_if_needed(uint16_t c) {


I think this should be constexpr, not simdutf_constexpr

If I do so, I get...

/Users/dlemire/CVS/github/simdutf/src/scalar/swap_bytes.h:34:3: warning: use of this statement in a constexpr function is a C++14 extension [-Wc++14-extensions] 34 | if simdutf_constexpr (!match_system(big_endian)) { | ^ /Users/dlemire/CVS/github/simdutf/src/scalar/swap_bytes.h:37:5: warning: multiple return statements in constexpr function is a C++14 extension [-Wc++14-extensions] 37 | return c;

pauldreik · 2025-12-03T14:46:29Z

src/scalar/utf16.h


 namespace simdutf {
 namespace scalar {
-namespace {


was this change intended and/or necessary?

Without it, the build is broke, please see #869

pauldreik · 2025-12-03T14:47:09Z

src/scalar/utf16.h


 // variable templates are a C++14 extension
-template <endianness big_endian> char16_t replacement() {
+template <endianness big_endian> simdutf_constexpr char16_t replacement() {


I think this should be an ordinary constexpr ( there are more places above with the same remark)

@pauldreik I changed this one. What is allowed to be constexpr in C++11 is quite limited. So we can't just go and replace every simdutf_constexpr by a constexpr outside of the if constexpr expressions. However, I did a pass to try to extend the reach of the standard constexpr.

pauldreik · 2025-12-03T15:20:14Z

src/simdutf/arm64/simd.h

-    int8x16x2_t pair = match_system(big_endian)
-                           ? int8x16x2_t{{this->value, vmovq_n_s8(0)}}
-                           : int8x16x2_t{{vmovq_n_s8(0), this->value}};
+    simdutf_constexpr int8x16x2_t pair =


ordinary constexpr (since it is not an if)

Fails with

/Users/dlemire/CVS/github/simdutf/src/simdutf/arm64/simd.h:313:27: fatal error: constexpr variable 'pair' must be initialized by a constant expression 313 | constexpr int8x16x2_t pair = | ^ 314 | match_system(big_endian) ? int8x16x2_t{{this->value, vmovq_n_s8(0)}} | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 315 | : int8x16x2_t{{vmovq_n_s8(0), this->value}}; | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

pauldreik · 2025-12-03T15:21:24Z

src/westmere/sse_utf16fix.cpp

-      _mm_cmpeq_epi16(lb_masked, _mm_set1_epi16(swap_if_needed(0xd800U)));
-  block_is_low =
-      _mm_cmpeq_epi16(block_masked, _mm_set1_epi16(swap_if_needed(0xdc00U)));
+  lb_masked = _mm_and_si128(


just an opinion: I think the previous code was easier to read

Yes. I agree. Changed with a using so that it is not so busy. Did similar changes elsewhere.

I keep forgetting that alias is only for types. Ok. Did it the long way around.

…++11.

lemire · 2025-12-04T05:05:56Z

@pauldreik Should be better now. C++11 is quite limited, however.

pauldreik

I think this is good to merge!

I also wonder if the compile time changes for the better or worse in C++17 builds with these changes?

lemire · 2025-12-04T15:12:55Z

I also wonder if the compile time changes for the better or worse in C++17 builds with these changes?

Before I merge, can you elaborate on your concern ?

Here is my belief: throwing in constexpr here and there should not affect compile time in a significant manner.

Do you have reasons to think otherwise?

Of course we can measure, but it would be useful to have a mental model. In the past, I compared the compile time of major libraries as you switched from C++ 11 to C++17 to C++20. I never formalized my findings, but recollection is that for the same code base, exact same code, the compile time got worse as we went from C++11 to C++20.

Thankfully, developers acquire much faster processors over time... So, in the real world, our code compiles faster.

pauldreik · 2025-12-04T16:04:27Z

I also wonder if the compile time changes for the better or worse in C++17 builds with these changes?

Before I merge, can you elaborate on your concern ?

speaking for an improvement: if constexpr means the compiler can throw away the uninteresting branch early
speaking against an improvement: constexpr means stuff is evaluated in the "virtual abstract machine" in the compiler, which might be slow (it has only been around for ten years or so, while normal compilation has been done forever)

Here is my belief: throwing in constexpr here and there should not affect compile time in a significant manner.

Do you have reasons to think otherwise?

Of course we can measure, but it would be useful to have a mental model. In the past, I compared the compile time of major libraries as you switched from C++ 11 to C++17 to C++20. I never formalized my findings, but recollection is that for the same code base, exact same code, the compile time got worse as we went from C++11 to C++20.

I agree it is good with a mental model! Unfortunately I don't have one. Instead I now measured before and after the change. The difference was within noise (gcc 14, debug, ninja, c++17).

Thankfully, developers acquire much faster processors over time... So, in the real world, our code compiles faster.

Agree.

lemire · 2025-12-04T16:09:32Z

Merging. This will be part of the next release.

anonrig and others added 3 commits November 18, 2025 11:17

add simdutf_constexpr macro

d75c48e

use C++17 on macos workflow

83eb98e

extending the constexpr over the whole code base.

97e6aa7

anonrig approved these changes Nov 22, 2025

View reviewed changes

lemire added 2 commits November 21, 2025 19:49

fixing typo

8b7f062

reverting another overeager change

1873198

lemire requested a review from pauldreik November 22, 2025 03:27

pauldreik requested changes Dec 3, 2025

View reviewed changes

lemire mentioned this pull request Dec 4, 2025

illustrate issue in issue 864 [NOT FOR MERGING] #869

Closed

lemire added 6 commits December 3, 2025 23:29

replacing some simdutf_constexpr by constexpr where possible as per C…

1c8ede6

…++11.

simplifying some verbose code by using 'using'.

5aabdc0

silly me, can't alias a function. Ok. Doing it the long way.

1133d99

undoing some pure constexpr

69a0b5a

constexpr lambdas require C++17

114a255

simplification

4bdd84d

lemire requested a review from pauldreik December 4, 2025 05:05

lint

35726ac

pauldreik approved these changes Dec 4, 2025

View reviewed changes

lemire merged commit e6cb2b9 into master Dec 4, 2025
70 checks passed

add simdutf constexpr more thoroughly #864

add simdutf constexpr more thoroughly #864

Uh oh!

Conversation

lemire commented Nov 21, 2025

Uh oh!

lemire commented Nov 22, 2025

Uh oh!

pauldreik left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lemire commented Dec 4, 2025

Uh oh!

pauldreik left a comment

Choose a reason for hiding this comment

Uh oh!

lemire commented Dec 4, 2025

Uh oh!

pauldreik commented Dec 4, 2025

Uh oh!

lemire commented Dec 4, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants