So, I think my main worry is this part. We already have two loop versions:

• The legacy that is unwrapped here.
• A new loop signature, which is largely the same (but adds more info and an error return). This is not accessible through PyUFunc_FromFuncAndDataAndSignatureAndIdentity, but we should aim to replace it sooner rather than later internally.
  • This signature is very much open to change, but I would like to be sure that updating existing code is easy eventually.
  • We don't need a "legacy" loop, we only have to support it when downstream gives us one.
  • This new-loop can be specialized for specific strides (in most ufunc calls) for speed, so that selection of SIMD can happen outside of the inner-loop itself.

Since you use the static void *data, the compiler cannot possibly optimize this away either. The actual (new) ufunc signature currently doesn't even include static data (but we can add that back in some form, it is likely useful also for generic heavy functions, where we currently just pass a function pointer also).

Note that one thing I would like mid-term, is to move the current if (stride[0] == 0), etc. branches out of the inner-loop.
The machinery exists, but we have to start using it. This refactor should help with that since it should make our ufunc generation just a bit less of a mess hopefully, and to get there we should update most/all ufunc signatures to the new-style and adapt the ufunc generation a fair bit.

I guess you want C++ iterators and I expect they are really helpful and maybe necessary for later SIMD machinery improvements since those make iteration more painful.

But I do need to ask what the alternatives are? Would it be really be terrible to build an iterator (or multiple iterators) inside the inner-loop with a one-line per-argument boilerplate? That way you dog-feed the C-signature explicitly but still iterate in whatever way we like. Also, I do think it would make it much clearer to have both code styles side-by-side without getting super confused. And we will always have both side-by-side, because the C-signature is still our public API (unless we have the guts to change that, which may be a path, but probably a more difficult and slower one than I would like right now).

One more question I have but not for Sayed explicitly. I suspected we would aim to just remove the Python generation completely, even if we end up with a rather long C++ file (that could be generated via an adapted old file once for reference).

But to me, that would be another advantage of starting of with not modifying the signature at all, because then we can do the two steps separately. (Yes, this requires a bit of double work, but on the plus side you can do it in chunks.)

A new loop signature, which is largely the same (but adds more info and an error return).

PyArrayMethod_Context, and int rather then void as return, well not sure how useful PyArrayMethod_Context is but it can be supported through C++ also new C++ signature can support either bool or int for handling errors even if I think that testing PyErr_Occurred() is quite enough or maybe we can enable C++ exception in some stages.

This is not accessible through PyUFunc_FromFuncAndDataAndSignatureAndIdentity, but we should aim to replace it sooner rather than later internally.

If we are moving to C++ then what the point of continue enhancing C interface? Building a new interface in C++ to count more on static optimization should significantly speed up the internal calls.

This signature is very much open to change, but I would like to be sure that updating existing code is easy eventually.

The current approach provides the best start for the highest layer but it costs extra later for sure we can enhance it later at least by cutting off the legacy layer.

This new-loop can be specialized for specific strides (in most ufunc calls) for speed, so that selection of SIMD can happen outside of the inner-loop itself.

I'm not sure how it's going to be useful for SIMD kernels, we usually support both contiguous and non-contiguous memory access as long as there's no memory overlapping. Specialize within the inner loop is much easier and can save binary size too, For example, we can easily implement a C++ meta class or function that takes two operators one for scalar operation and another for SIMD intrinsic then it's generating all routined branches including SIMD vector memory access, unrolling, reduction and scalar fallback.

Since you use the static void *data, the compiler cannot possibly optimize this away either.

I don't think it's necessary to support it for C++, there're better patterns.

Note that one thing I would like mid-term, is to move the current if (stride[0] == 0), etc. branches out of the inner-loop.

If counting on meta functions to generate these branches wouldn't be robust then we can discuss moving to specialize these branches through UFunc API, after this pr gets merged I will provide real examples in this matter.

I guess you want C++ iterators and I expect they are really helpful and maybe necessary for later SIMD machinery improvements since those make iteration more painful.

It is also important for type safety.

But I do need to ask what the alternatives are?
That way you dog-feed the C-signature explicitly but still iterate in whatever way we like.

Avoid using C++ functions pointers and wraps the C UFunc API through meta programming, this solution is zero-cost overhead but unfortunately not that flexible and increases the complexity of the code especially when exporting internal symbols for CPU runtime dispatching.

And we will always have both side-by-side, because the C-signature is still our public API (unless we have the guts to change that, which may be a path, but probably a more difficult and slower one than I would like right now).

I agree but nothing is perfect, maybe provides inline functions that can serve both directions when it's needs until C++ dominates.

suspected we would aim to just remove the Python generation

Yes, but we also have to get rid of C template sources too. all at once seems more convenient to me. The new signature works only internally and can be improved later at any time.

I made changes to the code, to remove the runtime overhead and pass the inner loops as objects rather than function pointers (fully meta) please take a look at commit 1969494 to realize the differences:

However, once we deal with runtime dispatch or any exported symbols the overhead will come back again. I still prefer the previous method.

Another suggestion to C++ signature is to change the output indicator from reference to non-const:

-(Slice<T> in0, Slice<T> in1, Slice<T> &out)
+(const Slice<T> in0, const Slice<T> in1, Slice<T> out) 

The number of inputs and outputs are deduced from the signature rather than specifying it manually in case we decided later to support non-static signatures similar to function overloading in C++.

I won't have time to look at this for a few weeks. But since Marten commented: Yes, we should revive this.

FWIW, I am still a bit unsure about the right choice of inner-loop. Being more used to C++ these days helps, though.

My main question I guess remains a bit: I think you wrote this under the assumption to make the full thing C++. I lack a bit of clarity that this is a good choice, since we must keep a first class public C-API.

I suppose I can rephrase this into: Please assume that you will never be able to remove the step to translate the current C call into a C++ internal call. Would you reconsider this API choice? If not, then probably fine. But if yes, what would it look like?
(Even if it is just something creating a MultiIter<...>, whether more complicated or not to do the C/C++ layer conversion.)

(I would have been very happy, and still would be happy, to tweak the C signature. But moving where the C/C++ interface remains a big unclear thing to me that I don't know if it could or should be done. If others are more clear on it, that would be fine...)

Why is Holder used for long but not float?

Theoretically, only the following data types are required to be distinguished:

using UByte = Holder<unsigned char, kUByte>;

using Bool = std::conditional_t<sizeof(bool) == sizeof(unsigned char),
    bool, Holder<unsigned char, kBool>>;

using LongDouble = std::conditional_t<kLongDoubleIsDouble,
      Holder<double, kLongDouble>, long double>;

using CLongDouble = std::conditional_t<kLongDoubleIsDouble,
      Holder<complex<double>, kCLongDouble>, complex<long double>>;

To forbid the use of built-in datatypes directly, using long instead of Long should break the build when kTypeID is involved:

https://github.com/numpy/numpy/blob/2beb25fa4e90923e7b260408bf45b81fe238cf44/numpy/core/src/common/npstd.hpp#L174C4-L176

Allowing raw integer built-in data types isn't type-safe when it comes to determining the data type IDs (template specialization), since they are commonly used as alias almost for all the generic std alais including fixed width data types.

Why not? It seems a bit of a trap at the moment, because Object(1); leaks a reference, but Object(some_obj) doesn't.

Supporting RAII should clean up the code, I'm going to support it. My reason at the moment is that we disable C++ exceptions by default which means there's no support for STD runtime including memory operators e.g. new/delete.

This is really nice!

Yes but then I changed it to meta objects rather than function pointers due to @seberg concerns, I kept the doc as-is since we still haven't decided to favor one another.

Function pointers:

template<typename T>
void Add(np::Slice<T> in0, np::Slice<T> in1, np::Slice<T> &out)
{...}

UFuncObject add_obj(
    "add", "this doc of add function",
    Add<Float>, Add<Double>
);

Meta objects:

template <typename T>
struct AddOP {
    void operator ()(Slice<T> in0, Slice<T> in1, Slice<T> &out)
    {...}
};
UFuncObject<AddOP<Float>, AddOP<Double>> add_obj(
    "add", "this doc of add function"
);

note that lambda can be supported too and also both styles above at the same time, however for simplicity of code I left only one.

It would seem best to pick just one. I tried to read the discussion with Stefan more on top, but it left me unclear what the advantage of function pointer vs meta object might be. Is it that one could quite easily use meta objects to wrap old-style ufunc loops?

Anyway, I think both styles will work, and I really like how either allows one to focus on what the loop does rather than on how the iteration works.

I think the point is the meta object should be able to optimize things away (so that the pointer arrays don't need to be copied around on every call). But I am not sure if I understood it right in the call.
This was indeed my main concern: That we for-ever keep translating different function signature styles. If this is all optimized away, then that doesn't matter much.

the advantage of function pointer vs meta object might be

Passing the function pointer as Sebastian explained will add an extra call for each time the inner loop is executed since the C++ function pointer will be stored on C ufunc data arbitrarily while the current approach (Meta-object function) is statically wrapping the C legacy function which allows the compiler to optimize it out.

Choosing function pointers provides more flexibility when it comes to sharing functions among different translation units which I would personally prefer also it improves the readability a bit, later on, we can cut extra calls by letting C++ take the lead over C code and initialize its own PyType_Slot.

Anyway, I'm still happy with meta-objects if this going to resolve this discussion and we can later discuss moving to C++ function pointers once we get rid of template sources and the umath python generator.

Ah, thanks, both, that really helps!

debug statement left in by mistake?

output -> outputs

at current moment -> at the moment (or at present)

Construct (also below).

This really does prove the point how nice this is!

Maybe rename to InitOperatorsCpp or so - that will help people looking at this understand why there are two similarly named functions (and find the relevant file where they are defined more easily).

Commented out because not sure what the best way is?

Bit unclear, at least just following the commits, why this indirection was made. I liked the first iteration better in that one is just writing an inner loop function, which seems more natural. But I guess this where documentation would come in...

I'm also curious 😸 a meta object of templated functions doesn't seem functionally different from the struct variant, which probably means I'm missing something obvious.