This is confusing - are you recasting? To what? Would seem worth a comment!

Ah, I added the -1 for "error" to the casting enum at some point, but always with an underscore and just returned -1 everywhere for error. Turns out C++ dosn't like just returning -1, so I need to either write _NPY_ERROR_OCCURRED_IN_CAST or add the cast...

I think _NPY_ERROR_OCCURRED_IN_CAST is clear; might as well use it for new code.

Nit, but I'd write

npy_bool first_is_pyint = dtypes[0] == &PyArray_PyIntAbstractDType;
int arr_idx = first_is_pyint? 1: 0;
int scalar_idx = first_is_pyint? 0: 1;   # or 1-arr_idx
PyObject *scalar = input_scalars[scalar_idx];

and then use first_is_pyint and scalar below.

Is -2 just the same as (NPY_CASTING)-1? If so, I'd made that explicit, i.e., if (value_range == (NPY_CASTING)-2)

I will change it to if (get_value_range(..., &value_range) < 0) {return...}, they are not the same, just translating one error to another, but its OK to be explicit (and it should get inlined anyway).

This name is rather bad. Just resolve_descriptors_pyscalars?

Thanks, never liked it either! just didn't have a good idea. Maybe just _scalars or even _with_scalars? Because I think it may be nice to allow that in some future.

This would be if (first_is_pyint)

Which string loops?

Don't decref NULL - dtype_tuple?

Again string???

Sorry, I had started with a string ufuncs as they are also templated in C++ for all comparisons, and wasn't careful enough with removing all th leftovers. Will look through more carefully.

Is it useful to pass in nin? I though that one could just get that with ufunc->nin? (Similarly, is nop necessary?)

More generally, I wonder if it wouldn't make more sense to just pass through everything one could possibly want to the _raw version. If a ufunc wants raw, perhaps it should deal with the nitty-gritty (including even check_safety, or set that up differently so one is not stuck with original_dtypes). In that case, the name is more appropriate too...

The question then is if there is anything missing from the current argument list above that could possibly be useful? Logically, I guess it is just the outputs -- which could be easily attached by passing through full_args rather than just full_args.in.

If so, I'd suggest to remove nin and add ufunc_full_args full_args

Yes we don't have to pass nin and nout, it is on the ufunc and ufuncimpl. I guess for a single nop it seemed easier, with both it is getting unclear.

Right, I would be happy to add outputs in principle. Right now I did this because I limited to true scalars and those are simply never outputs (i.e. the i < nout is only a sanity check).
One reason was that it isn't necessary trivial to distinguish a scalar from a 0-D object and the original 0-D object may need to be converted to an array to do anything with it so at that point I would have to pass both if you were to actually use that information reasonably. One thing that could work is checking for whether it is a scalar based on the DType, even when not abstract, and otherwise pass the array.)

I havne't thought of integrating the cast-safety check. Right now we have a "disable cast safety" special path for logical ufuncs loop (casting to bool is always OK).
I would be happy with passing the cast-safety and either allowing or forcing you to take care of it. I.e. we can remove the flag or if you want to deal with it you use set the flag so NumPy doesn't check again.
(I am not aware of where it would be useful to side-step the check except for that logical bool, bool -> bool loop.)

I would agree that there might also be a good reason to allow a "let me do everything" path where we let the user do as much as reasonable (e.g. also the full iteration). I think we should have that although finding the right API for it might not be trivial (unless we launch back into/through effectively python?!).
It isn't super easy to get feature parity, but would allow a lot of weirder stuff and speed experimentation that might be cool. (Also just making allowing more things to look like a ufunc and use the protocol.)

Yes, it is tricky. And indeed, if one wants more control, it would probably also be necessary to know not just the ufunc but also whether it is a call, at, reduce, etc. At which point it is no longer obvious this is the right place. Still, my tendency would be to add full_args instead of the inputs and just pass everything on, i.e., let the implementation deal with input_scalars (and original_types). But perhaps keep the check on cast safety, since that is what the function has to return anyway.

Well, I prefer passing everything on as PyObject *const * (I will add the const) rather than two tuples. Just because I think that is the more convenient API usually, and I also think changing the way we do this in the ufunc code itself would be an improvement probably (create the tuple when we need it, we don't need it often).

Passing the method additionally, would seem OK to me. I am not sure it is useful/needed, but I could imagine it can be (if just for printing a nicer custom error message).

My uncertainty is how implementations should deal with weird types (non-arrays but also not obvious scalars), so I am not quite sure what is the best thing to pass on there if that is what they want.
Thus, my maybe silly inclination to punt on it, but maybe rename the argument. We could pass all arguments as it is, we currently just keep them NULL when it isn't quite clear what is best.

OK, true, and I don't want to make too big a deal of it.

But I think my suggestion, passing on the outputs as well, remains easier to explain and work with. And while you are right that passing in the pre-conversion-to-array input may be quite useless from an ndarray perspective, who knows, maybe it is really handy from the perspective of a more random (sub)class that overrides __array_ufunc__ and needs some strange dtype resolution as well. (For quantities, like @ngoldbaum, we had this nice procedure of the final unit of an operation only depending on the units of the arguments, which works for all but np.power. For that case, the scalar value is all that is needed, but maybe for other cases one needs the class shape or so.)

Or, to turn the argument around, how does passing on all inputs and outputs hurt the current use cases?

🤷 Yes. I am fine if you wish to say that I largely don't like it because I don't want you to do things like np.emath (inspect values to guess result dtype). But even after you might convince me that there are actual use-cases where this would be useful – I am sure there are –, I am confident that doing it this way will still be a bad protocol for reasons beyond being brittle for alternate array-objects and conceptually not really extending to non-arrays at all.

One extension, we could also consider is pivoting to is passing char *const *scalar_vals which would hold the raw (possibly unaligned?) original dtype data pointer (whatever that is) or the Python objects for the current Python scalars.
If we talk about shapes, what I think you actually want is passing the core dimensions (reduction size/shape), not original objects (you can't really infer much from there), this is even more interesting in the get_loop() (where we have the data available in principle).

(In a sense, a lot of this to me comes back to: Yes, we probably want a "let me do everything", or even an alternative pivot where we allow more ufunc-like objects to dispatch via __array_ufunc__ to embrace a split of "ufunc as an implementation" and "ufunc as a protocol" a bit.)

My problem remains that the current approach feels super directed just at the particular use case. If that was done privately, without opening API for it, that would be perfectly fine. But if one adds new API, it would seem nice to be not so directed at one particular use case.

But maybe your idea for the extension is a way to resolve this. Though would one perhaps generalize it slightly? Either the original dtype or the object if it had no dtype (i.e., getattr(obj, "dtype", obj)). Or maybe just anything that did not have a dtype, and NULL otherwise? (I'm not quite sure any more what processing has happened to the dtype before we get to this stage; maybe passing the original dtype is best if it can be different.)

Yes, it is directed at the string multiplication/unit power use-case, because that is the biggest hole I am aware off and there is some overlap in the solution.
Maybe the main problem is that it isn't quite clear yet what the correct pattern is of enabling that use case.

I can live with getting it wrong (i.e. deprecating it later). But we really need a solution for NEP 50's sake more than anything. So I am OK either way:

1. Just pass all arguments and make it public. I dislike it, so I would add a scary warning to it (maybe an underscore), but in the end we are all grown-ups here after all.
2. Just make it private for now.
3. We iterate once and maybe land on the char ** solution (may make this use-case slightly more complex, but generalize the string mul/unit power one maybe).

Just FYI... Probably clear, but let me summarize what happens right now:

• __array_ufunc__, this is the only place where you can avoid any form of array conversion.
• Array coercion. This is a two step process:
  • Finding shape and dtype. Which includes calling __array__() when necessary.
  • If necessary, creating a new array and filling it in.
    Especially for scalars, we don't have to do the second step in principle. But for most array-like objects, we currently don't allow delaying the full conversion anyway.
• Promotion based on the DType classes.
  • This find the ArrayMethod
• The resolution step (this one). We clearly have the "original" dtype instances available here. (For Python integers, this right now means its could be an int64, uint64 or object dtype).
  • Normally, we will convert/cast these to the actual loop DType class so that the user doesn't have to worry about that. (This is normally a de-facto no-op, because most dtypes are non-parametric)

My problem remains that the current approach feels super directed just at the particular use case.

Yes, it clearly is directly at solving exactly two things. First, side-stepping the preparation because in its current form it fails for abstract DTypes and, second, trying to create/prepare a path forward for unit power and string multiply. (maybe it isn't particularly good at that, because you may want to support 3-4 things: unit**4, unit**np.uint8(4) and unit**np.array([4]); where the last IMO should include array-likes if you allow it).

I can't think of a clear solution to this... So, I made the slot raise an error when used as public API (effectively making it private).

Unrelated?

Uhh, yeah, the test had failed on me when I had a buggy version which caused this to fail cryptically...

EDIT: Left it for now, because its so small...

Did you skip 3 on purpose?

I'm fine with coming back to this when someone has a real need for it.

Same here, fine to come back to this.

Not adding, I can't see a comment that does more than repeat the error message. Added a leading underscore to the slot ID, though, so it is less awkward to keep it in a public header.

Would be good to add a comment here that this one is private for now given uncertainties about the signature.

Same here, fine to come back to this.

Hmm, I think I'd still remove this extra addition of nin, since you only need it for the private case: just use ufunc->nin there. (Of course, no choice for inputs_tup!)

So, remove 2, here.

And remove int nin here.

And change to ufunc->nin here.

Remove addition of nin.

Remove ufunc->nin

Here too