Codecov Report

Merging #2763 (175c52f) into main (b333fe4) will decrease coverage by 0.04%.
Report is 2 commits behind head on main.
The diff coverage is 91.66%.

Annoyingly, I've run into the problem of specifying scalar-promotion rules again. We kicked the bucket down the road somewhat, but I think it's time to robustly solve this. At this stage, I think we can just replicate NumPy's rules for Python scalars, because Python scalars are expected to be non-kernel outputs (i.e. static).

However, if we have places where this doesn't hold, we might need to think about implementing NEP-50 (https://numpy.org/neps/nep-0050-scalar-promotion.html#motivation-and-scope) which proposes to remove the value-dependence that we're worried about in Awkward

I also removed the __cast__ default that casts str to length-1 arrays of strings.

This might be needed for some commonsense broadcasting cases. For instance,

some_array["new_field"] = "a single string"

may need to be presented as ak.Array(["a single string"]) so that broadcasting sees this as one atom to duplicate len(some_array) times.

@jpivarski the failing test in 3.12 on windows is due to the change in scalar promotion; Awkward promotes scalars to the ArrayBuilder default type (int64) rather than the NumPy default for the platform. I don't think that is necessarily a problem, though I wonder whether ArrayBuilder should default to the NumPy default dtype for the current platform rather than hard-coding int64. Any strong feelings here? The real question, though, is are you OK with changing this behavior such that untyped scalars go via ArrayBuilder rather than using asarray?

We should be interpreting Python int as int64. That's a widespread convention; only NumPy on Windows doesn't do it and I think that's a mistake.

If the scalars are really untyped (i.e. Python int or float), they can go via ArrayBuilder, but NumPy-typed scalars (i.e. np.int32 and np.float64) would lose their types if they went through ArrayBuilder. ArrayBuilder takes any integer as int64 and any floating point number as float64; it just doesn't have other input types compiled into it (which would dramatically increase its surface area, since its number of methods has to be equal to the number of classes, so handling $n$ input types means $n^2$ method-class pairs).

We should be interpreting Python int as int64. That's a widespread convention; only NumPy on Windows doesn't do it and I think that's a mistake.

Fine! Although, now that's also a list that includes Emscripten whose default is 32 bit.

If the scalars are really untyped (i.e. Python int or float), they can go via ArrayBuilder, but NumPy-typed scalars (i.e. np.int32 and np.float64) would lose their types if they went through ArrayBuilder.

Yes, the np.generic scalars are handled via np.asarray :)

Emscripten whose default is 32 bit

I ran into JavaScript being limited to 32-bit in the Kaitai web tool. That was quite a surprise.

Okay, there are two exceptions: NumPy in Windows and JavaScript/Emscripten.

I just changed strings_as_characters to a new literal string_policy, and renamed scalar_policy to primitive_policy.

This allows finer control of strings, and brings some symmetry to the API