Tagging subscribers to this area: @dotnet/area-system-numerics-tensors
See info in area-owners.md if you want to be subscribed.

i think the build fails are unrelated?

would it make sense to change the implementation of Divide too?

    public static void Divide(ReadOnlySpan<float> x, float y, Span<float> destination) =>
        InvokeSpanScalarIntoSpan<MultiplyOperator_Single>(x, 1 / y, destination);

is there a reason SoftMax calls InvokeSpanScalarIntoSpan manually instead of using Divide?

i just realized that if x and y are very large the difference between x / y and x * (1/y) can be quite big (x * (1/y) could even become zero) so maybe this is not a good idea after all

I assume this will slightly change the output of SoftMax but since float operations aren't consistent across platforms this should not matter.

This is not true. IEEE 754 floating-point is deterministic, by design. This is particularly true for all primitive operations such as +, -, *, /, Sqrt, and a few operations that are listed as "required" by the spec.

The source of such "differences" are typically caused by users improperly opting into features like fast-math, from when the legacy x87 FPU was prominent and users did not correctly set the rounding mode before doing operations (and so were doing 80-bit, not 64-bit or 32-bit operations), or come from using a "recommended" operations (such as Sin) for which the underlying C runtime implementations explicitly opted for perf over accuracy.

If a user wishes for a different result, they can explicitly use the relevant operations themselves. Our own implementations are striving for accuracy first and foremost, particularly when that is trivial to achieve.