Wasted bit in random float32 generation #17478
Description
float64 is supposed to have 53 significand bits, and if I print random float64 values in [0, 1) with 54 fractional bits (i.e., one more), then I do see always 0 at the 54th bit and 0 or 1 at the 53rd bit:
0.110111010110001101011111111010011110010101011101001100 <class 'numpy.float64'>
0.110110101111010100011011000010011101000111011010110100 <class 'numpy.float64'>
0.110011111001111100111010101111100001000111001011110010 <class 'numpy.float64'>
0.010000101110111000100110000100111111000100100100100000 <class 'numpy.float64'>
0.000100111100001101010111111110100010101101100111111100 <class 'numpy.float64'>
0.111100100100101110010100110101011100110010001000100010 <class 'numpy.float64'>
0.100111010010000101110011110001111001101010010101001010 <class 'numpy.float64'>
0.000000001010110001101000101110001100100111101110001100 <class 'numpy.float64'>
0.111010010001000001110001110111111101011000011111110000 <class 'numpy.float64'>
0.111111000001110000010100100111110010000111101000000000 <class 'numpy.float64'>
But if I try the same with float32, which is supposed to have 24 significand bits, then not only the 25th bit is always 0 but also the 24th is always 0:
0.1101100101111100101111000 <class 'numpy.float32'>
0.0100100101001010101110100 <class 'numpy.float32'>
0.0111101010100010111111000 <class 'numpy.float32'>
0.1101000001001100000110000 <class 'numpy.float32'>
0.1110011010011100010010000 <class 'numpy.float32'>
0.0001010100011000101011100 <class 'numpy.float32'>
0.1010100001010011110001000 <class 'numpy.float32'>
0.0111000000110011000111100 <class 'numpy.float32'>
0.0101100000111110001100000 <class 'numpy.float32'>
0.1101000101010101000010000 <class 'numpy.float32'>
And I can make the 24th bit 1 by adding 2-24:
0.1101100101111100101111010 <class 'numpy.float32'>
0.0100100101001010101110110 <class 'numpy.float32'>
0.0111101010100010111111010 <class 'numpy.float32'>
0.1101000001001100000110010 <class 'numpy.float32'>
0.1110011010011100010010010 <class 'numpy.float32'>
0.0001010100011000101011110 <class 'numpy.float32'>
0.1010100001010011110001010 <class 'numpy.float32'>
0.0111000000110011000111110 <class 'numpy.float32'>
0.0101100000111110001100010 <class 'numpy.float32'>
0.1101000101010101000010010 <class 'numpy.float32'>
So it's not an issue with float32 but with the random number generator. It is not producing random float32 as well as it could.
Reproducing code example:
import numpy as np
def bits(x, n):
"""String for x in [0, 1) with n fractional bits."""
return f'0.{int(x * 2**n):0{n}b}'
rng = np.random.default_rng(13)
f64 = rng.random(10, dtype=np.float64)
for x in f64:
print(bits(x, 54), type(x))
print()
f32 = rng.random(10, dtype=np.float32)
for x in f32:
print(bits(x, 25), type(x))
print()
for x in f32:
x += np.float32(2**-24)
print(bits(x, 25), type(x))NumPy/Python version information:
>>> import sys, numpy; print(numpy.__version__, sys.version)
1.19.2 3.9.0 (tags/v3.9.0:9cf6752, Oct 5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)]