Skip to content

Lua to Redis integer conversion loses precision #1118

@ssimeonov

Description

@ssimeonov

Lua has a single number type that follows the IEEE 754 binary64 specification, which allows for 52 bits of precision. This makes the continuous range of positive integers that can be represented without loss of precision [1, (2^53) - 1].

When converting numbers from Lua to Redis, Redis uses default stringification which outputs in scientific notation with limited precision. This has the effect of losing precision for large integers, violating the Redis promise in the scripting documentation:

This conversion between data types is designed in a way that if a Redis type is converted into a Lua type, and then the result is converted back into a Redis type, the result is the same as of the initial value.

Because Redis has to stringify numbers in decimal notation, some precision loss for non-integer floating point numbers is to be expected. (The scripting documentation should probably be updated to point this out.) However, losing precision for integers that can be represented exactly in IEEE 754 is a bug that can have significant impact and can be very difficult to track down. The larger the integers, the larger the precision loss.

To reproduce, put the following in an Rspec suite:

it 'demonstrates Lua to Redis conversion precision loss' do
  redis = Redis.new

  val = 1.0 * ((2**53) - 1)

  lua = <<-EOS
      local value = (2^53) - 1
      local hashData = {
              "defaultPrecision",
              value,
              "extendedPrecision",
              string.format("%.0f", value)
          }
      redis.call("HMSET", KEYS[1], unpack(hashData))
      return redis.call("HGETALL", KEYS[1])
  EOS

  result = redis.eval(lua, keys: %w("test:key"))

  result[2].should == 'extendedPrecision'
  result[3].to_f.should == val

  result[0].should == 'defaultPrecision'
  result[1].to_f.should == val
end

This will generate the following HMSET command:

1369104793.981944 [0 lua] "HMSET" "test:key" "defaultPrecision" "9.007199254741e+15" "extendedPrecision" "9007199254740991"

It will fail with the following message:

     Failure/Error: result[1].to_f.should == val
       expected: 9007199254740991.0
            got: 9007199254741000.0 (using ==)

The suggested fix is to check whether a number is an integer and, in that case, stringify using full precision as opposed to scientific notation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions