Skip to content

clp-s: Incorrect rounding in double_as_int when converting floating-point numbers into integers for comparison against integer values. #1375

@gibber9809

Description

@gibber9809

Bug

The double_as_int function is used in the AST code to convert floating-point literals into integers such that they can be compared against integer values.

Nominally this function is meant to do the following sorts of conversions:

1 < 1.1 -> 1 < 2
1 > 0.9 -> 1 > 0

Where depending on the kind of operations being performed we may need to compare the integer value against the floor or the ceiling of the integer literal.

Unfortunately, the double_as_int function has a bug where it will always return the floor of the floating-point literal for all operations besides == and !=.

This results in in correct behaviour in the following situations:

  1. When filtering against a field like {"n": 1} the query n < 1.1 will erroneously not match the record, and the query n >= 1.1 erroneously will match the record.

  2. When filtering against the timestamp range index when it contains a range like [1.1, 1.5] a query like timestamp < 1.4 will erroneously not match the archive's time range. (This is because the timestamp range index evaluation code has some behaviour where it will convert float literals to integers before converting them back to floats to compare against a double-encoded time range, which is nominally correct, but overly complicated).

CLP version

v0.5.1

Environment

ubuntu focal

Reproduction steps

Simplest reproduction is:

  1. Ingest {"n": 1}
  2. Observe that the query n < 1.1 returns no results

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions