Skip to content

chore(test): Add test to verify that query ranges work as expected#445

Merged
igorbernstein2 merged 2 commits intogoogleapis:masterfrom
igorbernstein2:read-range-it
Oct 12, 2020
Merged

chore(test): Add test to verify that query ranges work as expected#445
igorbernstein2 merged 2 commits intogoogleapis:masterfrom
igorbernstein2:read-range-it

Conversation

@igorbernstein2
Copy link
Copy Markdown
Contributor

@igorbernstein2 igorbernstein2 commented Oct 6, 2020

No description provided.

@igorbernstein2 igorbernstein2 requested a review from a team October 6, 2020 20:57
@google-cla google-cla Bot added the cla: yes This human has signed the Contributor License Agreement. label Oct 6, 2020
@codecov
Copy link
Copy Markdown

codecov Bot commented Oct 6, 2020

Codecov Report

❗ No coverage uploaded for pull request base (master@9e693b4). Click here to learn what that means.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff            @@
##             master     #445   +/-   ##
=========================================
  Coverage          ?   80.63%           
  Complexity        ?     1115           
=========================================
  Files             ?      105           
  Lines             ?     6941           
  Branches          ?      369           
=========================================
  Hits              ?     5597           
  Misses            ?     1146           
  Partials          ?      198

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9e693b4...30db5c2. Read the comment docs.

@ghost
Copy link
Copy Markdown

ghost commented Oct 7, 2020

Hi Igor,
Rohan Ramnarine asked me to add some data to help reproduce the issue I'm running into where the CLOSED end range data is not being returned. Here is the integration test that reproduces the issue I'm seeing using the same row key data we have in BigTable and the range key format we use when querying using the client.

  @Test
  public void rangeQueries2() {
    BigtableDataClient client = testEnvRule.env().getDataClient();
    String tableId = testEnvRule.env().getTableId();
    String familyId = testEnvRule.env().getFamilyId();
    String startRange = "login|9223370437392774807";
    String endRange = "login|9223370437392775807";

    String keyA = "login|9223370437392775807|uniqueaa-4bff-44e4-8f55-241f733f8ae";
    String keyZ = "login|9223370437392775807|uniqueda-f31a-4f16-b1ed-c90288d68d62";

    long timestampMicros = System.currentTimeMillis() * 1_000;

    client.bulkMutateRows(
            BulkMutation.create(tableId)
                    .add(RowMutationEntry.create(keyA).setCell(familyId, "", timestampMicros, "A"))
                    .add(RowMutationEntry.create(keyZ).setCell(familyId, "", timestampMicros, "Z")));

    Row expectedRowA =
            Row.create(
                    ByteString.copyFromUtf8(keyA),
                    ImmutableList.of(
                            RowCell.create(
                                    testEnvRule.env().getFamilyId(),
                                    ByteString.copyFromUtf8(""),
                                    timestampMicros,
                                    ImmutableList.<String>of(),
                                    ByteString.copyFromUtf8("A"))));

    Row expectedRowZ =
            Row.create(
                    ByteString.copyFromUtf8(keyZ),
                    ImmutableList.of(
                            RowCell.create(
                                    testEnvRule.env().getFamilyId(),
                                    ByteString.copyFromUtf8(""),
                                    timestampMicros,
                                    ImmutableList.<String>of(),
                                    ByteString.copyFromUtf8("Z"))));

    // Closed/Closed
    assertThat(
            ImmutableList.copyOf(
                    client.readRows(
                            Query.create(tableId)
                                    .range(ByteStringRange.unbounded().startClosed(startRange).endClosed(endRange)))))
            .containsExactly(expectedRowA, expectedRowZ);
  }

@product-auto-label product-auto-label Bot added the api: bigtable Issues related to the googleapis/java-bigtable API. label Oct 7, 2020
@igorbernstein2
Copy link
Copy Markdown
Contributor Author

igorbernstein2 commented Oct 8, 2020

Thanks for putting together the example. However that test failure is expected.
The range [login|9223370437392774807, login|9223370437392775807] does not include the key login|9223370437392775807|uniqueda-f31a-4f16-b1ed-c90288d68d62.

This is because, lexicographically, prefix will always come before prefixSuffix.

@ghost
Copy link
Copy Markdown

ghost commented Oct 8, 2020 •
edited by ghost
Loading

Is there a way to construct the end range in the query such that it matches any row key containing that prefix? The login|<reverse timestamp> is all that we have and the unique id suffix is not known. The only thing I've seen work so far is to change the reverse timestamp itself e.g. use 9223370437392775808

@igorbernstein2
Copy link
Copy Markdown
Contributor Author

igorbernstein2 commented Oct 8, 2020 •
edited
Loading

ByteStringRange has a method called prefix which constructs a range that includes all keys with the supplied prefix. So you can do something like:

ByteStringRange startRange = ByteStringRange.prefix(....);
ByteStringRange endRange = ByteStringRange.prefix(....);

Preconditions.checkState(startRange.getStartBound() == BoundType.CLOSED);
Preconditions.checkState(endRange.getEndBound() == BoundType.OPEN);

ByteStringRange combinedRange = ByteStringRange.create(startRange.getStart(), endRange.getEnd());

If you have the inclination, I would also appreciate a PR to add a utility method ByteStringRange ByteStringRange#getBound(ByteStringRange ranges...). The implementation would do something similar to RowSetUtil#getBound(). Or something else along those lines that would let you compose 2 prefix ranges

@igorbernstein2 igorbernstein2 merged commit 1fa6f62 into googleapis:master Oct 12, 2020
ad548 pushed a commit to ad548/java-bigtable that referenced this pull request Mar 13, 2021
@igorbernstein2 @ad548
chore(test): Add test to verify that query ranges work as expected (g…
2e48503 
…oogleapis#445)

* chore(test): Add test to verify that query ranges work as expected

* format
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@kolea2 kolea2 kolea2 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

api: bigtable Issues related to the googleapis/java-bigtable API. cla: yes This human has signed the Contributor License Agreement.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@igorbernstein2 @kolea2