test: 2.x integration tests #112

prawilny · 2021-10-22T11:53:21Z

This change is

mwalkiewicz

Reviewed 10 of 14 files at r1, 9 of 9 files at r2, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @prawilny)

bigtable-hbase-mirroring-client-2.x-parent/bigtable-hbase-mirroring-client-2.x/src/test/java/com/google/cloud/bigtable/mirroring/hbase2_x/TestMirroringAsyncTable.java, line 653 at r2 (raw file):

  @Test
  public void testConditionalWriteHappensWhenSecondaryErred()

This change seems unrelated, I'm not sure if it tests something caught in integrations, or is it here by accident.

prawilny

Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @mwalkiewicz)

bigtable-hbase-mirroring-client-2.x-parent/bigtable-hbase-mirroring-client-2.x/src/test/java/com/google/cloud/bigtable/mirroring/hbase2_x/TestMirroringAsyncTable.java, line 653 at r2 (raw file):

Previously, mwalkiewicz (Mateusz Walkiewicz) wrote…

This change seems unrelated, I'm not sure if it tests something caught in integrations, or is it here by accident.

See

  // TODO(mwalkiewicz): fix
  @Ignore("Fails for unknown reasons")
  @Test
  public void testCheckAndMutateSecondaryErrors()

It's a unit test accompanying this failing integration test.

The base branch was changed.

mwalkiewicz

Reviewed 7 of 7 files at r3, 1 of 1 files at r4, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @mwalkiewicz)

… review comments (googleapis#3347) * chore: revert review comments * feat: add MirroringOperationException exception markers (#125) * feat: concurrent writes in MirroringBufferedMutator (#80) * refactor: implement multiple argument operations on MirroringAsyncTable with specific operations rather than batch() (#75) * feat: implement MirroringAsyncTable#getName() (#132) * feat: use Logger rather than stdout in DefaultMismatchDetector (#128) * feat: synchronous writes (#88) * fix: implement heapSize method for RowCell (#111) * feat: FlowController accounts for memory usage (#137) * refactor: remove Configuration as a base of MirroringConfiguration (#127) * feat: MirroringAsyncBufferedMutator (#81) * refactor: rename WRITE_MISMATCH to SECONDARY_WRITE_ERROR (#138) * fix: BufferedMutator close() waits for all secondary flushes to finish (#110) * feat: 2.x reads sampling (#114) * refactor: make MirroringResultScanner synchronize on itself rather than MirroringTable (#134) * ConcurrentBufferedMutator integration tests (#135) * feat: add synchronous MirroringConnection to 2.x (#109) * fix: MirroringConnection in 2.x failed to compile (#139) * fix: fix BufferedMutator ITs (#140) * feat: run 1.x integration tests on MirroringConnection etc. from 2.x (#108) * feat: 2.x - rewrite Increment and Append as Put in batch (#116) * fix: fix build (#142) * refactor: minor fixes after review (#117) * feat: MirroringAsyncTable#getScanner() (#58) * test: 2.x integration tests (#112) * feat: implement MirroringAsyncBufferedMutatorBuilder (#144) * feat: log rows and values in DefaultMismatchDetector (#129) * fix: ITs - add expected parameter to MismatchDetectors (#153) * fix: force Append and Increment to return results and discard that result before returning it to user (#136) * fix: review fixes in utils * fix: review fixes in BufferedMutator * fix: review fixes in Faillog * fix: fixed reference counting * fix: review fixes in FlowController * fix: review fixes in metrics * fix: review fixes in verification * fix: Review fixes in MirroringTable * fix: review fixes in HBase 2.x client * fix: fixes in ITs * feat: MirrorinAsyncTable: scan(), scanAll() (#131) * fix: review fixes in tests * feat: MirroringConnection: timeout in close() and abort() (#133) * feat: better mismatch detection of scan results (#130) * feat: quickstart (#105) * fix: 2.x scan ITs (#158) * fix: DefaultMismatchDetector tests (#157) * fix: ConcurrentBufferedMutator waits for both flushes to finish before closing (#161) * fix: additional minor fixes after review (#163) * fix: BufferedMutator review fixes (#164) - Simplify #flush(). - Add javadocs. - (sequential) Fix flush() exception handling. - (sequential) Move error handling to a separate inner class. * fix: PR fixes * fix: report zeroed error metrics after successful operations * fix: prepend MismatchDetectorCounter with Test to better reflect its purpose * feat: Client-side timestamping (#165) * fix: reduce timeout in TestBlocking to make the tests run faster * fix: asyncClose -> closePrimaryAndScheduleSecondaryClose * fix: remove unused Batcher#throwBatchDataExceptionIfPresent * fix: remove unused Comparators#compareRows * fix: extract failedReads from MatchingSuccessfulReadsResults to reduce confusion * feat: remove unused MirroringTracer from FailedMutationLogger * fix: MirroringAsyncBufferedMutator - test if failed mutation is passed to secondary write error consumer * fix: TestMirroringAsyncTableInputModification typo fix * fix: describe user flush() in Buffered Mutator in quickstart * fix: MirroringBufferedMutator - move flush threshold from BufferedMutations to FlushSerializer * refactor: MirroringBufferedMutator#close() - use AccumulatedExceptions insteand of List<Exception> * BufferedMutator - add close timeout * AsyncBufferedMutator - add close timeout * fix: remove stale addSecondaryMutation comment * fix: add a comment that addSecondaryMutation handles failed writes * fix: unify implementations of flushBufferedMutatorBeforeClosing * fix: BufferedMutator - throw exceptions on close * fix: BufferedMutator - add comment explaining that chain of flush operations is created * fix: BufferedMutator - clarify comments * fix: Concurrent BufferedMutator - fix throwFlushExceptionIfAvailable * fix: explain why flush is called in Sequential BufferedMutator test * fix: TestConcurrentMirroringBufferedMutator - make waiting for calls explicit * refactor: BufferedMutator rename scheduleFlushAll() to scheduleFlush() * refactor: make FlushSerializer non-static * fix: BufferedMutator - use HierarchicalReferenceCounter * feat: Add MirroringConnection constructor taking MirroringConfiguration * refactor: move releaseReservations to finally * fix: use less convoluted example in lastFlushFutures description * fix: merge small Timeestamper files into a single file * fix: add a comment explaining which exceptions are forwarded to the user and why in SequentialMirroringBufferedMutator * fix: use UnsupportedOperationException instead of RuntimeException when forbidden mutation type is encountered * fix: add comment explaining why batch is complicated * fix: add a TODO to implement point writes without batch Co-authored-by: Mateusz Walkiewicz <mwalkiewicz@unoperate.com> Co-authored-by: Adam Czajkowski <prawilny@unoperate.com> Co-authored-by: Kajetan Boroszko <kajetan@unoperate.com>

prawilny requested a review from mwalkiewicz October 22, 2021 11:53

prawilny mentioned this pull request Oct 22, 2021

feat: implement MirroringAsyncTable#getScanner() #58

Merged

prawilny force-pushed the ac/integration-tests-2.x/1 branch 2 times, most recently from c25ad7c to ce6e62a Compare October 22, 2021 12:18

mwalkiewicz linked an issue Oct 25, 2021 that may be closed by this pull request

2.x: client integration tests #90

Closed

prawilny force-pushed the ac/scan/1 branch from cbafc05 to f048a74 Compare November 1, 2021 14:28

prawilny force-pushed the ac/integration-tests-2.x/1 branch 2 times, most recently from cbafc05 to 3e19c19 Compare November 1, 2021 14:35

prawilny changed the title ~~test: first batch of 2.x integration tests~~ test: 2.x integration tests Nov 1, 2021

mwalkiewicz previously approved these changes Nov 5, 2021

View reviewed changes

prawilny commented Nov 9, 2021

View reviewed changes

prawilny force-pushed the ac/scan/1 branch from f048a74 to 12176b1 Compare November 10, 2021 12:06

prawilny changed the base branch from ac/scan/1 to master November 10, 2021 12:43

prawilny force-pushed the ac/integration-tests-2.x/1 branch from 97c5a4f to 8fb010a Compare November 10, 2021 12:49

test: 2.x mirroring integration tests

1dac7e4

prawilny force-pushed the ac/integration-tests-2.x/1 branch from 8fb010a to 1dac7e4 Compare November 10, 2021 14:10

ignore failing test

75042eb

prawilny force-pushed the ac/integration-tests-2.x/1 branch from 7747bf3 to 75042eb Compare November 23, 2021 14:36

mwalkiewicz approved these changes Nov 23, 2021

View reviewed changes

prawilny merged commit b91ef85 into master Nov 23, 2021

dopiera pushed a commit that referenced this pull request Dec 2, 2021

test: 2.x integration tests (#112)

5759741

dopiera pushed a commit that referenced this pull request May 11, 2022

test: 2.x integration tests (#112)

6ff2ff1

dopiera pushed a commit that referenced this pull request May 11, 2022

test: 2.x integration tests (#112)

c78194e

dopiera pushed a commit that referenced this pull request May 11, 2022

test: 2.x integration tests (#112)

2fdd1d0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

test: 2.x integration tests #112

test: 2.x integration tests #112

Uh oh!

prawilny commented Oct 22, 2021 •

edited by dopiera

Loading

Uh oh!

mwalkiewicz left a comment

Uh oh!

prawilny left a comment

Uh oh!

mwalkiewicz left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

test: 2.x integration tests #112

test: 2.x integration tests #112

Uh oh!

Conversation

prawilny commented Oct 22, 2021 • edited by dopiera Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mwalkiewicz left a comment

Choose a reason for hiding this comment

Uh oh!

prawilny left a comment

Choose a reason for hiding this comment

Uh oh!

mwalkiewicz left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

prawilny commented Oct 22, 2021 •

edited by dopiera

Loading