feat: Bigtable Authorized Views - Allow checkAndMutate and ReadModifyWriteRow calls on Authorized Views#1464
Merged
danieljbruce merged 43 commits intobigtable-authorized-viewsfrom Sep 23, 2024
Conversation
…github.com/googleapis/nodejs-bigtable into bigtable-authorized-views-initial-refactor
and readWriteModifyRow
…apis/nodejs-bigtable into bigtable-authorized-views-move-row-methods # Conflicts: # src/row.ts # src/table.ts # src/tabular-api-service.ts # src/utils/table.ts # test/table.ts
bhshkh
approved these changes
Sep 5, 2024
danieljbruce
added a commit
that referenced
this pull request
Nov 7, 2024
* Create the new tabular api service file * feat: Bigtable authorized views - move the code over to the TabularApiSurface class (#1463) * Move the constructor over to TabularApiService * Move sampleRowKeys over * Move sampleRowKeys functions over and use promisify * Adjust the proxyquire to work with TabularAPIserv * Move all the ReadRows functionality over * Solve the issue with the is dependency * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Add header for new class * Only include Table mocks that are necessary * Remove TODO * Rename TabularApiService to TabularApiSurface * Change all imports to tabular-api-surface * surface. not service --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * feat: Bigtable Authorized Views - Allow checkAndMutate and ReadModifyWriteRow calls on Authorized Views (#1464) * Move the constructor over to TabularApiService * Move sampleRowKeys over * Move sampleRowKeys functions over and use promisify * Adjust the proxyquire to work with TabularAPIserv * Move all the ReadRows functionality over * Solve the issue with the is dependency * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Add header for new class * Trying out the FilterInformation class * Add a DataUtils module for the shared row function * Outsource functionality of filter to helper * debugging * Adjust proxyquire to include mocks for moved fn * Remove imports * Outsource code to a createRulesUtil function * Change proxyquire to include createRulesUtil * Move increment over to the utils folder * Move the functions into a static class for mocks * Remove mockCreateRules and shorten mock * Stub out FakeRowDataUtil * Remove unused dependencies * Add the functions to work with checkAndMutate and readWriteModifyRow * Fix regressions from the merge * Add documentation for the class * Document the new methods of table * Change the interface of the rowUtils * Pull the generateProperties code out avoid duplica * Move duplicate code out into a getProperties object * Remove console.log * Add method for creating views * Object for making grpc calls for authorized views * Remove import * Update the documentation for the Table * More specific type * Add documentation for each of the functions * Remove TODO * Add headers * Add documentation for new class * Remove imports * Reintroduce before * Remove unused import --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> * feat: Create a view on the instance, not on the table (#1476) * Move view creation from the table class to instanc * feat: Create a view on the instance, not on the table * Add table name parameter to the documentation * Remove Table * feat: Create the unit tests for Authorized Views ensuring that requests are consistent (#1501) * Create a test for authorized views * Set the ReadRows options * Add authorized view test for createreadstream * Pass authorizedViewName along for createReadStream * Add the unit test for getRows * Add the unit test for getRows * Take out the test mock for the request function * Mutate rows request - test setup * Made corrections and src code changes so unit test passes * Fix the readrows test to end the stream Also add a test for insert and make the test easier to debug. * Finish the sampleRowKeys tests * Refactor part of the test for getting request opts * Authorized views for readRows test should be one assert * Add the mock function for request * Make mutate rows use the mockRequest function * Setup the sampleRowKeys setup fn to use mockReq * Add comments to the mocking functions * Fix comments. Move functions to right place * Setup the readModifyWriteRow tests * For createRules test use new mockCallbackRequest You need to call the callback in order to end the operation to createRules. * Use the new mockCallbackRequest function * Get rid of the unused mock function * Create a test file for createRules not got * increment * Add the test for increment * Change the response value for authorized view * Add the console logs and change return type * Flesh out response so that view call works * Exclude appropriate properties using promisify. * Removed console logs * Finished the filter test * Add the view call for filter * Remove only * Change to more readable values * Remove only * Add header to the test * For tests, expect view to be excluded from promis * run linter * Add a few @params and @returns * Add a comment about making requests for auth views * Added more comments - auth views vs table Also added a comment for calling increment * feat: Bigtable authorized views integration tests (#1504) * Create a test for authorized views * Set the ReadRows options * Add authorized view test for createreadstream * Pass authorizedViewName along for createReadStream * Add the unit test for getRows * Add the unit test for getRows * Take out the test mock for the request function * Mutate rows request - test setup * Made corrections and src code changes so unit test passes * Fix the readrows test to end the stream Also add a test for insert and make the test easier to debug. * Finish the sampleRowKeys tests * Refactor part of the test for getting request opts * Authorized views for readRows test should be one assert * Add the mock function for request * Make mutate rows use the mockRequest function * Setup the sampleRowKeys setup fn to use mockReq * Add comments to the mocking functions * Fix comments. Move functions to right place * Setup the readModifyWriteRow tests * For createRules test use new mockCallbackRequest You need to call the callback in order to end the operation to createRules. * Use the new mockCallbackRequest function * Get rid of the unused mock function * Create a test file for createRules not got * increment * Add the test for increment * Change the response value for authorized view * Add the console logs and change return type * Flesh out response so that view call works * Exclude appropriate properties using promisify. * Removed console logs * Finished the filter test * Add the view call for filter * Remove only * Change to more readable values * Remove only * Add header to the test * For tests, expect view to be excluded from promis * run linter * Create the before hook for auth views * Note to self on auth views - TODO * Add an insert statement * Make a test for getRows * Add the test for the getRows function. * Finish ‘should fail when writing to a row not in v * Add another mutate test, preserve table after test Add after hook to make sure table values stay the same. Add test for modifying from different column * Remove TODO * refactor the error message in the test * Add mutate, insert and sampleRowKeys tests * Fix sampleRowKeys test to require a fix upstream * Create samplerowkeys and createreadstream tests * Surround all tests in a try block This allows for better error reporting * Fixed the column filter so it works * Reduce verbosity in tests In many cases, 3 lines can be reduced to one variable. This is crucial for making the integration tests shorter. * Replace verbose data with data variable * Add an insert for multiple rows * Fix the check and mutate test to ignore corrupt d * Add the readModifyWriteRow tests * Add awaits * wrap columnFamily and columnIdInView in brackets * Eliminate the TODO * Remove only * Add key back to filter config option --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Kevin Kim <kevkim@google.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
This PR introduces the architecture required to service
checkAndMutateandreadModifyWriteRowrequests for Authorized views.Background:
this other PR has already been merged towards the effort of supporting data API calls for Authorized Views.
This current PR demonstrates the shape of the architecture that is required for Authorized View requests that correspond to the
checkAndMutateandreadModifyWriteRowcalls currently serviced by calling theRowclass methods. There may be clerical mistakes, but those will be flushed out by writing unit tests and integration tests.Changes:
Row Data Utils:
increment,filterandcreateRulesin theRowclass is moved into new methods inrow-data-utils. To support authorized view calls this has to be done. You see,increment,filterandcreateRuleson the Row class make use of a privatedataproperty in that Row class. For this reason, to supportincrement,filterandcreateRulesfor authorized views we cannot simply createincrement,filterandcreateRulesmethods on theTabularApiSurfaceclass because thendatawould have to be passed into them and we do not wantdatato be part of the method signature. Instead, we moveincrement,filterandcreateRulesfunctionality fromRowto methods in a sharedrow-data-utilsclass. These methods in therow-data-utilsclass will now make requests for both tables and authorized views.Authorized View:
increment,filterandcreateRulesmethods are added to the newAuthorizedViewclass for makingcheckAndMutateandreadModifyWriteRowcalls for an authorized view. These methods should not be exposed onTabularApiSurfacebecause then they will be exposed onTable, but this functionality is already available for tables on theincrement,filterandcreateRulesmethods onRow.createRules,filterandincrementmethods on theAuthorizedViewclass will have the same signature as the methods with the same name on theRowclass except they must also be provided with a row id that the request will be made for.AuthorizedViewclass will store row data for authorized views just like theRowclass has adataproperty that stores data for that row. ThisrowDatavariable inAuthorizedViewstores data for each row it makes a request on behalf of.Row:
RowDataUtilsand replaced with one line of code that delegates the call.Table:
TabularApiSurfaceconstructor is protected.TabularApiSurface:
test/Row.ts
Next Steps:
increment,createRulesandfiltermethods now available onAuthorizedView. Make sure that these methods pass the proper parameters along and that they target an authorized view.TabularApiSurfaceclass.