-
Notifications
You must be signed in to change notification settings - Fork 62
Description
Please run down the following list and make sure you've tried the usual "quick fixes":
- Search the issues already opened: https://github.com/googleapis/nodejs-bigtable/issues
- Search the issues on our "catch-all" repository: https://github.com/googleapis/google-cloud-node
- Search StackOverflow: http://stackoverflow.com/questions/tagged/google-cloud-platform+node.js
If you are still having issues, please be sure to include as much information as possible:
Environment details
- OS: macos 10.14.6, linux (on GKE)
- Node.js version: all (discovered while on
v8.16.2) - npm version: all (discovered while on
6.4.1) @google-cloud/bigtableversion: all (discovered via1.0.1)
Steps to reproduce
- write a very large number of increments to a single row (a simple loop that sends increment mutations should work)
- run
row.exists()against the row, monitor memory usage
Expected result
row.exists() should query enough information to accurately determine if the row exists, but not pull all versions of all cells within the row.
Actual result
With very active rows which are creating many versions due to increment operations, row.exists() fetches every version of the cell, consuming a massive amount of memory (potentially crashing node) and taking a long time to execute (while waiting for the versions) before finally resolving with the boolean result. This can cause server failures and API call timeouts.
Analysis
row.exists() fetches the entire row, including all versions of all cells and doesn’t accept an options object, so it cannot be made to limit the amount of data fetched.
Specifically, row.exists:
- accepts no parameters/options
- forces
gaxOptionsto an empty object - calls
row.getMetadata - which calls
table.getRows - which calls
createReadStream, then pipes all results in-memory to be concatenated in a single array