Avoid user key copying for Get/Put/Write with user-timestamp#5502
Avoid user key copying for Get/Put/Write with user-timestamp#5502riversand963 wants to merge 7 commits intofacebook:masterfrom
Conversation
|
Please update the title to mention user timestamp. |
46d31ac to
bd1fc54
Compare
|
Ping @siying |
facebook-github-bot
left a comment
There was a problem hiding this comment.
@elipoz has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
bd1fc54 to
d38c5b2
Compare
|
@riversand963 has updated the pull request. Re-import the pull request |
facebook-github-bot
left a comment
There was a problem hiding this comment.
@riversand963 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
d38c5b2 to
8a5e06d
Compare
|
@riversand963 has updated the pull request. Re-import the pull request |
facebook-github-bot
left a comment
There was a problem hiding this comment.
@riversand963 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
|
@riversand963 has updated the pull request. Re-import the pull request |
facebook-github-bot
left a comment
There was a problem hiding this comment.
@riversand963 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
|
@riversand963 merged this pull request in ae152ee. |
…k#5502) Summary: In previous facebook#5079, we added user-specified timestamp to `DB::Get()` and `DB::Put()`. Limitation is that these two functions may cause extra memory allocation and key copy. The reason is that `WriteBatch` does not allocate extra memory for timestamps because it is not aware of timestamp size, and we did not provide an API to assign/update timestamp of each key within a `WriteBatch`. We address these issues in this PR by doing the following. 1. Add a `timestamp_size_` to `WriteBatch` so that `WriteBatch` can take timestamps into account when calling `WriteBatch::Put`, `WriteBatch::Delete`, etc. 2. Add APIs `WriteBatch::AssignTimestamp` and `WriteBatch::AssignTimestamps` so that application can assign/update timestamps for each key in a `WriteBatch`. 3. Avoid key copy in `GetImpl` by adding new constructor to `LookupKey`. Test plan (on devserver): ``` $make clean && COMPILE_WITH_ASAN=1 make -j32 all $./db_basic_test --gtest_filter=Timestamp/DBBasicTestWithTimestampWithParam.PutAndGet/* $make check ``` If the API extension looks good, I will add more unit tests. Some simple benchmark using db_bench. ``` $rm -rf /dev/shm/dbbench/* && TEST_TMPDIR=/dev/shm ./db_bench -benchmarks=fillseq,readrandom -num=1000000 $rm -rf /dev/shm/dbbench/* && TEST_TMPDIR=/dev/shm ./db_bench -benchmarks=fillrandom -num=1000000 -disable_wal=true ``` Master is at a78503b. ``` | | readrandom | fillrandom | | master | 15.53 MB/s | 25.97 MB/s | | PR5502 | 16.70 MB/s | 25.80 MB/s | ``` Pull Request resolved: facebook#5502 Differential Revision: D16340894 Pulled By: riversand963 fbshipit-source-id: 51132cf792be07d1efc3ac33f5768c4ee2608bb8
In previous #5079, we added user-specified timestamp to
DB::Get()andDB::Put(). Limitation is that these two functions may cause extra memory allocation and key copy. The reason is thatWriteBatchdoes not allocate extra memory for timestamps because it is not aware of timestamp size, and we did not provide an API to assign/update timestamp of each key within aWriteBatch.We address these issues in this PR by doing the following.
timestamp_size_toWriteBatchso thatWriteBatchcan take timestamps into account when callingWriteBatch::Put,WriteBatch::Delete, etc.WriteBatch::AssignTimestampandWriteBatch::AssignTimestampsso that application can assign/update timestamps for each key in aWriteBatch.GetImplby adding new constructor toLookupKey.Test plan (on devserver):
If the API extension looks good, I will add more unit tests.
Some simple benchmark using db_bench.
Master is at a78503b.