Skip to content

Fix rocksdb prefix Seek performance.#4319

Merged
petermattis merged 1 commit intocockroachdb:masterfrom
petermattis:pmattis/rocksdb-prefix-seek
Feb 11, 2016
Merged

Fix rocksdb prefix Seek performance.#4319
petermattis merged 1 commit intocockroachdb:masterfrom
petermattis:pmattis/rocksdb-prefix-seek

Conversation

@petermattis
Copy link
Copy Markdown
Collaborator

Set ReadOptions.iterate_upper_bound when creating "prefix"
iterators. This avoids pathological performance when seeking into a
rocksdb MemTable which contains large expanses of deletion tombstones.

Added a benchmark which demonstrates the pathological performance of the
old code.

name             old time/op  new time/op  delta
MVCCPutDelete-8   185µs ± 8%    11µs ± 3%  -94.11%  (p=0.000 n=10+10)

Fixes #4196.

Review on Reviewable

@dt
Copy link
Copy Markdown
Contributor

dt commented Feb 11, 2016

This will probably need a quick rebase since #4308 merged


Review status: 0 of 11 files reviewed at latest revision, all discussions resolved, some commit checks failed.


Comments from the review on Reviewable.io

// When we're performing a prefix scan, we want to limit the scan
// to the keys that have the matching prefix. Prefix in this case
// refers to an exact match on the non-timestamp portion of a
// key. We do this be constructing an encoded mvcc key which has a
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/be/by/

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@petermattis petermattis force-pushed the pmattis/rocksdb-prefix-seek branch from fd06855 to e9c64c5 Compare February 11, 2016 15:42
@petermattis
Copy link
Copy Markdown
Collaborator Author

This PR also has a big impact on the {Update,Delete}1 benchmarks. Presumably they were also encountering the same performance degradation.

name                   old time/op    new time/op    delta
Insert1_Cockroach-8       312µs ± 1%     317µs ± 2%   +1.58%         (p=0.001 n=9+10)
Insert10_Cockroach-8      597µs ± 2%     590µs ± 2%   -1.07%        (p=0.043 n=10+10)
Insert100_Cockroach-8    3.14ms ± 1%    3.06ms ± 1%   -2.57%         (p=0.000 n=9+10)
Update1_Cockroach-8       752µs ± 1%     602µs ± 1%  -19.92%        (p=0.000 n=10+10)
Update10_Cockroach-8     3.51ms ± 1%    3.55ms ± 1%   +1.00%        (p=0.000 n=10+10)
Update100_Cockroach-8    31.0ms ± 1%    31.6ms ± 1%   +1.93%         (p=0.000 n=10+8)
Delete1_Cockroach-8      1.04ms ±13%    0.61ms ± 1%  -40.70%         (p=0.000 n=10+9)
Delete10_Cockroach-8     1.50ms ± 9%    1.26ms ± 1%  -16.40%        (p=0.000 n=10+10)
Delete100_Cockroach-8    6.46ms ± 1%    6.61ms ± 2%   +2.29%         (p=0.000 n=9+10)
Scan1_Cockroach-8         135µs ± 1%     135µs ± 1%     ~           (p=0.315 n=10+10)
Scan10_Cockroach-8        166µs ± 2%     167µs ± 1%   +0.82%         (p=0.013 n=10+9)
Scan100_Cockroach-8       421µs ± 2%     431µs ± 2%   +2.40%        (p=0.005 n=10+10)

Set ReadOptions.iterate_upper_bound when creating "prefix"
iterators. This avoids pathological performance when seeking into a
rocksdb MemTable which contains large expanses of deletion tombstones.

Added a benchmark which demonstrates the pathological performance of the
old code.

name             old time/op  new time/op  delta
MVCCPutDelete-8   185µs ± 8%    11µs ± 3%  -94.11%  (p=0.000 n=10+10)

Fixes cockroachdb#4196.
@petermattis petermattis force-pushed the pmattis/rocksdb-prefix-seek branch from e9c64c5 to 20cb646 Compare February 11, 2016 18:35
@petermattis
Copy link
Copy Markdown
Collaborator Author

Not sure what is up with the circleci failure. I did notice that an unrelated changed had snuck into my GLOCKFILE. Perhaps that was it.

Anyone want to stamp this with an LGTM?

@tbg
Copy link
Copy Markdown
Member

tbg commented Feb 11, 2016

LGTM. I hope you've seen Igor's comment which was on a commit that you've since replaced.

@tbg
Copy link
Copy Markdown
Member

tbg commented Feb 11, 2016

Reviewed 4 of 11 files at r1, 6 of 7 files at r2, 1 of 1 files at r3.
Review status: all files reviewed at latest revision, 1 unresolved discussion.


storage/engine/rocksdb_test.go, line 614 [r1] (raw file):
Still feels funny that this worked. Put a TODO for me please (I'm assuming you don't want to take 5 and figure it out).


Comments from the review on Reviewable.io

@petermattis
Copy link
Copy Markdown
Collaborator Author

Review status: all files reviewed at latest revision, 1 unresolved discussion.


storage/engine/rocksdb_test.go, line 614 [r1] (raw file):
The txn parameter is ignored if the timestamp is "zero". Changing to a non-zero timestamp and we see the error you expect.


Comments from the review on Reviewable.io

petermattis added a commit that referenced this pull request Feb 11, 2016
@petermattis petermattis merged commit 97d8862 into cockroachdb:master Feb 11, 2016
@petermattis petermattis deleted the pmattis/rocksdb-prefix-seek branch February 11, 2016 18:57
@tbg
Copy link
Copy Markdown
Member

tbg commented Feb 11, 2016

Gotcha. I may rest in piece. Thanks for looking.

On Thu, Feb 11, 2016 at 1:57 PM Peter Mattis notifications@github.com
wrote:

Review status: all files reviewed at latest revision, 1 unresolved

discussion.

storage/engine/rocksdb_test.go, line 614 [r1]
https://reviewable.io:443/reviews/cockroachdb/cockroach/4319#-KAFjXC4YePWHJSqPgEf-r1-614:-KAGfHbe0bmeR7BQb7fK:2016351573
(raw file

if err := MVCCPut(rocksdb, nil, key, zeroTS, value, txn1); err != nil {
):
The txn parameter is ignored if the timestamp is "zero". Changing to a

non-zero timestamp and we see the error you expect.

Comments from the review on Reviewable.io
https://reviewable.io:443/reviews/cockroachdb/cockroach/4319

Reply to this email directly or view it on GitHub
#4319 (comment)
.

@bdarnell
Copy link
Copy Markdown
Contributor

The unrelated GLOCKFILE change is because the docker API has a dependency hidden behind a build tag that glock save can't see (#4282). For now it is only safe to run glock save from linux (build/builder.sh glock save will work); if you run it from a mac you'll have to manually tweak the output.

@tamird
Copy link
Copy Markdown
Contributor

tamird commented Feb 11, 2016

See robfig/glock#29

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Progressively degrading performance while running block writer example

5 participants