Skip to content

storage: Do not create nil data pointers in rocksdb slices#45759

Merged
knz merged 1 commit intocockroachdb:masterfrom
miretskiy:empty_slice
Mar 6, 2020
Merged

storage: Do not create nil data pointers in rocksdb slices#45759
knz merged 1 commit intocockroachdb:masterfrom
miretskiy:empty_slice

Conversation

@miretskiy
Copy link
Copy Markdown
Contributor

@miretskiy miretskiy commented Mar 5, 2020

Fixes #45524

Libroach's DBSlice data type gets converted to the rocksdb::Slice
whenever we need to access underlying rocksdb (iterators, get/set, etc).

However, internally, rocksdb::Slice asserts that the underlying data
pointer may not be null in the rocksdb::Slice::compare as observed in #45524

This change modifies our rocksdb bridge code to never generate
DBSlices with nullptr data.

Release notes: None

@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

Copy link
Copy Markdown
Contributor

@itsbilal itsbilal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: nice catch!

Reviewed 1 of 1 files at r1.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @petermattis and @sumeerbhola)

@itsbilal
Copy link
Copy Markdown
Contributor

itsbilal commented Mar 5, 2020

Spoke too soon. Looks like this is going to require some test changes as well as more len(result) != 0 checks in MVCC code. Happy to help with that.

@miretskiy
Copy link
Copy Markdown
Contributor Author

I see. I'm going to try to make the change in libroach.h instead.

@itsbilal
Copy link
Copy Markdown
Contributor

itsbilal commented Mar 5, 2020

Looking at the assertion, I think the most straightforward way might be to just remove it, instead of changing our code around for it. It's the only assertion I could find in rocksdb that has a check like this, and in all other cases, we look at len before diving into the data pointer. Even the rest of the Slice::compare code is lenaware

We already extensively pass nil slices around between rocksdb and go, so I'm confident in being able to continue doing that.

@miretskiy
Copy link
Copy Markdown
Contributor Author

@petermattis can you comment on this please?

So, the initial approach w/ changing rocksdb.go produced many test failures.

We have 3 alternatives:

  1. Stick w/ the current approach & fix test failures.

  2. A change to libroach.h to modify 2 inline functions (I think this might be my preferred approach)

inline rocksdb::Slice ToSlice(DBSlice s) { return rocksdb::Slice(s.data == nullptr ? "" : s.data, s.len); }
inline rocksdb::Slice ToSlice(DBString s) { return rocksdb::Slice(s.data == nullptr ? "" : s.data, s.len); }

I suspect (given the fact that these are inline and copmare against nullptr) that the impact will be 0 (it should all get compiled away).

  1. Remove assertion in our fork of rocksdb code. That assertion is silly. compare() works just fine
    w/ nullptr data. We would have seen many failures if this weren't the case.
    Even with this assertion removed, I'm worried that there might be some implicit assumptions lurking
    in rocks code (e.g. that *slice.data is defined).

(And my least favorite, but I suppose a possibility as well, is to define a static emtpySlice sentinel
in libroach.h (empty DBSlice w/ data initialized to ""). This is not super desirable since, unfortunately,
DBSlice is, in fact, not a constant data structure, and so there might be a chance that some
code might mutate data ptr).

Copy link
Copy Markdown
Collaborator

@petermattis petermattis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. A change to libroach.h to modify 2 inline functions (I think this might be my preferred approach)

I would do this.

(And my least favorite, but I suppose a possibility as well, is to define a static emtpySlice sentinel in libroach.h (empty DBSlice w/ data initialized to ""). This is not super desirable since, unfortunately, DBSlice is, in fact, not a constant data structure, and so there might be a chance that some code might mutate data ptr).

This sounds ok too, but sounds equivalent to option 2. Maybe I'm misunderstanding the problem you see. You'd define static const rocksdb::Slice emptySlice("", 0). There would be no way to modify emptySlice.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @miretskiy and @sumeerbhola)


pkg/storage/rocksdb.go, line 2643 at r1 (raw file):

// assertions checking for slice.data != nullptr.
// Therefore, we represent an empty slice with an empty []byte array.
var emptySlice = []byte{0}

I realize this approach doesn't work, but wanted to point out this is a bit of overkill. You're create a slice of length 1 in order to get a non-nil *byte pointer. There is an easier way:

var emptyByte byte
...
  if len(b) == 0 {
    return C.DBSlice(data: (*C.char)(&emptyByte), len : 0}
  }

This is equivalent to what you wrote, but just goes directly to a *byte rather than using a []byte. The insight here is that []byte is really just sugar for:

type byteSliceHeader {
  ptr *byte
  len int
  cap int
}

Copy link
Copy Markdown
Contributor Author

@miretskiy miretskiy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's just I saw that, even though DBSlice should be an immutable data structure, the fact that we do use it
in a mutable way (e.g. chunked_buffer) made me a bit worried. If you're okay w/ inline function changes, I'd do that.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @petermattis and @sumeerbhola)


pkg/storage/rocksdb.go, line 2643 at r1 (raw file):

Previously, petermattis (Peter Mattis) wrote…

I realize this approach doesn't work, but wanted to point out this is a bit of overkill. You're create a slice of length 1 in order to get a non-nil *byte pointer. There is an easier way:

var emptyByte byte
...
  if len(b) == 0 {
    return C.DBSlice(data: (*C.char)(&emptyByte), len : 0}
  }

This is equivalent to what you wrote, but just goes directly to a *byte rather than using a []byte. The insight here is that []byte is really just sugar for:

type byteSliceHeader {
  ptr *byte
  len int
  cap int
}

Ahh.. Yeah... I didnt' try ptr *byte. I tried creating e.g. []byte{} and pass in the value of that, but cgo was complaining.
Thanks for tip.

Copy link
Copy Markdown
Collaborator

@petermattis petermattis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The libroach/db.h change looks good, but don't you need to revert the rocksdb.go change? Or did you revert it and reviewable is messing up?

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @petermattis and @sumeerbhola)

Libroach's DBSlice data type gets converted to the rocksdb::Slice
whenever we need to access underlying rocksdb (iterators, get/set, etc).

However, internally, rocksdb::Slice asserts that the underlying data
pointer may not be null in the rocksdb::Slice::compare as observed in cockroachdb#45524

This change modifies our rocksdb bridge code to never generate
DBSlices with nullptr data.

Release notes: None
Copy link
Copy Markdown
Contributor

@itsbilal itsbilal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: good work!

Reviewed 1 of 1 files at r2, 1 of 1 files at r3.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @petermattis and @sumeerbhola)

Copy link
Copy Markdown
Collaborator

@petermattis petermattis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @sumeerbhola)

Copy link
Copy Markdown
Collaborator

@sumeerbhola sumeerbhola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, the initial approach w/ changing rocksdb.go produced many test failures.

I did not follow what was causing those test failures -- could you elaborate?

:lgtm:

Reviewable status: :shipit: complete! 2 of 0 LGTMs obtained (waiting on @sumeerbhola)

@miretskiy
Copy link
Copy Markdown
Contributor Author

miretskiy commented Mar 6, 2020 via email

@miretskiy
Copy link
Copy Markdown
Contributor Author

bors r+

@miretskiy
Copy link
Copy Markdown
Contributor Author

Fixes #45524

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Mar 6, 2020

Merge conflict (retrying...)

craig bot pushed a commit that referenced this pull request Mar 6, 2020
45704: colexec: add distinct mode to hashTable r=Azhng a=Azhng

Previously hashTable will buffer all tuples before building `head` and
`same` vector.
Now hashTable supports distinct mode where it only buffers
the distinct tuples from upstream operator. This removes the need of traversing
through the `head` and `same` vectors. Instead, now user of the hashTable
can directly build hashTable in distinct mode and copy the buffered tuples.

Closes #44404

Release note: None

45727: cli: allow for `cockroach demo` to start in secure mode r=knz a=rohany

Fixes #45607.

Release note (cli change): This PR adds support for `cockroach demo`
to start in secure mode using the flag `--insecure=false`.

45759: storage: Do not create nil data pointers in rocksdb slices r=miretskiy a=miretskiy

Fixes #45524

Libroach's DBSlice data type gets converted to the rocksdb::Slice
whenever we need to access underlying rocksdb (iterators, get/set, etc).

However, internally, rocksdb::Slice asserts that the underlying data
pointer may not be null in the rocksdb::Slice::compare as observed in #45524

This change modifies our rocksdb bridge code to never generate
DBSlices with nullptr data.

Release notes: None

Co-authored-by: Azhng <archer.xn@gmail.com>
Co-authored-by: Rohan Yadav <rohany@alumni.cmu.edu>
Co-authored-by: Yevgeniy Miretskiy <yevgeniy@cockroachlabs.com>
@knz knz merged commit aed76cd into cockroachdb:master Mar 6, 2020
@craig
Copy link
Copy Markdown
Contributor

craig bot commented Mar 6, 2020

Canceled (will resume)

@knz
Copy link
Copy Markdown
Contributor

knz commented Mar 6, 2020

bors r-

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Mar 6, 2020

Canceled

@miretskiy miretskiy deleted the empty_slice branch March 6, 2020 17:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ccl/storageccl: TestRandomKeyAndTimestampExport failed

6 participants