Skip to content

[Safety] Add safety checks to vector indexing#6927

Merged
Mytherin merged 32 commits intoduckdb:masterfrom
Tishj:safe_vector
Apr 7, 2023
Merged

[Safety] Add safety checks to vector indexing#6927
Mytherin merged 32 commits intoduckdb:masterfrom
Tishj:safe_vector

Conversation

@Tishj
Copy link
Contributor

@Tishj Tishj commented Mar 31, 2023

This PR is less massive ;)

The only thing that needed to be done was remove explicit use of std::vector, and luckily we already had duckdb/common/vector.hpp to introduce vector into the duckdb namespace.

Same as the unique_ptr, this will likely be a two-phase PR. The first phase is moving away from std::vector to our class (which is derived from std::vector), and the next phase will be to introduce the safety check.

The safety check will likely incur some performance issues where vectors are being indexed heavily, and will require some profiling to figure out where this happens.
That's the main reason for doing this in two phases

@hannes
Copy link
Member

hannes commented Mar 31, 2023

Does it make sense to add a CI run where the use of std::vector triggers a compilation failure similar to what we have for unqualified move?

@Tishj
Copy link
Contributor Author

Tishj commented Mar 31, 2023

Does it make sense to add a CI run where the use of std::vector triggers a compilation failure similar to what we have for unqualified move?

That would be great, but I'm not sure how we would add something like this.
The std::move one is different because we made duckdb::move throw the static assert, this case is reverse.

We could add a script that scans files for the use of std::vector?

@Tishj
Copy link
Contributor Author

Tishj commented Apr 2, 2023

A couple changes made to safe_unique_ptr are also relevant here:

  • Changes to exception.hpp, so we can inline the bounds check (probably needs more changes, because 'exception.hpp' heavily uses 'vector')
  • clear() needs the same if (DUCKDB_CLANG_TIDY) -> clang::reinitializes treatment

Copy link
Contributor

@carlopi carlopi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@Tishj
Copy link
Contributor Author

Tishj commented Apr 5, 2023

Merged with #6967 - I suspect that will get merged first, so I figured I'd skip a need to merge later

@Tishj
Copy link
Contributor Author

Tishj commented Apr 7, 2023

The PR on the wasm side just got merged, that should be the final puzzle piece to get this PR passing CI 🤞
Not sure if the wasm version upgrade should have been a separate PR instead?

@carlopi
Copy link
Contributor

carlopi commented Apr 7, 2023

I think it makes sense to have the changes to duckdb-wasm hash here, since they are connected. Looks good.

Also worth writing down somewhere that currently duck-wasm hash is used only for testing changes in duckdb, and no one is relying on generated build artifacts, so there are no visible consequences, only ducdkb developers will see potentially different test results (but that's true with any PR...).

This is to say, it make sense they are tied, and also avoid an extra CI round for no reason.

@Tishj Tishj requested a review from Mytherin April 7, 2023 12:47
@Tishj
Copy link
Contributor Author

Tishj commented Apr 7, 2023

The inlining of the check is going to require some more work (managing the includes so we don't deal with circular dependencies)
But I can work on that after this is merged.

I also need to deal with regressions from enabling the check on release anyways, so this could be grouped with that

@Mytherin Mytherin merged commit 2f6cbb9 into duckdb:master Apr 7, 2023
@Mytherin
Copy link
Collaborator

Mytherin commented Apr 7, 2023

Thanks! LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants