Add FlatArray<'_, T>#2207
Conversation
d09cfa3 to
2f2958b
Compare
|
There's one mystery of why the lifetime isn't something I can use, here, but I'll leave that one for tomorrow. It will require digging into our macro expansion code, which is always "fun". |
ad6143c to
667dc32
Compare
|
Dismantling this incrementally into other PRs so that it becomes more obvious what is actually a key part of this PR overall, and also to make it easier to pinpoint the macro/lifetime issue. |
667dc32 to
f847ce8
Compare
|
Biggest remaining question for this PR that I've been thinking about:
|
|
The answer is yes, partly because |
|
I think the other Q is "can we get rid of the inner lifetime?" |
f847ce8 to
e31ea80
Compare
FlatArray<'_, T> and PBox<'_, T>FlatArray<'_, T>
|
In order to make this work, we need to break
|
b799fc6 to
500148e
Compare
TODO
|
|
|
| TooManyBytes, | ||
| TooManyElems, | ||
| /// One or more dimensions are zero | ||
| ZeroLenDim(usize), | ||
| ZeroLenDim, |
There was a problem hiding this comment.
While returning the information we had was cute, it wasn't viable in all cases without compromising the code significantly, and it was always at risk of being incomplete, so I stopped doing that.
67529b9 to
057dc3c
Compare
|
Hm. I think I should/could probably add testing that only relies on direct calls instead of SPI, and that would be enough for some basic functionality, even though it doesn't test the "full cycle". |
Introduce a new Rust type for what ArrayTypes "really are": FlatArray. As an unsized type, it will primarily be used as `&FlatArray<'_, T>` or similar in actual extension code. The long-term plan is for FlatArray to phase out `pgrx::datum::Array<'_, T>` entirely, as it has several advantages over its predecessor: - It uses `Element`, a subset of `BorrowDatum`, to power its simplicity in iteration, making it more easily verifiable and maintainable. - It is the unsized type itself, allowing composing it with various Rusty pointer types like `&`, `&mut`, or `PBox`. - It is no longer endangered by performance cliffs or hidden allocations depending on the type of `T`. - Because of these previous traits, it is possible to soundly and safely allocate a fresh `FlatArray`, mutate it, and return that one allocation, instead of allocating an intermediate Vec in Rust memory to mutate! It also has an associated cost in that it no longer supports any `T` that requires complex unboxing of items like `datum::Array<'a, String>`. You must instead use `array.iter().map(closure)` to accomplish that. This itself summarizes the difference: `datum::Array` is effectively an iteration protocol, but `array::FlatArray` is an actual data structure. This addresses the implied feature request in, and thus supercedes, pgcentralfoundation#2086.
Welcome to pgrx v0.17.0. This is a new minor release that brings a bunch of internal code refactoring, cleanup, new Postgres headers, and additional `cargo-pgrx regress` CLI options. As always, please install the latest `cargo-pgrx` with `cargo install cargo-pgrx --version 0.17.0 --locked` and also update your extension crate dependencies with `cargo pgrx upgrade`. ## What's Changed ### New Headers/Symbols * include `access/tsmapi.h` by @usamoi in #2155 * Include access/subtrans.h & access/commit_ts.h by @isdaniel in #2147 * include `utils/guc_tables.h` by @usamoi in #2243 * Include `scanner.h` in Rust bindings by @piki in #2163 * Add `commands/publicationcmds.h` to generated `pg_sys` bindings by @Sinjo in #2221 * Add the ClientAuthentication_hook by @daamien in #2231 * add storage/dsm_*.h headers by @eeeebbbbrrrr in #2244 * pg-sys: add catalog/heap.h bindings for PG18 by @charmitro in #2150 * Add back `peer_dn` field to `Port` on `pg17..` by @workingjubilee in #2200 * Add `repr(C)` to `struct Port` on `pg13..=16` by @workingjubilee in #2201 ### `cargo-pgrx` improvements * Add a psql_verbosity parameter for cargo pgrx regress by @daamien in #2230 ### Code Cleanup * Replace with let-chains in pgrx-bindgen by @workingjubilee in #2168 * Replace with let-chains in pgrx-sql-entity-graph by @workingjubilee in #2169 * Restore so-called needless lifetimes by @workingjubilee in #2167 * Drop stray Postgres 12 support in cargo-pgrx by @workingjubilee in #2173 * Remove deprecated enum##member constants by @workingjubilee in #2170 * Remove `variadic!` macro by @workingjubilee in #2171 * Semi-automated code cleanup by @workingjubilee in #2189 * Deref instead of borrowing `Option<&String>` by @workingjubilee in #2185 * Refactor schema.rs using a cargo command builder by @workingjubilee in #2187 * Replace `impl AsRef<T>` with `&T` by @workingjubilee in #2182 * Make `PgRelation::heap_relation` unsafe by @workingjubilee in #2176 * Move `cargo_pgrx::env` to `cargo_pgrx::cargo` by @workingjubilee in #2186 * Move `CargoProfile` into `cargo_pgrx::cargo` by @workingjubilee in #2188 * Add Scalar trait for generic handling of primitive arrays by @workingjubilee in #2153 * Set BGWORKER_SHMEM_ACCESS when building background workers by @cbandy in #2161 * Stop referencing `pg_config` in schema generation by @workingjubilee in #2174 * Show line number of failed asserts by @piki in #2175 * Ghostbust `used_type.rs` a bit by @workingjubilee in #2190 * Manually bind `static MyProcPort` and `struct Port` by @workingjubilee in #2162 * Make datetime types import each other via `use super` by @workingjubilee in #2198 * Migrate datetime types into `pgrx::datetime` by @workingjubilee in #2199 * Use diagnostic attrs for ABI/SQL-related traits by @workingjubilee in #2203 * Make `Scalar` provide `pg_sys::Oid` constant by @workingjubilee in #2209 * Hoist varlena encoding fn by @workingjubilee in #2208 * Recurse through `T` while anonymizing `&T` by @workingjubilee in #2212 * `impl BorrowDatum for TimeTz` by @workingjubilee in #2213 * Change `MemCx::alloc_bytes` to return `NonNull` by @workingjubilee in #2214 * Add `MemCx<'current, T>` arguments and `PBox<'current, T>` returns by @workingjubilee in #2210 * Further improve diagnostics for signatures with no SQL form by @workingjubilee in #2215 * Add `array::Element` and `callconv::DatumPass` by @workingjubilee in #2218 * Remove unused line in `memcxt_tests.rs` by @workingjubilee in #2220 * Add `FlatArray<'_, T>` by @workingjubilee in #2207 * Allow explicit handling of allocation errors by @workingjubilee in #2226 * Place lbound before updating array len product by @workingjubilee in #2236 * introduce `#[pg_guard(unsafe_entry_thread)]` by @eeeebbbbrrrr in #2242 ### Project Administrativa * Use Rust Edition 2024 by @workingjubilee in #2165 * Remove testing for deprecated Intel Macs by @workingjubilee in #2156 * Remove unused Cargo.lock for version-updater by @workingjubilee in #2180 * Add rust-analyzer.toml by @workingjubilee in #2179 * Note pgrx requires an unknown minimum Xcode version by @workingjubilee in #2164 ## New Contributors * @isdaniel made their first contribution in #2147 * @cbandy made their first contribution in #2161 * @piki made their first contribution in #2163 * @Sinjo made their first contribution in #2221 **Full Changelog**: v0.16.1...v0.17.0
Introduce a new Rust type for what ArrayTypes "really are": FlatArray. As an unsized type, it will primarily be used as
&FlatArray<'_, T>or similar in actual extension code. The long-term plan is for FlatArray to phase outpgrx::datum::Array<'_, T>entirely, as it has several advantages over its predecessor:Element, a subset ofBorrowDatum, to power its simplicity in iteration, making it more easily verifiable and maintainable.&,&mut, orPBox.T.FlatArray, mutate it, and return that one allocation, instead of allocating an intermediate Vec in Rust memory to mutate!It also has an associated cost in that it no longer supports any
Tthat requires complex unboxing of items likedatum::Array<'a, String>. You must instead usearray.iter().map(closure)to accomplish that. This itself summarizes the difference:datum::Arrayis effectively an iteration protocol, butarray::FlatArrayis an actual data structure.This addresses the implied feature request in, and thus supercedes, #2086.