Skip to content

improve shmem api#2107

Merged
eeeebbbbrrrr merged 4 commits into
pgcentralfoundation:developfrom
usamoi:shmem
Aug 1, 2025
Merged

improve shmem api#2107
eeeebbbbrrrr merged 4 commits into
pgcentralfoundation:developfrom
usamoi:shmem

Conversation

@usamoi

@usamoi usamoi commented Jul 4, 2025

Copy link
Copy Markdown
Contributor

closes #2106

  1. heapless is removed due to unsoundness
  2. atomic_traits is removed due to unsoundness
  3. PgAtomic::new and PgLwLock::new is marked unsafe; caller must be confident that there are no name conflicts (including pg_shmem_init is called for a variable twice)
  4. unsafe impl<T: PGRXSharedMemory> PGRXSharedMemory for PgSpinLock<T> {}, in order to allow it work on shared memory
  5. pg_shmem_init!(ATOMIC = AtomicUsize::new(0)); is allowed now
  6. checks if the pointer to the allocated shared memory is properly aligned

It breaks api.

@usamoi usamoi force-pushed the shmem branch 2 times, most recently from 17c20f3 to ed369ff Compare July 15, 2025 13:48
Comment thread pgrx-examples/bgworker/src/lib.rs Outdated
Comment on lines +33 to +35
if unsafe { pgrx::pg_sys::IsUnderPostmaster } {
pgrx::error!("this extension must be loaded via shared_preload_libraries.");
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've also seen !process_shared_preload_libraries_in_progress used for this.

    /* Must be loaded with shared_preload_libraries */
    if (!process_shared_preload_libraries_in_progress)
        ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
                errmsg("pgaudit must be loaded via shared_preload_libraries")));

@eeeebbbbrrrr

Copy link
Copy Markdown
Contributor

Hey @usamoi! Can you elaborate on heapless being unsound?

@usamoi

usamoi commented Aug 1, 2025

Copy link
Copy Markdown
Contributor Author

Can you elaborate on heapless being unsound?

static S: PgLwLock<Vec<&'static str>>;

// process 1
S.write().push(Box::leak(String::from("aaaaaa")));

// process 2
dbg!(S.read().pop().unwrap()); // boom!

@eeeebbbbrrrr

Copy link
Copy Markdown
Contributor

Doesn't removing heapless drastically limit the usefulness of our shared memory support?

On the other hand, does anyone even use it, I wonder?

I'm going to cut a new release today and I'm on the fence about if we should merge this PR now or later.

@usamoi

usamoi commented Aug 1, 2025

Copy link
Copy Markdown
Contributor Author

Doesn't removing heapless drastically limit the usefulness of our shared memory support?

Now AssertPGRXSharedMemory is added, so they could still use it by:

static DEQUE: PgLwLock<AssertPGRXSharedMemory<heapless::Deque<Pgtest, 400>>> =
    unsafe { PgLwLock::new(c"shmem_deque") };

pg_shmem_init!(DEQUE = unsafe { AssertPGRXSharedMemory::new(Default::default()) });

compared to before:

static DEQUE: PgLwLock<heapless::Deque<Pgtest, 400>> =
    unsafe { PgLwLock::new(c"shmem_deque") };

pg_shmem_init!(DEQUE);

@usamoi usamoi marked this pull request as draft August 1, 2025 16:26
@usamoi usamoi marked this pull request as ready for review August 1, 2025 16:30
@eeeebbbbrrrr

Copy link
Copy Markdown
Contributor

Now AssertPGRXSharedMemory is added, so they could still use it by:

Okay, I see. Then yeah, I suppose this is fine. I'll merge it now and then start on a v0.16.0

@eeeebbbbrrrr eeeebbbbrrrr merged commit 14dcb18 into pgcentralfoundation:develop Aug 1, 2025
16 checks passed
eeeebbbbrrrr added a commit that referenced this pull request Aug 1, 2025
Welcome to pgrx v0.16.0.

This release contains support for Postgres 18beta2 and has some breaking
changes in that support for pgrx' "hooks" implementation, which has been
deprecated for over a year, has finally been removed.

Additionally, due to unsoundness issues, direct support for using
`heapless` in shared memory has been removed. Users can still do this
themselves, which requires them to assert they're taking responsibility
of possible unsoundness issues.

As always, first install the latest `cargo-pgrx` with:

```shell
$ cargo install cargo-pgrx --version 0.16.0 --locked
```

Then you're free to run `cargo pgrx upgrade` in the root of all your
extension crates.

To pickup pg18beta2 support you'll also want to run `cargo pgrx init` so
that it can be downloaded and compiled.

# What's Changed

## Breaking Changes

* delete `pgrx::hooks` by @usamoi in
#2120
* improve shmem api by @usamoi in
#2107


## New Features

* update to Postgres v18beta2 by @usamoi in
#2111
* feat: `BackgroundWorker::connect_worker_to_spi_by_oid` by @if0ne in
#2116
* teach `#[pg_cast]` to support 3-argument CAST functions by
@eeeebbbbrrrr in #2119
* teach `#[pg_extern]` about a SUPPORT function by @eeeebbbbrrrr in
#2121

## Bug Fixes

* fix name_data_to_str by @usamoi in
#2108
* add pg_guard_ffi_boundary to direct_pg_extern_function_call_as_datum
by @usamoi in #2118

## `cargo-pgrx` Improvements

* add `--valgrind` to more cargo-pgrx subcommands by @usamoi in
#2109
* `cargo pgrx regress --resetdb` will run `setup.sql` by @ccleve in
#2113


## Code Cleanup

* filter out unrecognized attributes in PostgresGucEnum by @usamoi in
#2102
* Elide needless lifetime by @nyurik in
#2097

## Package/Build System Cleanup

* chore: include pgrx-version-updater into workspace by @nyurik in
#2101
* chore: consolidate package settings in workspace by @nyurik in
#2100
* chore: prepare for 2024 edition by @nyurik in
#2103

# Thanks!  

Thanks to all contributors -- y'alls work helps keep pgrx moving
forward.

**Full Changelog**:
v0.15.0...v0.16.0
daamien pushed a commit to daamien/pgrx that referenced this pull request Dec 15, 2025
closes pgcentralfoundation#2106

1. `heapless` is removed due to unsoundness
2. `atomic_traits` is removed due to unsoundness
3. `PgAtomic::new` and `PgLwLock::new` is marked unsafe; caller must be
confident that there are no name conflicts (including `pg_shmem_init` is
called for a variable twice)
4. `unsafe impl<T: PGRXSharedMemory> PGRXSharedMemory for PgSpinLock<T>
{}`, in order to allow it work on shared memory
5. `pg_shmem_init!(ATOMIC = AtomicUsize::new(0));` is allowed now
6. checks if the pointer to the allocated shared memory is properly
aligned

It breaks api.
daamien pushed a commit to daamien/pgrx that referenced this pull request Dec 15, 2025
Welcome to pgrx v0.16.0.

This release contains support for Postgres 18beta2 and has some breaking
changes in that support for pgrx' "hooks" implementation, which has been
deprecated for over a year, has finally been removed.

Additionally, due to unsoundness issues, direct support for using
`heapless` in shared memory has been removed. Users can still do this
themselves, which requires them to assert they're taking responsibility
of possible unsoundness issues.

As always, first install the latest `cargo-pgrx` with:

```shell
$ cargo install cargo-pgrx --version 0.16.0 --locked
```

Then you're free to run `cargo pgrx upgrade` in the root of all your
extension crates.

To pickup pg18beta2 support you'll also want to run `cargo pgrx init` so
that it can be downloaded and compiled.

# What's Changed

## Breaking Changes

* delete `pgrx::hooks` by @usamoi in
pgcentralfoundation#2120
* improve shmem api by @usamoi in
pgcentralfoundation#2107


## New Features

* update to Postgres v18beta2 by @usamoi in
pgcentralfoundation#2111
* feat: `BackgroundWorker::connect_worker_to_spi_by_oid` by @if0ne in
pgcentralfoundation#2116
* teach `#[pg_cast]` to support 3-argument CAST functions by
@eeeebbbbrrrr in pgcentralfoundation#2119
* teach `#[pg_extern]` about a SUPPORT function by @eeeebbbbrrrr in
pgcentralfoundation#2121

## Bug Fixes

* fix name_data_to_str by @usamoi in
pgcentralfoundation#2108
* add pg_guard_ffi_boundary to direct_pg_extern_function_call_as_datum
by @usamoi in pgcentralfoundation#2118

## `cargo-pgrx` Improvements

* add `--valgrind` to more cargo-pgrx subcommands by @usamoi in
pgcentralfoundation#2109
* `cargo pgrx regress --resetdb` will run `setup.sql` by @ccleve in
pgcentralfoundation#2113


## Code Cleanup

* filter out unrecognized attributes in PostgresGucEnum by @usamoi in
pgcentralfoundation#2102
* Elide needless lifetime by @nyurik in
pgcentralfoundation#2097

## Package/Build System Cleanup

* chore: include pgrx-version-updater into workspace by @nyurik in
pgcentralfoundation#2101
* chore: consolidate package settings in workspace by @nyurik in
pgcentralfoundation#2100
* chore: prepare for 2024 edition by @nyurik in
pgcentralfoundation#2103

# Thanks!  

Thanks to all contributors -- y'alls work helps keep pgrx moving
forward.

**Full Changelog**:
pgcentralfoundation/pgrx@v0.15.0...v0.16.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

shmem api is unsound

3 participants