Skip to content

Python node event mutex#1244

Merged
haixuanTao merged 8 commits intodora-rs:mainfrom
ultra-robotics:python-node-event-mutex
Dec 2, 2025
Merged

Python node event mutex#1244
haixuanTao merged 8 commits intodora-rs:mainfrom
ultra-robotics:python-node-event-mutex

Conversation

@oortlieb
Copy link
Copy Markdown
Contributor

@oortlieb oortlieb commented Nov 28, 2025

Allows concurrent read/write access through a node (to address #748). "Already borrowed" seems to happen when there are concurrent calls to read/write functions because both currently require an exclusive Node ref.

  1. Updates DelayedCleanup.get_mut and callers to use &self instead of &mut self.
  2. Adds a mutex to Events.inner so callers can use &self instead of &mut self.
  3. Adds an example dataflow that fails on main (with the "Already borrowed" error) but works with this change

@oortlieb
Copy link
Copy Markdown
Contributor Author

oortlieb commented Nov 29, 2025

Another option would be to use a RefCell instead of a Mutex on Events.inner. This would theoretically improve performance a bit, but would introduce panics in cases where Events.inner is accessed concurrently.

@haixuanTao
Copy link
Copy Markdown
Collaborator

RefCell

I think refcell would be single threaded making multi-threaded application not working...

@oortlieb
Copy link
Copy Markdown
Contributor Author

Good point!

@haixuanTao What needs to happen to get this merged? Are you happy with it or do you want to consider alternative implementations?

Ok(Node {
events: Events {
inner: EventsInner::Dora(events),
inner: Mutex::new(EventsInner::Dora(events)),
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think Arc Mutex is better in this case for multi-threaded use cases

Suggested change
inner: Mutex::new(EventsInner::Dora(events)),
inner: Arc::new(Mutex::new(EventsInner::Dora(events))),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would be the advantage of adding an Arc here? Is there a way to clone the event channel that I don't see?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, yes you could have python cloning the node within Python.

I don't know if python reference counting is enough for making this thread safe.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK pyo3 already wraps the whole thing into some reference counted wrapper. So I would assume that the new Arc here will always have a reference count of exactly 1.

@haixuanTao
Copy link
Copy Markdown
Collaborator

haixuanTao commented Dec 1, 2025

I have been thinking about this and I will merge it once we have Arc> to make sure we can safely share nodes between thread!

Thanks for the PR!

Comment on lines -43 to 45
pub fn get_mut(&mut self) -> std::sync::MutexGuard<T> {
pub fn get_mut(&self) -> std::sync::MutexGuard<T> {
self.0.try_lock().expect("failed to lock DelayedCleanup")
}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this will still panic on concurrent send operations. This PR seems to be mostly about supporting send and receive concurrently, so I don't think that we need to fix this as part of the same PR. I just wanted to mention it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, good callout.

Copy link
Copy Markdown
Collaborator

@phil-opp phil-opp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@oortlieb
Copy link
Copy Markdown
Contributor Author

oortlieb commented Dec 1, 2025

@haixuanTao I wrapped the new Mutex in an ARC and fixed a merge conflict with main.

@haixuanTao haixuanTao merged commit ddb64e8 into dora-rs:main Dec 2, 2025
50 checks passed
@haixuanTao
Copy link
Copy Markdown
Collaborator

Thanks a lot!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants