Skip to content

src: add uv_loop watcher_queue update apis#1921

Closed
codebytere wants to merge 1 commit into
libuv:masterfrom
codebytere:add-watcher-queue-update-api
Closed

src: add uv_loop watcher_queue update apis#1921
codebytere wants to merge 1 commit into
libuv:masterfrom
codebytere:add-watcher-queue-update-api

Conversation

@codebytere

@codebytere codebytere commented Jul 11, 2018

Copy link
Copy Markdown
Contributor

This PR adds to uv_loop_t a callback field on_queue_watcher_updated which is called whenever the loop's watcher queue changes. It also adds a public getter and setter for this field.

Electron's Node Integration works by listening to Node's backend file descriptor in a separate thread; when an event is ready the backend file descriptor will trigger a new event for it, and the main thread will then iterate the libuv loop. For certain operations (ex. adding a timeout task) the backend file descriptor isn't informed, & as a result the main thread doesn't know it needs to iterate the libuv loop so the timeout task will never execute until something else trigger a new event. This PR mitigates that problem.

I didn't add a test for the initial iteration of this API addition, but I can go ahead and add one if need be!

Comment thread include/uv.h Outdated
@codebytere codebytere changed the base branch from v1.x to master July 12, 2018 04:40
@codebytere codebytere changed the base branch from master to v1.x July 12, 2018 04:41
@codebytere codebytere changed the base branch from v1.x to master July 12, 2018 04:52
@codebytere

Copy link
Copy Markdown
Contributor Author

@cjihrig any updates on this?

@cjihrig cjihrig left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't add a test for the initial iteration of this API addition, but I can go ahead and add one if need be!

Yes, tests would be preferable.

Is Electron currently floating any similar patches, or is this fixing an existing bug?

Comment thread include/uv.h Outdated
@codebytere

Copy link
Copy Markdown
Contributor Author

@cjihrig we're currently floating this commit.

i'll add a test soon!

@cjihrig

cjihrig commented Jul 19, 2018

Copy link
Copy Markdown
Contributor

Sounds good, thanks.

Do any other collaborators want to weigh in?

@cjihrig cjihrig left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If no other collaborator objects, this LGTM with tests.

@santigimeno

Copy link
Copy Markdown
Member

Code LGTM but I was wondering if it would make sense integrating this into the tracing API proposal

codebytere added a commit to electron/node that referenced this pull request Jul 30, 2018
Electron's Node Integration works by listening to Node's backend file descriptor in a separate thread; when an event is ready the backend file descriptor will trigger a new event for it, and the main thread will then iterate the libuv loop. For certain operations (ex. adding a timeout task) the backend file descriptor isn't informed, & as a result the main thread doesn't know it needs to iterate the libuv loop so the timeout task will never execute until something else trigger a new event. This commit should be removed when libuv/libuv#1921 is merged
@saghul

saghul commented Jul 31, 2018

Copy link
Copy Markdown
Member

The Windows side to this is missing. Is this intentional? Also, this needs some docs explaining what the callback actually does and the platforms where it's supported.

If this is only needed on Unix, how does Electron deal with this on Windows?

@codebytere

Copy link
Copy Markdown
Contributor Author

bump @cjihrig the test should be in place now! i'll resolve conflicts asap once it looks good from the CI side :)

@saghul

saghul commented Aug 30, 2018

Copy link
Copy Markdown
Member

@codebytere please see #1921 (comment)

codebytere added a commit to electron/node that referenced this pull request Sep 13, 2018
Electron's Node Integration works by listening to Node's backend file descriptor in a separate thread; when an event is ready the backend file descriptor will trigger a new event for it, and the main thread will then iterate the libuv loop. For certain operations (ex. adding a timeout task) the backend file descriptor isn't informed, & as a result the main thread doesn't know it needs to iterate the libuv loop so the timeout task will never execute until something else trigger a new event. This commit should be removed when libuv/libuv#1921 is merged
nornagon pushed a commit to electron/node that referenced this pull request Sep 13, 2018
Electron's Node Integration works by listening to Node's backend file descriptor in a separate thread; when an event is ready the backend file descriptor will trigger a new event for it, and the main thread will then iterate the libuv loop. For certain operations (ex. adding a timeout task) the backend file descriptor isn't informed, & as a result the main thread doesn't know it needs to iterate the libuv loop so the timeout task will never execute until something else trigger a new event. This commit should be removed when libuv/libuv#1921 is merged
codebytere added a commit to electron/node that referenced this pull request Oct 2, 2018
Electron's Node Integration works by listening to Node's backend file descriptor in a separate thread; when an event is ready the backend file descriptor will trigger a new event for it, and the main thread will then iterate the libuv loop. For certain operations (ex. adding a timeout task) the backend file descriptor isn't informed, & as a result the main thread doesn't know it needs to iterate the libuv loop so the timeout task will never execute until something else trigger a new event. This commit should be removed when libuv/libuv#1921 is merged
MarshallOfSound pushed a commit to electron/node that referenced this pull request Oct 25, 2018
Electron's Node Integration works by listening to Node's backend file descriptor in a separate thread; when an event is ready the backend file descriptor will trigger a new event for it, and the main thread will then iterate the libuv loop. For certain operations (ex. adding a timeout task) the backend file descriptor isn't informed, & as a result the main thread doesn't know it needs to iterate the libuv loop so the timeout task will never execute until something else trigger a new event. This commit should be removed when libuv/libuv#1921 is merged
@zcbenz

zcbenz commented Nov 30, 2018

Copy link
Copy Markdown
Contributor

The Windows side to this is missing. Is this intentional? Also, this needs some docs explaining what the callback actually does and the platforms where it's supported.

@saghul @codebytere
For Unix systems, adding a new fd to epoll/kqueue does not notify the backend fd, but for Windows IOCP adding new tasks always notifies backend port. That's why in Electron the original patch was only applied for Unix side.

deepak1556 pushed a commit to electron/node that referenced this pull request Dec 10, 2018
Electron's Node Integration works by listening to Node's backend file descriptor in a separate thread; when an event is ready the backend file descriptor will trigger a new event for it, and the main thread will then iterate the libuv loop. For certain operations (ex. adding a timeout task) the backend file descriptor isn't informed, & as a result the main thread doesn't know it needs to iterate the libuv loop so the timeout task will never execute until something else trigger a new event. This commit should be removed when libuv/libuv#1921 is merged
deepak1556 pushed a commit to electron/node that referenced this pull request Dec 19, 2018
Electron's Node Integration works by listening to Node's backend file descriptor in a separate thread; when an event is ready the backend file descriptor will trigger a new event for it, and the main thread will then iterate the libuv loop. For certain operations (ex. adding a timeout task) the backend file descriptor isn't informed, & as a result the main thread doesn't know it needs to iterate the libuv loop so the timeout task will never execute until something else trigger a new event. This commit should be removed when libuv/libuv#1921 is merged
@codebytere

Copy link
Copy Markdown
Contributor Author

@saghul or @cjihrig is this ready once i resolve conflicts?

@bnoordhuis

Copy link
Copy Markdown
Member

@codebytere It got a lgtm and no nacks so I'd say yes, it's good to go after a rebase.

@cjihrig

cjihrig commented Jan 23, 2019

Copy link
Copy Markdown
Contributor

Do we want to decide on anything regarding @santigimeno's #1921 (comment)?

@codebytere

Copy link
Copy Markdown
Contributor Author

@cjihrig @bnoordhuis should be set to go, let me know if I should address anything else!

@bnoordhuis

Copy link
Copy Markdown
Member

@cjihrig Do you still expect that other PR to land someday? I wasn't a big fan last time I looked.

Comment thread docs/src/loop.rst Outdated
@cjihrig

cjihrig commented Jan 23, 2019

Copy link
Copy Markdown
Contributor

I'm fine with landing this independent of #1815. I just didn't want Santi's comment to slip past. Also, sorry to @codebytere for the delay.

@cjihrig

cjihrig commented Jan 23, 2019

Copy link
Copy Markdown
Contributor

Oh, and should this target v1.x instead of master?

EDIT: Nevermind, I see in an old comment this was discussed.

@codebytere codebytere changed the base branch from master to v1.x January 23, 2019 20:52
@saghul

saghul commented Jan 24, 2019

Copy link
Copy Markdown
Member

For Unix systems, adding a new fd to epoll/kqueue does not notify the backend fd, but for Windows IOCP adding new tasks always notifies backend port. That's why in Electron the original patch was only applied for Unix side.

Even so, wouldn't we want to provide the same API on Windows too?

@codebytere

Copy link
Copy Markdown
Contributor Author

@bnoordhuis i think that should be everything; if you think this is good to retarget to v1.x i can go ahead and do that!

Comment thread test/test-fs-event.c Outdated
Comment thread test/test-fs-event.c Outdated
@cjihrig

cjihrig commented Jan 25, 2019

Copy link
Copy Markdown
Contributor

I think targeting v1.x should be fine now.

@codebytere codebytere changed the base branch from master to v1.x January 25, 2019 06:44

@saghul saghul left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this can land in v1.x due to the ABI breakeage, plus I still want to see it implemented on Windows. The fact that Electron achieves the same goal in a different way is not relevant here, we strive to provide cross-platform consistency, so Windows should get this callback to function as well.

Comment thread include/uv.h
@codebytere codebytere changed the base branch from v1.x to master January 25, 2019 17:18
@codebytere

Copy link
Copy Markdown
Contributor Author

@bnoordhuis repointed and removed the version-added from docs since this is pointing at master now!

@codebytere

Copy link
Copy Markdown
Contributor Author

@saghul mind taking a last look at this one?

@saghul saghul left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments, PTAL. Sorry, but I have to insist: Windows is still missing and I'm -1 to landing without it.

Comment thread include/uv.h
unsigned int stop_flag;
void* reserved[4];
uv_loop_cb watcher_queue_updated_cb;
void* reserved[3];

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this targets master now, please leave 4 reserved fields. We can break the ABI here.

Comment thread include/uv.h
};
#undef XX

typedef void (*uv_loop_cb)(uv_loop_t*);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we give this a better name it reads like "loop callback" but we might introduce more in the future.

Comment thread docs/src/loop.rst

.. c:function:: uv_loop_cb uv_loop_get_watcher_queue_changed_callback(uv_loop_t* loop)

Returns `loop->watcher_queue_updated_cb`.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please explain what the function does, not how it's implemented.

Comment thread docs/src/loop.rst
.. c:function:: void uv_loop_set_watcher_queue_changed_callback(uv_loop_t* loop,
uv_loop_cb cb)

Sets `loop->watcher_queue_updated_cb` to `cb`.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

nornagon pushed a commit to electron/node that referenced this pull request Feb 27, 2019
Electron's Node Integration works by listening to Node's backend file descriptor in a separate thread; when an event is ready the backend file descriptor will trigger a new event for it, and the main thread will then iterate the libuv loop. For certain operations (ex. adding a timeout task) the backend file descriptor isn't informed, & as a result the main thread doesn't know it needs to iterate the libuv loop so the timeout task will never execute until something else trigger a new event. This commit should be removed when libuv/libuv#1921 is merged
nitsakh pushed a commit to electron/node that referenced this pull request Mar 1, 2019
Electron's Node Integration works by listening to Node's backend file descriptor in a separate thread; when an event is ready the backend file descriptor will trigger a new event for it, and the main thread will then iterate the libuv loop. For certain operations (ex. adding a timeout task) the backend file descriptor isn't informed, & as a result the main thread doesn't know it needs to iterate the libuv loop so the timeout task will never execute until something else trigger a new event. This commit should be removed when libuv/libuv#1921 is merged
nitsakh pushed a commit to electron/node that referenced this pull request Mar 22, 2019
Electron's Node Integration works by listening to Node's backend file descriptor in a separate thread; when an event is ready the backend file descriptor will trigger a new event for it, and the main thread will then iterate the libuv loop. For certain operations (ex. adding a timeout task) the backend file descriptor isn't informed, & as a result the main thread doesn't know it needs to iterate the libuv loop so the timeout task will never execute until something else trigger a new event. This commit should be removed when libuv/libuv#1921 is merged
nitsakh pushed a commit to electron/node that referenced this pull request Mar 26, 2019
Electron's Node Integration works by listening to Node's backend file descriptor in a separate thread; when an event is ready the backend file descriptor will trigger a new event for it, and the main thread will then iterate the libuv loop. For certain operations (ex. adding a timeout task) the backend file descriptor isn't informed, & as a result the main thread doesn't know it needs to iterate the libuv loop so the timeout task will never execute until something else trigger a new event. This commit should be removed when libuv/libuv#1921 is merged
zcbenz pushed a commit to electron/node that referenced this pull request Apr 16, 2019
Electron's Node Integration works by listening to Node's backend file descriptor in a separate thread; when an event is ready the backend file descriptor will trigger a new event for it, and the main thread will then iterate the libuv loop. For certain operations (ex. adding a timeout task) the backend file descriptor isn't informed, & as a result the main thread doesn't know it needs to iterate the libuv loop so the timeout task will never execute until something else trigger a new event. This commit should be removed when libuv/libuv#1921 is merged
codebytere added a commit to electron/node that referenced this pull request Jun 20, 2019
Electron's Node Integration works by listening to Node's backend file descriptor in a separate thread; when an event is ready the backend file descriptor will trigger a new event for it, and the main thread will then iterate the libuv loop. For certain operations (ex. adding a timeout task) the backend file descriptor isn't informed, & as a result the main thread doesn't know it needs to iterate the libuv loop so the timeout task will never execute until something else trigger a new event. This commit should be removed when libuv/libuv#1921 is merged
@stale stale Bot added the stale label Aug 10, 2019
@stale stale Bot closed this Aug 17, 2019
@bnoordhuis

Copy link
Copy Markdown
Member

@codebytere mentioned she plans on picking this up again so I'll go ahead and reopen this.

@bnoordhuis bnoordhuis reopened this Aug 17, 2019
@stale stale Bot removed the stale label Aug 17, 2019
@libuv libuv deleted a comment from stale Bot Aug 17, 2019
@stale

stale Bot commented Nov 18, 2019

Copy link
Copy Markdown

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants