Skip to content

Allow rpc-triggered builds in eager watch mode#11622

Merged
gridbugs merged 1 commit intoocaml:mainfrom
gridbugs:allow-rpc-build-in-eager-watch-mode
Apr 28, 2025
Merged

Allow rpc-triggered builds in eager watch mode#11622
gridbugs merged 1 commit intoocaml:mainfrom
gridbugs:allow-rpc-build-in-eager-watch-mode

Conversation

@gridbugs
Copy link
Copy Markdown
Collaborator

When dune is started in eager watch mode (e.g. by running dune build --watch) it starts an RPC server, but prior to this change it would reject RPC requests to build targets. This is a step towards being able to run dune commands that trigger build (e.g. build, exec, test) while watch mode is running.

Kind of testing the waters with this change as there may be consequences I'm unaware of of running the entire build system concurrently in two different fibers. Conceptually I think it makes sense, as the task of rebuilding targets named on the command line in response to file changes is unrelated to the task of rebuilding targets in response to RPC messages. Do we need to take extra care to prevent the two fibers from interfering with each other?

@gridbugs gridbugs force-pushed the allow-rpc-build-in-eager-watch-mode branch from d8f4b55 to 24dc317 Compare April 11, 2025 07:11
@maiste maiste added the build Issue related to what dune build label Apr 16, 2025
Comment on lines +323 to +326
let ivar = Fiber.Ivar.create () in
let targets = List.map targets ~f:server.parse_build in
let* () = Job_queue.write server.pending_build_jobs (targets, ivar) in
Fiber.Ivar.read ivar
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make sure I understand: you delete this part because either in passive or eager mode, we want to handle the request. Am I correct?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's correct. The only purpose of the watch_mode_config field was to prevent build requests from being handled by eager watch mode, so if we're allowing eager watch mode to handle build requests now then there's no need for the field or for this match statement (it now just always does what was formerly the Yes Passive case).

@gridbugs gridbugs force-pushed the allow-rpc-build-in-eager-watch-mode branch from 24dc317 to 94fcfb8 Compare April 18, 2025 06:33
@Alizter
Copy link
Copy Markdown
Collaborator

Alizter commented Apr 18, 2025

You'll need to update the documentation rpc.rst since that mentioned you can only connect to passive watch mode. This PR allows for both to happen.

I tested dune build -w together with dune rpc ... commands and everything appears to be working fine. I also made sure that restricting to -j1 doesn't cause a lock with the two concurrent fibers happening. I didn't find any issues, so I think that part is fine.

@gridbugs gridbugs force-pushed the allow-rpc-build-in-eager-watch-mode branch from 94fcfb8 to 660297e Compare April 22, 2025 03:03
@gridbugs
Copy link
Copy Markdown
Collaborator Author

Thanks for testing this out @Alizter. I've update the rpc docs.

When dune is started in eager watch mode (e.g. by running `dune build
--watch`) it starts an RPC server, but prior to this change it would
reject RPC requests to build targets. This is a step towards being able
to run dune commands that trigger build (e.g. `build`, `exec`, `test`)
while watch mode is running.

Signed-off-by: Stephen Sherratt <stephen@sherra.tt>
@gridbugs gridbugs force-pushed the allow-rpc-build-in-eager-watch-mode branch from 660297e to 2a7d875 Compare April 28, 2025 01:39
@gridbugs gridbugs merged commit 8743f31 into ocaml:main Apr 28, 2025
24 of 25 checks passed
@gridbugs gridbugs deleted the allow-rpc-build-in-eager-watch-mode branch April 28, 2025 03:20
maxim092001 pushed a commit to maxim092001/dune that referenced this pull request May 3, 2025
When dune is started in eager watch mode (e.g. by running `dune build
--watch`) it starts an RPC server, but prior to this change it would
reject RPC requests to build targets. This is a step towards being able
to run dune commands that trigger build (e.g. `build`, `exec`, `test`)
while watch mode is running.

Signed-off-by: Stephen Sherratt <stephen@sherra.tt>
maiste added a commit to maiste/opam-repository that referenced this pull request May 20, 2025
CHANGES:

### Fixed

- Fixed a bug that was causing cram tests attached to multiple aliases to be run multiple
  times. (ocaml/dune#11547, @Alizter)

- Fix: pass pkg-config (extra) args in all pkgconfig invocations. A missing --personality
  flag would result in pkgconf not finding libraries in some contexts. (ocaml/dune#11619, @MisterDA)

- Fix: Evaluate `enabled_if` when computing the stubs for stanzas such as
  `foreign_library` (ocaml/dune#11707, @Alizter, @rgrinberg)

- Fix $ dune describe pp for libraries in the presence of `(include_subdirs
  unqualified)` (ocaml/dune#11729, fixes ocaml/dune#10999, @rgrinberg)

- Fix `$ dune subst` in sub directories of a git repository (ocaml/dune#11760, fixes
  ocaml/dune#11045, @Richard-Degenne)

- Fix a crash involving `Path.drop_prefix` when using Melange on Windows
  (ocaml/dune#11767, @nojb)

### Added

- Added detection and warning for common typos in package dependency
  constraints (ocaml/dune#11600, fixes ocaml/dune#11575, @kemsguy7)

- Added `(extra_objects)` field to `(foreign_library)` stanza with `(:include)` support.
  (ocaml/dune#11683, @Alizter)

### Changed

- Allow build RPC messages to be handled by dune's RPC server in eager watch
  mode (ocaml/dune#11622, @gridbugs)

- Allow concurrent build with RPC server (ocaml/dune#11712, @gridbugs)
maiste added a commit to maiste/opam-repository that referenced this pull request May 21, 2025
CHANGES:

### Fixed

- Fixed a bug that was causing cram tests attached to multiple aliases to be run multiple
  times. (ocaml/dune#11547, @Alizter)

- Fix: pass pkg-config (extra) args in all pkgconfig invocations. A missing --personality
  flag would result in pkgconf not finding libraries in some contexts. (ocaml/dune#11619, @MisterDA)

- Fix: Evaluate `enabled_if` when computing the stubs for stanzas such as
  `foreign_library` (ocaml/dune#11707, @Alizter, @rgrinberg)

- Fix $ dune describe pp for libraries in the presence of `(include_subdirs
  unqualified)` (ocaml/dune#11729, fixes ocaml/dune#10999, @rgrinberg)

- Fix `$ dune subst` in sub directories of a git repository (ocaml/dune#11760, fixes
  ocaml/dune#11045, @Richard-Degenne)

- Fix a crash involving `Path.drop_prefix` when using Melange on Windows
  (ocaml/dune#11767, @nojb)

### Added

- Added detection and warning for common typos in package dependency
  constraints (ocaml/dune#11600, fixes ocaml/dune#11575, @kemsguy7)

- Added `(extra_objects)` field to `(foreign_library)` stanza with `(:include)` support.
  (ocaml/dune#11683, @Alizter)

### Changed

- Allow build RPC messages to be handled by dune's RPC server in eager watch
  mode (ocaml/dune#11622, @gridbugs)

- Allow concurrent build with RPC server (ocaml/dune#11712, @gridbugs)
Sudha247 pushed a commit to Sudha247/dune that referenced this pull request Jul 23, 2025
When dune is started in eager watch mode (e.g. by running `dune build
--watch`) it starts an RPC server, but prior to this change it would
reject RPC requests to build targets. This is a step towards being able
to run dune commands that trigger build (e.g. `build`, `exec`, `test`)
while watch mode is running.

Signed-off-by: Stephen Sherratt <stephen@sherra.tt>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build Issue related to what dune build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants