Add `dune cache clear` command by nojb · Pull Request #8975 · ocaml/dune

nojb · 2023-10-20T07:09:34Z

Of course, if someone does DUNE_CACHE_ROOT=/ dune cache clear or similar they will zap their whole system...

Alizter · 2023-10-20T14:41:05Z

Maybe we should prompt the user saying:

This will delete ALL contents directory /foo/bar, are you sure? (yes/NO)

Saves anybody accidentally zapping something. I don't believe we have a precedent for user prompts but I think this is a good time to do that. As for people using this in scripts, I would argue they really shouldn't be using it so a prompt is OK.

Alizter · 2023-10-20T14:43:58Z

We could also perform a simple check to make sure it is Dune cache we are deleting and not some other files. I'm a bit worried about giving users a blanketed way to shoot themselves.

rgrinberg · 2023-10-22T19:08:55Z

What's the difference between clearing the cache and doing dune clean? I don't see a need for a prompt for either.

A nice improvement would be to see how many files were deleted, but it's not required for this PR.

@pmwhite could you review this one?

emillon · 2023-10-23T08:39:28Z

I find Path.rm_rf ~external:true a bit too unsafe. Is it possible to iterate on Dune_cache_storage.Layout.Versioned.list_*_entries instead?

nojb · 2023-10-23T08:45:32Z

Is it possible to iterate on Dune_cache_storage.Layout.Versioned.list_*_entries instead?

We could. An alternative could be to change the interpretation of DUNE_CACHE_ROOT as follows. Currently, DUNE_CACHE_ROOT is (by default) ~/.cache/dune/db and the cache creates a number of directories inside this directory. Instead we could consider that DUNE_CACHE_ROOT is ~/.cache/dune by default and the Dune cache will create all its directories inside $DUNE_CACHE_ROOT/db. The dune cache clear command could then just zap the $DUNE_CACHE_ROOT/db directory which would be safer than deleting $DUNE_CACHE_ROOT.

Opinions?

pmwhite · 2023-10-23T11:57:09Z

@nojb The most significant part of that alternative to me is that we require DUNE_CACHE_ROOT to contain a db sub-directory, thus providing a little bit of validation that the variable actually does point to a cache. Whether we delete the cache directory itself or just db is an orthogonal decision, I think; that is, we could check that there exists a db sub-directory before deleting the whole cache. Are there other bits of structure that we could check before deleting anything?

Separately, a few bikeshedding thoughts: dune cache clear does sound to me like we're removing the contents of the cache, rather than deleting the cache itself. So I would support either changing the behavior to delete just the contents, or renaming the command to dune cache delete. Maybe dune delete-cache is clearer because dune cache delete might seem like it lets you delete individual cache entries, but it's also a bit less discoverable. I don't have strong feelings about any of these naming things, to be clear; do what you think is best.

Alizter · 2023-10-23T12:16:44Z

On the topic of naming, both dune clean and dune cache clear sound very similar and might confuse future users trying to remember clean vs clear. I had to actually double check before writing this. dune cache clean sounds a bit funny so I don't have a good solution for this, so you can disregard this comment if you wish.

nojb · 2023-10-23T19:43:05Z

I find Path.rm_rf ~external:true a bit too unsafe. Is it possible to iterate on Dune_cache_storage.Layout.Versioned.list_*_entries instead?

I pushed a commit with this approach; I agree that it is an improvement.

Signed-off-by: Nicolás Ojeda Bär <n.oje.bar@gmail.com>

nojb · 2024-01-05T10:10:12Z

Sorry, this one slipped through the cracks :) I rebased it, added a test and an entry to the changelog.

It's been already approved by @pmwhite, so am planning to merge it once the CI passes.

Signed-off-by: Nicolás Ojeda Bär <n.oje.bar@gmail.com>

emillon · 2024-01-05T17:15:45Z

Thanks!

@rgrinberg

CHANGES: - Do not ignore `(formatting ..)` settings in context or workspace files (ocaml/dune#8447, @rgrinberg) - Add command `dune cache clear` to completely delete all traces of the Dune cache. (ocaml/dune#8975, @nojb) - Fixed a bug where Dune was incorrectly parsing the output of coqdep when it was escaped, as is the case on Windows. (ocaml/dune#9231, fixes ocaml/dune#9218, @Alizter) - Copying mode for sandboxes will now follow symbolic links (ocaml/dune#9282, @rgrinberg) - Forbid the empty `(binaries ..)` field in the `env` stanza in the workspace file unless language version is at least 3.2. - [coq] Fix bug in computation of flags when composed with boot theories. (ocaml/dune#9347, fixes ocaml/dune#7909, @ejgallego) - Fixed a bug where the `(select)` field of the `(libraries)` field of the `(test)` stanza wasn't working properly. (ocaml/dune#9387, fixes ocaml/dune#9365, @Alizter) - Allow to disable Coq 0.8 deprecation warning (ocaml/dune#9439, @ejgallego) - Fix handling of the `PATH` argument to `dune init proj NAME PATH`. An intermediate directory called `NAME` is no longer created if `PATH` is supplied, so `dune init proj my_project .` will now initialize a project in the current working directory. (ocaml/dune#9447, fixes ocaml/dune#9209, @shonfeder) - Allow `OCAMLFIND_TOOLCHAIN` to be set per context in the workspace file through the `env` stanza. (ocaml/dune#9449, @rgrinberg) - Experimental doc rules: Correctly handle the case when a package depends upon its own sublibraries (ocaml/dune#9461, fixes ocaml/dune#9456, @jonludlam) - Resolve various public binaries to their build location, rather than to where they're copied in the `_build/install` directory (ocaml/dune#9496, fixes ocaml/dune#7908, @rgrinberg). - Menhir: generate `.conflicts` file by default. Add new field to the `(menhir)` stanza to control the generation of this file: `(explain <blang expression>)`. Introduce `(menhir (flags ...) (explain ...))` field in the `(env)` stanza, delete `(menhir_flags)` field. All changes are guarded under a new version of the Menhir extension, 3.0. (ocaml/dune#9512, @nojb) - Correctly ignore warning flags in vendored projects (ocaml/dune#9515, @rgrinberg) - Directory targets can now be caches. (ocaml/dune#9535, @rleshchinskiy) - Remove warning 30 from default set for projects where dune lang is at least 3.13 (ocaml/dune#9568, @gasche) - It is now possible to use special forms such as `(:include)` and variables `%{read-lines:}` in `(modules)` and similar fields. Note that the dependencies introduced in this way (ie the files being read) must live in a different directory than the stanza making use of them. (ocaml/dune#9578, @nojb) - Use watch exclusions in watch mode on MacOS (ocaml/dune#9643, fixes ocaml/dune#9517, @PoorlyDefinedBehaviour) - Fix merlin configuration for `(include_subdirs qualified)` modules (ocaml/dune#9659, fixes ocaml/dune#8297, @rgrinberg) - Fix handling of `enabled_if` in binary install stanzas. Previously, we'd ignore the result of `enabled_if` when evaluating `%{bin:..}` (ocaml/dune#9707, @rgrinberg) - Add `coqdoc_flags` field to `coq` field of `env` stanza allowing the setting of workspace-wide defaults for `coqdoc_flags`. (ocaml/dune#9280, fixes ocaml/dune#9139, @Alizter) - ctypes: fix an error where `(ctypes)` with no `(function_description)` would cause an error trying refer to a nonexistent `_stubs.a` dependency (ocaml/dune#9302, fix ocaml/dune#9300, @emillon)

@nojb

CHANGES: ### Added - Add command `dune cache clear` to completely delete all traces of the Dune cache. (ocaml/dune#8975, @nojb) - Allow to disable Coq 0.8 deprecation warning (ocaml/dune#9439, @ejgallego) - Allow `OCAMLFIND_TOOLCHAIN` to be set per context in the workspace file through the `env` stanza. (ocaml/dune#9449, @rgrinberg) - Menhir: generate `.conflicts` file by default. Add new field to the `(menhir)` stanza to control the generation of this file: `(explain <blang expression>)`. Introduce `(menhir (flags ...) (explain ...))` field in the `(env)` stanza, delete `(menhir_flags)` field. All changes are guarded under a new version of the Menhir extension, 3.0. (ocaml/dune#9512, @nojb) - Directory targets can now be cached. (ocaml/dune#9535, @rleshchinskiy) - It is now possible to use special forms such as `(:include)` and variables `%{read-lines:}` in `(modules)` and similar fields. Note that the dependencies introduced in this way (ie the files being read) must live in a different directory than the stanza making use of them. (ocaml/dune#9578, @nojb) - Remove warning 30 from default set for projects where dune lang is at least 3.13 (ocaml/dune#9568, @gasche) - Add `coqdoc_flags` field to `coq` field of `env` stanza allowing the setting of workspace-wide defaults for `coqdoc_flags`. (ocaml/dune#9280, fixes ocaml/dune#9139, @Alizter) - ctypes: fix an error where `(ctypes)` with no `(function_description)` would cause an error trying refer to a nonexistent `_stubs.a` dependency (ocaml/dune#9302, fix ocaml/dune#9300, @emillon) ### Changed - Check that package names in `(depends)` and related fields in `dune-project` are well-formed. (ocaml/dune#9472, fixes ocaml/dune#9270, @ElectreAAS) ### Fixed - Do not ignore `(formatting ..)` settings in context or workspace files (ocaml/dune#8447, @rgrinberg) - Fixed a bug where Dune was incorrectly parsing the output of coqdep when it was escaped, as is the case on Windows. (ocaml/dune#9231, fixes ocaml/dune#9218, @Alizter) - Copying mode for sandboxes will now follow symbolic links (ocaml/dune#9282, @rgrinberg) - Forbid the empty `(binaries ..)` field in the `env` stanza in the workspace file unless language version is at least 3.2. - [coq] Fix bug in computation of flags when composed with boot theories. (ocaml/dune#9347, fixes ocaml/dune#7909, @ejgallego) - Fixed a bug where the `(select)` field of the `(libraries)` field of the `(test)` stanza wasn't working properly. (ocaml/dune#9387, fixes ocaml/dune#9365, @Alizter) - Fix handling of the `PATH` argument to `dune init proj NAME PATH`. An intermediate directory called `NAME` is no longer created if `PATH` is supplied, so `dune init proj my_project .` will now initialize a project in the current working directory. (ocaml/dune#9447, fixes ocaml/dune#9209, @shonfeder) - Experimental doc rules: Correctly handle the case when a package depends upon its own sublibraries (ocaml/dune#9461, fixes ocaml/dune#9456, @jonludlam) - Resolve various public binaries to their build location, rather than to where they're copied in the `_build/install` directory (ocaml/dune#9496, fixes ocaml/dune#7908, @rgrinberg). - Correctly ignore warning flags in vendored projects (ocaml/dune#9515, @rgrinberg) - Use watch exclusions in watch mode on MacOS (ocaml/dune#9643, fixes ocaml/dune#9517, @PoorlyDefinedBehaviour) - Fix merlin configuration for `(include_subdirs qualified)` modules (ocaml/dune#9659, fixes ocaml/dune#8297, @rgrinberg) - Fix handling of `enabled_if` in binary install stanzas. Previously, we'd ignore the result of `enabled_if` when evaluating `%{bin:..}` (ocaml/dune#9707, @rgrinberg)

@nojb

CHANGES: ### Added - Add command `dune cache clear` to completely delete all traces of the Dune cache. (ocaml/dune#8975, @nojb) - Allow to disable Coq 0.8 deprecation warning (ocaml/dune#9439, @ejgallego) - Allow `OCAMLFIND_TOOLCHAIN` to be set per context in the workspace file through the `env` stanza. (ocaml/dune#9449, @rgrinberg) - Menhir: generate `.conflicts` file by default. Add new field to the `(menhir)` stanza to control the generation of this file: `(explain <blang expression>)`. Introduce `(menhir (flags ...) (explain ...))` field in the `(env)` stanza, delete `(menhir_flags)` field. All changes are guarded under a new version of the Menhir extension, 3.0. (ocaml/dune#9512, @nojb) - Directory targets can now be cached. (ocaml/dune#9535, @rleshchinskiy) - It is now possible to use special forms such as `(:include)` and variables `%{read-lines:}` in `(modules)` and similar fields. Note that the dependencies introduced in this way (ie the files being read) must live in a different directory than the stanza making use of them. (ocaml/dune#9578, @nojb) - Remove warning 30 from default set for projects where dune lang is at least 3.13 (ocaml/dune#9568, @gasche) - Add `coqdoc_flags` field to `coq` field of `env` stanza allowing the setting of workspace-wide defaults for `coqdoc_flags`. (ocaml/dune#9280, fixes ocaml/dune#9139, @Alizter) - ctypes: fix an error where `(ctypes)` with no `(function_description)` would cause an error trying refer to a nonexistent `_stubs.a` dependency (ocaml/dune#9302, fix ocaml/dune#9300, @emillon) ### Changed - Check that package names in `(depends)` and related fields in `dune-project` are well-formed. (ocaml/dune#9472, fixes ocaml/dune#9270, @ElectreAAS) ### Fixed - Do not ignore `(formatting ..)` settings in context or workspace files (ocaml/dune#8447, @rgrinberg) - Fixed a bug where Dune was incorrectly parsing the output of coqdep when it was escaped, as is the case on Windows. (ocaml/dune#9231, fixes ocaml/dune#9218, @Alizter) - Copying mode for sandboxes will now follow symbolic links (ocaml/dune#9282, @rgrinberg) - Forbid the empty `(binaries ..)` field in the `env` stanza in the workspace file unless language version is at least 3.2. - [coq] Fix bug in computation of flags when composed with boot theories. (ocaml/dune#9347, fixes ocaml/dune#7909, @ejgallego) - Fixed a bug where the `(select)` field of the `(libraries)` field of the `(test)` stanza wasn't working properly. (ocaml/dune#9387, fixes ocaml/dune#9365, @Alizter) - Fix handling of the `PATH` argument to `dune init proj NAME PATH`. An intermediate directory called `NAME` is no longer created if `PATH` is supplied, so `dune init proj my_project .` will now initialize a project in the current working directory. (ocaml/dune#9447, fixes ocaml/dune#9209, @shonfeder) - Experimental doc rules: Correctly handle the case when a package depends upon its own sublibraries (ocaml/dune#9461, fixes ocaml/dune#9456, @jonludlam) - Resolve various public binaries to their build location, rather than to where they're copied in the `_build/install` directory (ocaml/dune#9496, fixes ocaml/dune#7908, @rgrinberg). - Correctly ignore warning flags in vendored projects (ocaml/dune#9515, @rgrinberg) - Use watch exclusions in watch mode on MacOS (ocaml/dune#9643, fixes ocaml/dune#9517, @PoorlyDefinedBehaviour) - Fix merlin configuration for `(include_subdirs qualified)` modules (ocaml/dune#9659, fixes ocaml/dune#8297, @rgrinberg) - Fix handling of `enabled_if` in binary install stanzas. Previously, we'd ignore the result of `enabled_if` when evaluating `%{bin:..}` (ocaml/dune#9707, @rgrinberg)

nojb mentioned this pull request Oct 20, 2023

feature: clear dune cache #8971

Closed

nojb force-pushed the dune_cache_clear branch from 0e8a1ef to 8cfc7a4 Compare October 24, 2023 10:53

pmwhite approved these changes Oct 24, 2023

View reviewed changes

nojb force-pushed the dune_cache_clear branch from 8cfc7a4 to f96f7cf Compare January 1, 2024 10:27

Add 'dune cache clear' command

1fb0837

Signed-off-by: Nicolás Ojeda Bär <n.oje.bar@gmail.com>

nojb force-pushed the dune_cache_clear branch from f96f7cf to 6debedf Compare January 5, 2024 09:58

Add test

dd1331a

Signed-off-by: Nicolás Ojeda Bär <n.oje.bar@gmail.com>

nojb force-pushed the dune_cache_clear branch from 6debedf to dd1331a Compare January 5, 2024 10:06

Changes

ea911e8

Signed-off-by: Nicolás Ojeda Bär <n.oje.bar@gmail.com>

nojb added 3 commits January 5, 2024 11:24

Make test reproducible

a0a0f27

Signed-off-by: Nicolás Ojeda Bär <n.oje.bar@gmail.com>

Accept test results

6202077

Signed-off-by: Nicolás Ojeda Bär <n.oje.bar@gmail.com>

Merge branch 'main' into dune_cache_clear

9258690

nojb added this to the 3.13.0 milestone Jan 5, 2024

Merge branch 'main' into dune_cache_clear

d6a112f

nojb merged commit 36231e4 into ocaml:main Jan 5, 2024

nojb deleted the dune_cache_clear branch January 5, 2024 12:18

emillon mentioned this pull request Jan 12, 2024

[new release] dune (15 packages) (3.13.0~alpha1) ocaml/opam-repository#25067

Closed

emillon mentioned this pull request Jan 16, 2024

[new release] dune (15 packages) (3.13.0) ocaml/opam-repository#25081

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `dune cache clear` command#8975

Add `dune cache clear` command#8975
nojb merged 7 commits intoocaml:mainfrom
nojb:dune_cache_clear

nojb commented Oct 20, 2023

Uh oh!

Alizter commented Oct 20, 2023 •

edited

Loading

Uh oh!

Alizter commented Oct 20, 2023

Uh oh!

rgrinberg commented Oct 22, 2023

Uh oh!

emillon commented Oct 23, 2023 •

edited

Loading

Uh oh!

nojb commented Oct 23, 2023

Uh oh!

pmwhite commented Oct 23, 2023 •

edited

Loading

Uh oh!

Alizter commented Oct 23, 2023 •

edited

Loading

Uh oh!

nojb commented Oct 23, 2023

Uh oh!

nojb commented Jan 5, 2024

Uh oh!

emillon commented Jan 5, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

nojb commented Oct 20, 2023

Uh oh!

Alizter commented Oct 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Alizter commented Oct 20, 2023

Uh oh!

rgrinberg commented Oct 22, 2023

Uh oh!

emillon commented Oct 23, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nojb commented Oct 23, 2023

Uh oh!

pmwhite commented Oct 23, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Alizter commented Oct 23, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nojb commented Oct 23, 2023

Uh oh!

nojb commented Jan 5, 2024

Uh oh!

emillon commented Jan 5, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Alizter commented Oct 20, 2023 •

edited

Loading

emillon commented Oct 23, 2023 •

edited

Loading

pmwhite commented Oct 23, 2023 •

edited

Loading

Alizter commented Oct 23, 2023 •

edited

Loading