Skip to content

Dune cache on Windows: link errors #6992

Closed
#6993
Closed
Dune cache on Windows: link errors#6992
#6993
@nojb

Description

@nojb
nojb
opened on Feb 4, 2023

We are using the Dune cache under Windows we see the following errors:

Shared cache miss [d59504c9eafd215f1d2e6e6071e395de] (_build/default/applications/apropos/addins/.addins.objs/alerts_manager.intf.all-deps): error: Unix.Unix_error(Unix.EUNKNOWNERR -1142, "link", "_build/default/applications/apropos/addins/.addins.objs/alerts_manager.intf.all-deps")
Shared cache miss [ba0fea2959eff234cc831591ef4d7ab3] (_build/default/applications/apropos/addins/.addins.objs/algebra_script.intf.all-deps): error: Unix.Unix_error(Unix.EUNKNOWNERR -1142, "link", "_build/default/applications/apropos/addins/.addins.objs/algebra_script.intf.all-deps")
Shared cache miss [48d6ee193488549a39a2b074ae85ec7e] (_build/default/applications/apropos/addins/.addins.objs/amortized_floating_rate_note.intf.all-deps): error: Unix.Unix_error(Unix.EUNKNOWNERR -1142, "link", "_build/default/applications/apropos/addins/.addins.objs/amortized_floating_rate_note.intf.all-deps")
Shared cache miss [bdaf7400db8ee546010b80a987355eac] (_build/default/applications/apropos/addins/.addins.objs/amortized_margin.intf.all-deps): error: Unix.Unix_error(Unix.EUNKNOWNERR -1142, "link", "_build/default/applications/apropos/addins/.addins.objs/amortized_margin.intf.all-deps")
Shared cache miss [9529c3e50ff734b1970074a12fc4183d] (_build/default/applications/apropos/addins/.addins.objs/apropos_web.intf.all-deps): error: Unix.Unix_error(Unix.EUNKNOWNERR -1142, "link", "_build/default/applications/apropos/addins/.addins.objs/apropos_web.intf.all-deps")
Shared cache miss [d8cdcd89fcd07b64e53b55e8b621d06c] (_build/default/applications/apropos/addins/.addins.objs/apropos_web_config.intf.all-deps): error: Unix.Unix_error(Unix.EUNKNOWNERR -1142, "link", "_build/default/applications/apropos/addins/.addins.objs/apropos_web_config.intf.all-deps")
Shared cache miss [2bdaf211d32d947f46569a63140f09fb] (_build/default/applications/apropos/addins/.addins.objs/apropos_web_external_auth.intf.all-deps): error: Unix.Unix_error(Unix.EUNKNOWNERR -1142, "link", "_build/default/applications/apropos/addins/.addins.objs/apropos_web_external_auth.intf.all-deps")
Shared cache miss [40066c0e524a70a8b04edcf8b188ba02] (_build/default/applications/apropos/addins/.addins.objs/apropos_web_file_sync.intf.all-deps): error: Unix.Unix_error(Unix.EUNKNOWNERR -1142, "link", "_build/default/applications/apropos/addins/.addins.objs/apropos_web_file_sync.intf.all-deps")
Shared cache miss [c20e1a1465bde213dcd3aa7616f5bec0] (_build/default/applications/apropos/addins/.addins.objs/apropos_web_pricing.intf.all-deps): error: Unix.Unix_error(Unix.EUNKNOWNERR -1142, "link", "_build/default/applications/apropos/addins/.addins.objs/apropos_web_pricing.intf.all-deps")
Shared cache miss [ae6226d8afb2e7d7752e77321c177a5c] (_build/default/applications/apropos/addins/.addins.objs/apropos_web_pricing_api.intf.all-deps): error: Unix.Unix_error(Unix.EUNKNOWNERR -1142, "link", "_build/default/applications/apropos/addins/.addins.objs/apropos_web_pricing_api.intf.all-deps")
etc

The error code 1142 is ERROR_TOO_MANY_LINKS. There is some mitigation code for this error in

dune/src/dune_cache/local.ml

Lines 44 to 59 in f7cf8d4

(* This function is like [Unix.link] but handles the "Too many links" error by
creating a copy of the [src] in a temporary directory, then atomically
replacing the [src] with the copy, and finally creating the requested [dst]
by calling [Unix.link src dst] again.
We hit the "Too many links" error because we store a lot of empty files in
the cache, which all get deduplicated into the same cache entry. This
function essentially deletes the "overlinked" entry from the cache, creating
a fresh copy with the 0 link count. This leads to some duplication but it's
negligible: we might store the empty file several times across all workspaces
instead of just storing it once.
If you need to debug this function, you can trigger the "Too many links"
error by running [for i in {1..100000}; do ln $file tmp/$i; done], where the
[$file] is the shared cache entry for the empty file. After that, no more
hard links on [$file] will be allowed, triggering the [EMLINK] code path. *)
, but only handles Unix.EMLINK. This should actually be handled in the compiler by translating between ERROR_TOO_MANY_LINKS and EMLINK, and I opened a PR fixing this ocaml/ocaml#11991. However, we should still fix the error for previous versions of OCaml by catching Unix_error (EUNKNOWNERR, -1142, _) in addition to Unix_error (EMLINK, _, _).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions