Looks reasonable.
The correctness, of course, depends on if we ever modify (and refresh the Cached_digest for) files during a build after having computed its trace, but I don't think we do that.

Do you have any performance measurements of this change?

I'll prepare some benchmarks. Off the top of my head, dune's own build and other common projects aren't representative since they reuse globs that much. @Khady mind testing this?

@Khady ping on testing this.

Sorry I was on holidays, I didn't have time to look at this. Will try to schedule an attempt this week.

It looks like this version of dune doesn't provide dune.configurator anymore. And one of our dependencies needs it. So I am not able to try this PR currently.

It looks like this version of dune doesn't provide dune.configurator anymore. And one of our dependencies needs it. So I am not able to try this PR currently.

dune.configurator has been deprecated to dune-configurator a while ago, and dune has maintained this alias for a while. So I'm not sure what the problem is.

I did some benchmarking of this myself. I created a setup where

• There's a directory with 10k small files
• There's 20 rules that all depend on the directory above in a glob.

This PR makes clean builds around 2x faster (0.6s vs. 1.2s).

Sounds great!

My problem was with an old package that didn't declare explicitly the dependency on dune-configurator and was previously installed in the switch only by luck.

I gave a try to dune master after this branch was merged. The clean build of my project takes about the same time than with 2.7.1. The next dune build calls (so with nothing to build) take 6.5s with dune master vs 8.5s with dune 2.7.1 so there is a nice improvement even if 6.5s is still too long.

Could you profile with memtrace again with master? 6 seconds is indeed quite slow.

Yes, I will give it a go

While you're at it, could you give us a perf report as well? Would be quite useful.