It may be noteworthy to mention that Buildbarn currently ignores this flag altogether; it behaves as if it's always set to true. We currently assume that if a client actually calls into Execute() it already checked the AC beforehand. Bazel seems to do that at least.

Is it really worth the effort to have this flag?

I think it's a mistake for Bazel to do two calls. This just adds another round-trip for no benefit. I think at some point Bazel will move to doing a single call to the remote executor, and expect to be able to get a cache hit back.

+1 to Ulf, I'm pretty sure Bazel can already switch to doing the single Execute call and get cache hits back (from RBE), since it already has code in place to do all the uploads if a missing input status is returned. That wasn't the case in the beginning, when we wanted to enforce the order of cache check -> get missing -> upload missing -> execute. But note that while doing that may be a bit simpler, I'm not sure that it will save a round trip often. The saving will only happen when the action is (a) not in the cache, and (b) can be re-executed without any additional uploads. This seems like an unlikely case.
Nevertheless, I think it's important to have the flag to that servers may return cached results, or merge actions in flight, as an optimization.

Ulf: I think it's equivalent right now (as Ola says) - either:
GetActionResult (NOT_FOUND) -> upload action & blobs -> Execute, or
Execute(NOT_FOUND) -> upload actions & blobs -> Execute

You could pre-upload actions & blobs before calling Execute, but you'd only want to do so on incremental compilation paths where you know the action is going to be a miss anyways (at which point whether or not Execute checks the action cache is moot); on clean executions where the action is most likely a cache hit calling any upload APIs before calling Execute would be net worse than calling GetActionResult or Execute and uploading only on miss.

So bazel could call one fewer distinct API if it wanted to (Execute*2 instead of Get...Execute), but still needs to call GetActionResult for cache-only flows. So not really better? On the service owner side, Ed's point holds: it's simpler if a service doesn't have to check the action cache on Execute, and doesn't have to be able to quickly reject actions where blobs are missing.

On the other side, one feature I'd like to eventually add to the API is inline uploads in the Execute call itself. (Probably gated on something in the Capabilities API, so services with separation between CAS and Execute don't have to implement it). At that point build tools can make an optimization when executions are probably incremental or are likely to be critical-path bound of using a single-RPC fast-path:
Execute(inc. inline upload of changed files; merkle tree nodes; action)
You could still maybe argue that skip_cache_lookup is still unnecessary (just set it to true always), as in Ed's case though then the failure mode of build tools calling the "fast" path when there is actually a cache entry is a lot worse.

We already have code in Blaze to upload locally modified files eagerly (we detect locally modified files very early in the build), and we've found that to be a significant performance improvement for highly incremental builds at the time, so it seems likely that that's going to happen here as well. (Caveat: it's possible that Bazel's internals have changed sufficiently that eager uploading is no longer a win.)

+1 for eagerly uploading files - I think it'd still be a boon - but note that merkle trees and action protos also need upload, and can only be constructed at around the time the action is ready for execution anyways.