Skip to content

caching system: support non-exclusive locks for some resources #7596

@andrewrk

Description

@andrewrk

0b46c27 addresses a deadlock where:

zig test held an exclusive lock on compiler_rt.o but then executed a child process, which attempted to obtain an exclusive lock on compiler_rt.o. Both of these processes should be able to have the lock at the same time because neither of them are going to modify this object, it's all read-only access.

For project-local assets, because we do in-place binary patching, we grab an exclusive lock on the binary we want to patch before attempting to do it. However for global shared resources it does not make sense to do in-place binary patching. I proposed that for global resources we put everything into the cache hash, including linker options, and use a shared (read-only) lock for this.

For global resources it would look something like this:

  • try to open the manifest file with shared lock. If you get it, it's a cache hit and you have a shared lock, others can grab it too.
  • otherwise create the manifest file with an exclusive lock. this makes shared lockers wait until you finish putting the resource there, and downgrade the lock to shared, waking up the waiters who may now use the resource.

This will allow child processes to run while holding shared locks, and the behavior will be exactly what we want: prevent the cache garbage collector from reaping the resources so that their stack traces, debug info, etc will still work correctly.

The commit mentioned at the top could then be reverted, cleaning up the code, and resolving a race condition (what if the binary gets modified in between comp.destroy() and the child process spawning?).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementSolving this issue will likely involve adding new logic or components to the codebase.frontendTokenization, parsing, AstGen, Sema, and Liveness.

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions