Skip to content

Conversation

@gspencergoog
Copy link
Contributor

@gspencergoog gspencergoog commented May 19, 2020

Description

Update the flutter and dart scripts' locking mechanism and follow_links function to be more robust and support more platforms.

This adds support for using mkdir as a fallback if the system doesn't have flock instead of using shlock, since shlock doesn't work on shared filesystems.

It also fixes a problem in the follow_links function where it failed when the link resolved to the root directory.

Related Issues

Tests

  • Existing tests don't fail.

Breaking Change

  • No, this is not a breaking change.

@gspencergoog gspencergoog force-pushed the flutter_links_locks branch 3 times, most recently from 8e54354 to 45a4013 Compare May 19, 2020 20:11
@gspencergoog
Copy link
Contributor Author

I did consider using the mkdir method for all platforms, simply because then we would be able to not be limited to locking for a single host if the flutter tree is on a file share, but the downside is that if someone kills a running script with kill -9, the flock file descriptor will be closed, whereas the exit trap won't run, leaving a lock directory around.

This is a tradeoff, but I think a file-shared repo is a less common situation than killing a running script on a single host, so I preferred flock where it exists.

I've switched the flock call to be in a loop though, so that we can immediately print the "Waiting for another flutter command..." prompt: as it is now, it doesn't print that until the Dart snapshot is built and runs, so if you don't have it built, it takes a while to say why it's waiting (like 10-20 seconds).

@gspencergoog gspencergoog force-pushed the flutter_links_locks branch 3 times, most recently from 73d2170 to c35bbeb Compare May 19, 2020 21:01
@gspencergoog
Copy link
Contributor Author

After looking into it some more, I think I'm just going to use the mkdir method on all platforms. I found out that shlock doesn't work on file shares with macOS (I tried it locally, and it says "operation not permitted" when trying to lock), and I tried killing the process with -9, and it still was able to remove the lock directory, and that was the one advantage that I thought flock had over the mkdir method.

@gspencergoog
Copy link
Contributor Author

The advantage of using mkdir is that it works on network shares and (nearly all) NFS volumes.

I did find after fixing this, though, that the locking that the flutter tools use internally also doesn't work over a network share (at least on an AFP-over-TCP share mounted on a macOS machine), so I'm not sure that this is a real fix to a problem anyone has right now: they already couldn't use Flutter from that kind of share, even if the bash script worked perfectly.

@gspencergoog
Copy link
Contributor Author

Even more data! :-) Actually, if I kill -9 the subshell that is doing the upgrade, then it will leave around the lock directory, and flock won't (because it's just a file descriptor). Killing the parent shell with kill -9 still allows the subshell to clean up.

So I guess I'm going to leave it using flock for systems that have it. Which is too bad: it made the code a lot simpler to just have one way of doing things.

@christopherfujino
Copy link
Contributor

Even more data! :-) Actually, if I kill -9 the subshell that is doing the upgrade, then it will leave around the lock directory, and flock won't (because it's just a file descriptor). Killing the parent shell with kill -9 still allows the subshell to clean up.

So I guess I'm going to leave it using flock for systems that have it. Which is too bad: it made the code a lot simpler to just have one way of doing things.

We need to link to this PR in the code, lest someone end up going through the whole thing again in a few years :P

@gspencergoog gspencergoog force-pushed the flutter_links_locks branch 3 times, most recently from b4d5e5e to 54792b6 Compare May 20, 2020 00:12
@gspencergoog
Copy link
Contributor Author

Well, the PR is implicitly linked, since it's in the history for the file.

@gspencergoog gspencergoog marked this pull request as ready for review May 20, 2020 00:22
@gspencergoog
Copy link
Contributor Author

OK, I've updated this to HEAD, and I think this is ready for review "for reals".

@gspencergoog gspencergoog removed the request for review from jcollins-g May 26, 2020 16:20
Copy link
Contributor

@jonahwilliams jonahwilliams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes and comments look good for the most part.

I did not exhaustively verify the new lock logic, I think that if it works on CI and locally thats really the best we can do with shells

bin/shared.sh Outdated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit. should we print a newline here? when I issued dart -h while another process had a lock, it looked like this:

~/git/flutter$ bin/dart -h
A command-line utility for Dart development.se the startup lock...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason this prints with a return is that the Dart code itself will start up and lock, printing that same message.

So, with the newline, it can look like this:

~/git/flutter$ bin/dart -h
Waiting for another flutter command to release the startup lock...
Waiting for another flutter command to release the startup lock...
A command-line utility for Dart development.

Maybe printing it twice is better than printing it once and sometimes overwriting it, but...

I had an idea: the code now prints

printf "                                                                   \r"

when this lock is released, and then if the Dart code locks, it'll just flicker once, and then there's no confusion. Does that work?

Copy link
Contributor

@christopherfujino christopherfujino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with a nit.

@christopherfujino
Copy link
Contributor

Thanks so much for doing this @gspencergoog!

@goderbauer goderbauer added the tool Affects the "flutter" command-line tool. See also t: labels. label May 26, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

tool Affects the "flutter" command-line tool. See also t: labels.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants