feat: Add `Builder::permissions()` method. by Byron · Pull Request #273 · Stebalien/tempfile

Byron · 2024-02-04T09:38:02Z

This PR allows the user to control the file mode set for newly created NameTempfile instances.

This is paramount for when tempfiles are used as scratch space that will be persisted and renamed into the final destination once all writes are completed. This is an effective way to prevent half-written files on disk and to assure that readers can only observe the final result of a change.

This paradigm is implemented in the gix-lock crate which is used by the gix-ref crate for editing Git references. Currently these are created with 0o600 permissions, which can cause issues when using sudo in downstream applications.

When this PR is merged, gix-ref can be adjusted to allow setting the mode just like Git does.

Review Notes

I did my best to adapt the least amount of code while staying true to the current organisation. Initially I named the new method mode, but realized that std::fs::Permissions exists which would allow the API to be more general, despite only one available platform for implementation.

Let me explain the commits in some detail and why it's one more than needed:

the first commit is a failing test that adds the new Builder method without an implementation. CI is red to show it's not working.
the second commits implements it to be least invasive, which violates the principle that platform dependent code should remain in imp. CI is green.
the third commit implements it so that permissions can be passed down to the platforms, which is more invasive, yet cleaner. CI is also green.

Please let me know which way you prefer as its also a trade-off.

With it it's possible to change the default mode of tempfiles on unix systems. The example also doesn't hide the fact that it is currently only useful on Unix, but as the ecosystem matures, thanks to the usage of `std::fs::Permissions` it should be able grow into more platforms over time.

The implementation favors the least invasive solution even though its forced to pull platform dependent code up. The alternative would have been to alter the method signatures of `create_named()` on all platforms.

This way, permissions can be passed and handled by platform specific code, which avoid having to use platform dependent code in the top-level.

Stebalien · 2024-02-04T18:40:50Z

@NobodyXu I agree if you're only target is Linux, but it's still useful to have a platform-independent way to do this (to, e.g., support MacOS which hasn't seen any filesystem API improvements in, basically ever).

Stebalien · 2024-02-04T18:54:02Z

+    /// Permissions for directories aren't currently set even though it would
+    /// be possible on Unix systems.


We need to implement this before we merge this patch, unfortunately. This builder is used to create temporary directories as well, and I don't really trust users to read the docs.

See the discussion in #61.

I took a look at this and saw that std::fs::create_dir() is used to create the directory, I don't think the standard library has a way to handle this.
Did you have an idea on how to do this? Maybe a rustix chmod call?

I think we'll need to use rustix::fs::mkdir.

This comment is now out of date.

Stebalien · 2024-02-04T18:59:03Z

Thanks! And yes, I prefer the final version (more invasive approach).

Byron · 2024-02-04T19:50:28Z

Thanks for the quick review! If CI passes I have addressed the windows portion of it, but will definitely need some guidance to deal with the directory. If you have ideas and want to tinker, please feel free to push into this branch - sometimes that's easiest.

@NobodyXu Thanks for sharing! It's definitely something to consider for the time when performance needs to be optimised further. Personally I am most interested in doing what Git does at first, which seems to be pretty much what's happening here.

That way, there will be no surprises as would be the case with doing nothing.

Byron · 2024-02-04T21:00:28Z

I have added mode-support for directories thanks to your hint with rustix::fs::mkdir and it appears to work. Further, trying to change permissions on unsupported platforms will result in an error that as well.
When CI is green it's ready for review, with the remark that I kept dir.rs nimble, with a structure sufficient for the task even though it's not quite the same as in file.

Stebalien · 2024-02-04T23:05:18Z

+    imp::create(path, permissions)
+}
+
+mod imp {


I'd prefer it if we created the entire directory structure.

I can do that, even though it's a change that might have performance implications. For that reason alone I wouldn't do it without also introducing a new method for it, and do so in a separate PR.

Wait, no. I mean: put the modules in different files/directories (imp/...). Sorry, I realize that that was confusing.

Ah, right, I could have guessed it. And it's done now. A note regarding naming: It feels like any should be other, but I decided not to do that as other means unsupported in the file folder, while here it is a catch-all.

Hm. Yeah, I may go through and do a rename later but those are implementation details so.. meh.

Stebalien · 2024-02-04T23:23:33Z

So... somehow I missed this but: https://doc.rust-lang.org/std/fs/struct.DirBuilder.html#method.mode. We don't need to use rustix here.

Byron · 2024-02-05T06:17:11Z

So... somehow I missed this but: https://doc.rust-lang.org/std/fs/struct.DirBuilder.html#method.mode. We don't need to use rustix here.

Oh my, I missed that too! I clearly was biased when researching this topic and "didn't believe in mode being available for directories" 🤦‍♂️. Lesson learned, and the code was adjusted to something much better.

I made a note that recommends to not recursively create the directory by default but leave that to a separate PR along with a new flag in the builder maybe to control this, to keep the previous behaviour exactly the same. Right now, this still is the case.

If CI is green, this PR should be ready for re-review.

Stebalien · 2024-02-05T16:17:08Z

+pub(crate) struct PathError {
+    pub(crate) path: PathBuf,
+    pub(crate) err: io::Error,


This change can be revered.

Stebalien · 2024-02-05T16:17:57Z

+    }
+    dir_options
+        .create(&path)
+        .with_err_path(|| path.clone())


Suggested change

.with_err_path(|| path.clone())

.with_err_path(|| &path)

(the path is cloned automatically if there's an error)

Stebalien · 2024-02-05T16:18:38Z

+    /// Permissions for directories aren't currently set even though it would
+    /// be possible on Unix systems.


This comment is now out of date.

Stebalien · 2024-02-05T16:20:19Z

+    imp::create(path, permissions)
+}
+
+mod imp {


Hm. Yeah, I may go through and do a rename later but those are implementation details so.. meh.

Stebalien · 2024-02-05T16:21:12Z

And with those changes, this should be good to go. Thanks!

Byron · 2024-02-05T17:21:27Z

Thanks a lot for the QA and sorry for the sloppiness, at least 2 of 3 I should have caught.
But here we are, and I think that's the version that is ready to go.

PS: A new release with these changes would help, as that way everything can trickle downstream to StackedGit. Thanks again.

Stebalien

❤️

Stebalien · 2024-02-05T18:18:25Z

Thanks a lot for the QA and sorry for the sloppiness, at least 2 of 3 I should have caught.

Don't be. That'll just make me feel worse about my PRs and that's why we have review 😆.

PS: A new release with these changes would help, as that way everything can trickle downstream to StackedGit. Thanks again.

Will do.

* feat: Add `Builder::permissions()` method. With it it's possible to change the default mode of tempfiles on unix systems. The example also doesn't hide the fact that it is currently only useful on Unix, but as the ecosystem matures, thanks to the usage of `std::fs::Permissions` it should be able grow into more platforms over time. * Use `permissions` field of `Builder` on Unix to affect file mode The implementation favors the least invasive solution even though its forced to pull platform dependent code up. The alternative would have been to alter the method signatures of `create_named()` on all platforms. * Respect module boundaries and make all platforms permissions aware. This way, permissions can be passed and handled by platform specific code, which avoid having to use platform dependent code in the top-level. * Fail on Windows if somebody tries to set permissions there. That way, there will be no surprises as would be the case with doing nothing. * Add support for setting permissions on directories as well.

Byron added 3 commits February 4, 2024 10:06

Use permissions field of Builder on Unix to affect file mode

334eb14

The implementation favors the least invasive solution even though its forced to pull platform dependent code up. The alternative would have been to alter the method signatures of `create_named()` on all platforms.

Respect module boundaries and make all platforms permissions aware.

5ffc706

This way, permissions can be passed and handled by platform specific code, which avoid having to use platform dependent code in the top-level.

This was referenced Feb 4, 2024

tempfile with permissions GitoxideLabs/gitoxide#1276

Merged

2.4.2: sudo -u nobody stg operations override system umask stacked-git/stgit#413

Closed

This comment was marked as off-topic.

Sign in to view

Stebalien marked this pull request as ready for review February 4, 2024 18:48

Stebalien reviewed Feb 4, 2024

View reviewed changes

Comment thread src/lib.rs Outdated

Byron force-pushed the permissions branch 2 times, most recently from 8bb3386 to c0c925d Compare February 4, 2024 19:45

Fail on Windows if somebody tries to set permissions there.

7a80c77

That way, there will be no surprises as would be the case with doing nothing.

Byron force-pushed the permissions branch 2 times, most recently from 8a172f4 to 19ec472 Compare February 4, 2024 20:58

NobodyXu mentioned this pull request Feb 4, 2024

feat: Support creating unreachable named tempfile on Linux #274

Closed

Stebalien reviewed Feb 4, 2024

View reviewed changes

Byron force-pushed the permissions branch from 19ec472 to 223a718 Compare February 5, 2024 06:13

Byron force-pushed the permissions branch from 223a718 to 1bf4bca Compare February 5, 2024 06:57

Stebalien reviewed Feb 5, 2024

View reviewed changes

Add support for setting permissions on directories as well.

e374091

Byron force-pushed the permissions branch from 1bf4bca to e374091 Compare February 5, 2024 17:20

Stebalien approved these changes Feb 5, 2024

View reviewed changes

Stebalien merged commit 4a05e47 into Stebalien:master Feb 5, 2024

Stebalien mentioned this pull request Feb 5, 2024

Add parameter for not restricting access to the named tempfile #61

Closed

Byron deleted the permissions branch February 5, 2024 19:09

		/// Permissions for directories aren't currently set even though it would
		/// be possible on Unix systems.

Conversation

Byron commented Feb 4, 2024

Review Notes

Uh oh!

This comment was marked as off-topic.

Stebalien commented Feb 4, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Stebalien commented Feb 4, 2024

Uh oh!

Byron commented Feb 4, 2024

Uh oh!

Byron commented Feb 4, 2024

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Stebalien commented Feb 4, 2024

Uh oh!

Byron commented Feb 5, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Stebalien commented Feb 5, 2024

Uh oh!

Byron commented Feb 5, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Stebalien left a comment

Choose a reason for hiding this comment

Uh oh!

Stebalien commented Feb 5, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Byron commented Feb 5, 2024 •

edited

Loading