Skip to content

Add export option: py_hermetic_scripts#20763

Merged
cognifloyd merged 11 commits intomainfrom
cognifloyd/mutable_virtualenv_with_non_hermetic_scripts
Apr 16, 2024
Merged

Add export option: py_hermetic_scripts#20763
cognifloyd merged 11 commits intomainfrom
cognifloyd/mutable_virtualenv_with_non_hermetic_scripts

Conversation

@cognifloyd
Copy link
Member

This implements #17764 but only for mutable_virtualenv. If we also want to do something for symlinked immutable, then I will remove the Closes line below.

Closes: #17764

@cognifloyd cognifloyd requested a review from benjyw April 7, 2024 15:02
@cognifloyd cognifloyd self-assigned this Apr 7, 2024
@huonw
Copy link
Contributor

huonw commented Apr 7, 2024

Just exploring design space: what do you think about a separate option py_resolve_hermetic_scripts = True/False?

In my mind, that seems like it might be nicer if/when we allow non-hermtic immutable venvs too, but potentially in the short term might lead to having to have a dedicated error about "cannot specify py_resolve_hermetic_scripts = False with py_resolve_format = "symlinked_immutable_virtualenv"" or similar.

@cognifloyd
Copy link
Member Author

cognifloyd commented Apr 11, 2024

To support non-hermetic scripts with symlinked immutable, we would probably have to have a separate cache. I think that would eliminate a lot of the benefits of using the symlink. So if someone wants to use non-hermetic scripts, they should probably go with a mutable venv.

I could turn this into a boolean, and error if it is true when the mode is symlinked immutable. I'm not sure that that is better. If we added a py_resolve_format=requirements (exporting a requirements.txt file instead of a whole venv), then we would also need to exclude that format as well.

py_editable_in_resolve also only applies to mutable_virtualenv. So maybe another boolean option would be ok.

This is the whati have implemented:

[export]
py_resolve_format = "mutable_virtualenv_with_non_hermetic_scripts"
py_editable_in_resolve = [...]

This is one possible Boolean:

[export]
py_resolve_format = "mutable_virtualenv"
py_hermetic_scripts = false # defaults to true
py_editable_in_resolve = [...]

And this is another:

[export]
py_resolve_format = "mutable_virtualenv"
py_non_hermetic_scripts = true # defaults to false
py_editable_in_resolve = [...]

Considering those options, I think I prefer the option with py_hermetic_scripts = false.

I will adjust this PR accordingly.

@kaos
Copy link
Member

kaos commented Apr 11, 2024

Considering those options, I think I prefer the option with py_hermetic_scripts = false.

I'm not entirely clued in, but without context, I also prefer py_hermetic.. over py_non_hermetic... as option name. I always tend to get confused when a ft toggle is a negative, as the end result with the option itself then becomes a double negative (or a negated positive)..

@cognifloyd cognifloyd requested review from huonw and kaos April 12, 2024 03:15
@cognifloyd
Copy link
Member Author

cognifloyd commented Apr 12, 2024

Thanks @huonw. The py_hermetic_scripts option feels cleaner.

I didn't add any validation logic that errors if someone has py_hermetic_scripts set with py_resolve_format="symlinked_immutable_virtualenv". The doc strings specify that this option only applies to mutable_virtualenv. Plus, if someone used a cli arg to temporarily override the format (to try out the symlinked venv maybe?) throwing an error over an option that is essentially ignored doesn't feel like the best experience. Do you think some validation logic is still needed? Or is it ok without?

@cognifloyd cognifloyd changed the title Add py export type: mutable_virtualenv_with_non_hermetic_scripts, Add export option: py_hermetic_scripts Apr 12, 2024
@kaos
Copy link
Member

kaos commented Apr 12, 2024

Do you think some validation logic is still needed? Or is it ok without?

For me, I think pants should not complain about options not being in effect.

Copy link
Contributor

@huonw huonw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looks great.


Keying off the the py_editable_in_resolve option behaviour sounds good, but I'm in two minds about the validation for both of the options. Not enough to block the incremental improvement, though.

In particular, if someone has the following, it seems... unhelpful for Pants to blithely ignore py_hermetic_scripts and not provide the user any hints about it. If they're not getting the behaviour they need, it becomes up to them to either remember to go check the docs, or more likely (people don't read docs 😅 ) first spend a bunch of dig into the (immutable) venv and notice that the scripts are still hermetic.

# pants.toml
[export]
py_resolve_format = "symlinked_immutable_virtualenv"
py_hermetic_scripts = False

I agree with the point about strict validation being annoying if overriding options on the CLI, though, so, happy to get this in.

Also removes a reference to legacy exports which were deprecated
and removed in earlier releases.
Comment on lines -98 to -99
NOTE: If you are using legacy exports (not using the '--resolve' option), then
this option has no effect. Legacy exports will not include any editable installs.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops. We didn't remove this note when legacy exports were removed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

export Python virtualenvs with non-hermetic console scripts

3 participants