Skip to content

Deprecate SafeYAML in favor of Psych#8772

Open
fauno wants to merge 5 commits intojekyll:masterfrom
fauno:psych
Open

Deprecate SafeYAML in favor of Psych#8772
fauno wants to merge 5 commits intojekyll:masterfrom
fauno:psych

Conversation

@fauno
Copy link
Copy Markdown
Member

@fauno fauno commented Aug 19, 2021

This is a 🔨 code refactoring.

Summary

I've had this on my mind for some time and I found it was planned a long time ago at #3951 so I gave it a try.

I expected it to be a little slower but script/profile-docs returns similar build times. Given dtao/safe_yaml#99 maybe I should add a benchmark?

Context

Closes #3951

@fauno fauno force-pushed the psych branch 2 times, most recently from 9c494ca to f82a73a Compare August 19, 2021 17:27
@ashmaroli
Copy link
Copy Markdown
Member

There are some things here that is unclear and making me uneasy:

  • There is no deprecation here. Seems to have dropped SafeYAML entirely. What happens to downstream plugins that uses SafeYAML?
  • The module name Jekyll::YAML is ambiguous. When used without the Jekyll:: id, it is unclear whether we are calling the Ruby stdlib module (::YAML) or the one under Jekyll.
  • There is inconsistent use of read_opts: kwarg based on the diff. Why the inconsistency?
  • Lastly, the commit author and PR author seem to be different ids. Are they both the same person?

@fauno
Copy link
Copy Markdown
Member Author

fauno commented Aug 20, 2021 via email

@ashmaroli
Copy link
Copy Markdown
Member

should I add some deprecation warnings so they can start using the Jekyll wrapper?

Yes. That's the way it is usually done. We need to give time to downstream plugins to make changes as necessary since we are only going to bump the minor version in the next release.

IMHO, The correct way to go about doing this is to:

export wrapper method(s) > intercept calls to 'SafeYAML' > issue deprecation warnings > forward to associated wrapper

Without deprecation warnings and interception-forwarding, we might as well use Psych directly but that'll happen only with
Jekyll 5.0. However, now is not the time for a major version bump.

@ashmaroli
Copy link
Copy Markdown
Member

I also found some cases where SafeYAML was loading files without read options from configuration, specially when it's used on commands, how would you handle that?

My recommendation would be to not use the read_opts configuration for YAML loading at all. It is meant for File.read calls.

@ashmaroli
Copy link
Copy Markdown
Member

Note that there are two comments above this. (in case you don't see them in your email notification.)

@DirtyF DirtyF added this to the 5.0 milestone Aug 20, 2021
@fauno
Copy link
Copy Markdown
Member Author

fauno commented Aug 21, 2021 via email

@fauno
Copy link
Copy Markdown
Member Author

fauno commented Aug 21, 2021 via email

@ashmaroli
Copy link
Copy Markdown
Member

@fauno Based on the errors reported in the check_run, and in sync with @DirtyF's decision to add this to the v5.0 milestone, I'm deferring this for the future. The maintainers will revisit this if SafeYAML become unusable before that.

In the interim, you're free to continue improving the code in this branch.

@fauno fauno mentioned this pull request Dec 29, 2021
@fauno fauno mentioned this pull request Sep 28, 2022
@fauno fauno mentioned this pull request Oct 21, 2022
18 tasks
@ulm
Copy link
Copy Markdown

ulm commented Mar 30, 2023

Just a heads up, Gentoo is going to remove Jekyll packages because of the dependency on safe_yaml (which doesn't work with recent Ruby versions):
https://marc.info/?l=gentoo-dev&m=168015966309355&w=2

AFAICS Debian's Jekyll package includes this PR as a patch, 0016-Drop-usage-of-safe_yaml.patch in http://deb.debian.org/debian/pool/main/j/jekyll/jekyll_4.3.1+dfsg-1.debian.tar.xz because they no longer support Ruby versions 2.x either.

@Flowdalic
Copy link
Copy Markdown

@ashmaroli
Copy link
Copy Markdown
Member

Thank you for letting us know, @ulm and @Flowdalic.

@fauno
Copy link
Copy Markdown
Member Author

fauno commented Mar 30, 2023

Hey, I'm quite busy at the moment but I can carve some time in May

@ashmaroli
Copy link
Copy Markdown
Member

No hurry, @fauno.
We haven't started developing towards v5.0 yet.

@hartwork
Copy link
Copy Markdown

No hurry, @fauno. We haven't started developing towards v5.0 yet.

@ashmaroli safe_yaml is beyond dead, Debian packaging was apparantly forced to apply this pull request here before being merged (which is not common practice and signals urgency) and the Gentoo deadline for remove is 2023-04-30 at the moment, so "no hurry" is not realistic here. Maybe it can be moved, but we have a fire here already, not smoke, not potential future smoke. What am I missing?

CC @ulm @fauno

@ashmaroli
Copy link
Copy Markdown
Member

@hartwork I understand the situation with safe_yaml being dead an all. Ruby 2.7 (the last in Ruby 2.x) itself reaches EOL tomorrow (31/03/2023). That means, the Ruby ecosystem is at a cusp of two major versions of the runtime language. Maintainers have to decide whether to continue supporting older versions and move ahead entirely.

All that said, Jekyll 4.x cannot yeet safe_yaml out to maintain backwards-compatibility with plugins out in the wild. It would have been possible gradually if Psych and SafeYAML did not differ significantly from each other. (My experience with various versions of Psych wasn't exactly smooth).

Gentoo Linux will have to forego packaging Jekyll n co, for the time being.

@hartwork
Copy link
Copy Markdown

@ashmaroli if external plugins remain using safe_yaml does that mean that yekyll itself has to keep using it? I don't understand why that would be the case, e.g. Debian would be in big trouble now after patching it out, else. Please help me understand what I'm missing.

@fauno
Copy link
Copy Markdown
Member Author

fauno commented Mar 30, 2023 via email

@ashmaroli
Copy link
Copy Markdown
Member

ashmaroli commented Mar 31, 2023

if external plugins remain using safe_yaml does that mean that yekyll itself has to keep using it? I don't understand why that would be the case...

@hartwork Jekyll follows Semantic Versioning protocol and therefore cannot remove existing runtime dependencies under the assumption that downstream plugins with requirements jekyll ~> 4.0 could be using the dependencies directly and therefore, is implicitly part of the public API. For example, a plugin calling SafeYAML.load_file(filename) is valid usage especially since the latest release Jekyll 4.3.x itself uses the exact incantation.
The way forward is to either first ship a release which deprecates using SafeYAML directly, and then remove the dependency in a future major release;; or remove the dependency directly in the very next major release.

Regarding the state of the Debian package, that would be upto respective project maintainers to worry about. Its their responsibility to have adequate tests to check if an incoming patch has an effect on existing compatibility.

@hartwork
Copy link
Copy Markdown

@hartwork Jekyll follows Semantic Versioning protocol and therefore cannot remove existing runtime dependencies under the assumption that downstream plugins with requirements jekyll ~> 4.0 could be using the dependencies directly and therefore, is implicitly part of the public API.

@ashmaroli just for the record, I would disagree on that assessment in any other project. It's the responsibility of plug-in code to explicitly depend on all direct dependencies and not rely on other software to pull them in. So that would just be a build system bug in those plugins, nothing more. Debian seems to agree with me on that.

@ashmaroli
Copy link
Copy Markdown
Member

Alright @hartwork I [agree to disagree].
However, Jekyll 4.x will not be doing away with SafeYAML. At max, it will only be deprecated.

@svillemot
Copy link
Copy Markdown

svillemot commented Aug 30, 2023

Note that this PR breaks the minimal-mistakes theme, because Psych by default does not accept YAML aliases (but it can be asked to by using aliases: true). For more details, see this Debian bug report.

@fauno
Copy link
Copy Markdown
Member Author

fauno commented Aug 30, 2023

@svillemot thanks! I added your patch on the PR with your authorship :)


# Safely load YAML strings
def safe_load_yaml(yaml)
Psych.safe_load(yaml, :permitted_classes => [Date, Time], aliases: true)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am wondering if it would make sense to add Symbol to the list of permitted classes. Indeed, with this patch in Debian, there is a bug open because of failing tests in jekyll asciidoc plugin

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1030465

caused by the fact that Symbol is not allowed to be loaded by Psych.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Build a YAML wrapper and move to Psych.safe_load.

8 participants