Skip to content

Introduce initial version 2 and stronger versioning support for the spec#384

Merged
jsoriano merged 22 commits intoelastic:mainfrom
jsoriano:format-version-unique
Aug 24, 2022
Merged

Introduce initial version 2 and stronger versioning support for the spec#384
jsoriano merged 22 commits intoelastic:mainfrom
jsoriano:format-version-unique

Conversation

@jsoriano
Copy link
Copy Markdown
Member

@jsoriano jsoriano commented Jul 28, 2022

What does this PR do?

This PR introduces some changes in how versioning is done in the spec:

  • Versioned spec directories are removed.
  • format_version is strongly checked now, it must exist in the spec changelog.yml, prereleases of the version are allowed.
  • Multiple prereleases may exist in the changelog file, one for each major. This allows to keep development of multiple majors in a single branch. Some alternatives for this:
    • Keep a separated changelog.next.yml file for features only enabled for next major.
    • Have a branch per major, and use backports.
  • Semantic rules can be selected by version now. Validation of unique fields definition (Enable check to avoid duplicated fields #331) has been re-enabled to illustrate that.
  • YAML Schema files can now include a versions list, with patches to apply to reconstruct older versions of the schema. license field is removed from manifests to illustrate that.

Why is it important?

To allow introducing breaking changes (or loosening validation) in a more controlled way.

Checklist

Related issues

@jsoriano jsoriano added the discuss Issue needs discussion label Jul 28, 2022
@jsoriano jsoriano self-assigned this Jul 28, 2022
@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Jul 28, 2022

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-08-23T14:04:41.850+0000

  • Duration: 8 min 22 sec

Test stats 🧪

Test Results
Failed 0
Passed 585
Skipped 0
Total 585

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@elasticmachine
Copy link
Copy Markdown

elasticmachine commented Jul 28, 2022

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (7/7) 💚
Files 70.833% (17/24) 👍
Classes 78.788% (26/33) 👍
Methods 58.824% (60/102) 👍 0.408
Lines 44.629% (536/1201) 👍 0.754
Conditionals 100.0% (0/0) 💚

@jsoriano jsoriano mentioned this pull request Jul 28, 2022
2 tasks
@jsoriano jsoriano mentioned this pull request Aug 15, 2022
4 tasks
@jsoriano jsoriano marked this pull request as ready for review August 17, 2022 10:30
@jsoriano jsoriano requested a review from a team as a code owner August 17, 2022 10:30
mtojek
mtojek previously approved these changes Aug 22, 2022
Copy link
Copy Markdown
Contributor

@mtojek mtojek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went back and forth through this PR a few times as it's a relatively big one, but it brings the schema patching feature. I haven't noticed anything concerning or blocking.

LGTM!

Comment thread code/go/internal/spec_test.go Outdated
func TestBundledSpecsForIntegration(t *testing.T) {
fs := spec.FS()
_, err := fs.Open("1/integration/spec.yml")
_, err := fs.Open("integration/spec.yml")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most likely it will break the docs repository. You will need another PR to fix it. FYI it isn't a major problem.

Copy link
Copy Markdown
Member Author

@jsoriano jsoriano Aug 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I will prepare the follow-up.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have updated the pull request template in this repository, and opened PRs in the integrations and docs repositories. Also added a placeholder to avoid broken links: https://github.com/elastic/package-spec/blob/26925a80d08a39cd0d188ba81bc71b09464b1d04/versions/1/README.md

Comment thread spec.go Outdated
func CheckVersion(version *semver.Version) (*semver.Version, error) {
d, err := fs.ReadFile(content, "spec/changelog.yml")
if err != nil {
panic(err)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it safe to panic here? Is it executed only during init?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I followed the same approach as in FS() when accessing the filesystem. But yeah, it'd be better to don't panic here.

@mtojek
Copy link
Copy Markdown
Contributor

mtojek commented Aug 22, 2022

/test

mrodm
mrodm previously approved these changes Aug 22, 2022
Copy link
Copy Markdown
Contributor

@mrodm mrodm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This approach to get a new specification version (V2) looks promising! 🚀
I've been thinking in some scenarios like removing fields (as in the license property example) or change allowed values for some property (it would be a replace operation), and it would fit perfectly 👍

Comment thread README.md Outdated
Comment on lines +56 to +59
specification for a package's contents. It describes the the folder structure of packages and expected
files within these folders (this is point 1. above). The specification is expressed using a schema simil
ar to [JSON Schema](https://json-schema.org/), but with a couple of differences:
-- The `type` field can be either `folder` or `file`,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
specification for a package's contents. It describes the the folder structure of packages and expected
files within these folders (this is point 1. above). The specification is expressed using a schema simil
ar to [JSON Schema](https://json-schema.org/), but with a couple of differences:
-- The `type` field can be either `folder` or `file`,
specification for a package's contents. It describes the folder structure of packages and expected
files within these folders (this is point 1. above). The specification is expressed using a schema similar
to [JSON Schema](https://json-schema.org/), but with a couple of differences:
-- The `type` field can be either `folder` or `file`,

for pkgName, expectedErrorMessage := range tests {
t.Run(pkgName, func(t *testing.T) {
errs := ValidateFromPath(filepath.Join("..", "..", "..", "..", "test", "packages", pkgName))
errs := ValidateFromPath(path.Join("..", "..", "..", "..", "test", "packages", pkgName))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@jsoriano
Copy link
Copy Markdown
Member Author

I have done more updates on the README to asjust the text to current structure. And added tests to ensure that possible conflicts in patches are detected for all versions defined in changelog.

@jsoriano jsoriano requested review from mrodm and mtojek August 23, 2022 09:57
mrodm
mrodm previously approved these changes Aug 23, 2022
Copy link
Copy Markdown
Contributor

@mrodm mrodm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Comment thread README.md Outdated
Copy link
Copy Markdown
Contributor

@mrodm mrodm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@jsoriano jsoriano merged commit a6b8e9d into elastic:main Aug 24, 2022
@jsoriano jsoriano deleted the format-version-unique branch August 24, 2022 09:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

discuss Issue needs discussion

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Introduce new package spec version Enable check to avoid duplicated fields

4 participants