Skip to content

Remove deprecated corpora.generator.size in favour of `corpora.gene…#639

Merged
mrodm merged 4 commits intoelastic:mainfrom
francescayeye:corpora-generator-tot-events
Oct 11, 2023
Merged

Remove deprecated corpora.generator.size in favour of `corpora.gene…#639
mrodm merged 4 commits intoelastic:mainfrom
francescayeye:corpora-generator-tot-events

Conversation

@francescayeye
Copy link
Copy Markdown

…rator.tot_events`

What does this PR do?

Remove deprecated corpora.generator.size in favour of corpora.generator.tot_events for benchmark/spec.yml

Why is it important?

Corpus generator tool remove since v0.6.0 size in favour of totEvents: this is now reflected in the package-spec so that we will be able to bump the library. Since it's a breaking-change it would be better to include in 3.0.0

Checklist

Related issues

@francescayeye francescayeye self-assigned this Oct 6, 2023
@francescayeye francescayeye requested a review from a team as a code owner October 6, 2023 09:16
additionalProperties: false
required:
- size
- tot_events
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be called total_events? Or it is too late to rename?

Suggested change
- tot_events
- total_events

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tot_events is already part of the spec, but it's not used anywhere yet but for this PR and one in elastic-package.
if we include in the 3.0.0 we can rename taking the opportunity of the breaking change, up to you

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is not being used, let's rename to total_events if possible, as we are doing this a breaking change in any case.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done!

- description: Using non-GA versions of the spec in GA packages produces a filterable validation error instead of a warning.
type: enhancement
link: https://github.com/elastic/package-spec/pull/627
- description: Remove deprecated `corpora.generator.size` in favour of `corpora.generator.tot_events`
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can still include it in 3.0.0.

Comment on lines -74 to -78
description: A string describing the amount of data to generate.
type: string
example: 10MiB
deprecated: true # as for https://github.com/elastic/elastic-integration-corpus-generator-tool/pull/94
tot_events:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In order to not break current packages using these variables.
Should we add a json patch to at least add size parameter for packages using spec version <3.0.0 ? cc @jsoriano

Probably, this would be a similar example:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't found any usage of these benchmark folders in the integrations repository.

Commands used in the integrations repository:

 $ find . -type d -name system  | grep benchmark
 $ find . -type d -name benchmark

Maybe it's not needed to add a JSON Patch to re-add the size parameter.

WDYT ?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mrodm yes, to my knowledge we added the specs but the reference to them is still only it the specs itself and in the elastic-package repo

that's why I was positive to not deal with a migration path like you suggested

once we merge this and make part of v3 elastic/elastic-package#1491 will be green and everything will be fine

what do you think?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I agree that we don't need the migration path as this is not used yet.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mrodm yes, to my knowledge we added the specs but the reference to them is still only it the specs itself and in the elastic-package repo

that's why I was positive to not deal with a migration path like you suggested

Perfect, thanks for checking this out. Agreed, it's not need to add the JSON patch 👍

@elasticmachine
Copy link
Copy Markdown

💚 Build Succeeded

History

cc @aspacca

Comment on lines -74 to -78
description: A string describing the amount of data to generate.
type: string
example: 10MiB
deprecated: true # as for https://github.com/elastic/elastic-integration-corpus-generator-tool/pull/94
tot_events:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mrodm yes, to my knowledge we added the specs but the reference to them is still only it the specs itself and in the elastic-package repo

that's why I was positive to not deal with a migration path like you suggested

Perfect, thanks for checking this out. Agreed, it's not need to add the JSON patch 👍

@mrodm mrodm merged commit 0998b91 into elastic:main Oct 11, 2023
@francescayeye
Copy link
Copy Markdown
Author

@jsoriano . @mrodm thanks for the prompt review

do we have time to include this PR in v3.0.0? do we need to release a v3.0.0-rc2?

@jsoriano
Copy link
Copy Markdown
Member

do we have time to include this PR in v3.0.0? do we need to release a v3.0.0-rc2?

Yes, we will include it in v3.0.0, that we will likely release today or tomorrow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants