Skip to content

Disallow negative TimeValues#53913

Merged
AthenaEryma merged 21 commits intoelastic:masterfrom
AthenaEryma:fix-timevalue
Mar 26, 2020
Merged

Disallow negative TimeValues#53913
AthenaEryma merged 21 commits intoelastic:masterfrom
AthenaEryma:fix-timevalue

Conversation

@AthenaEryma
Copy link
Copy Markdown
Contributor

@AthenaEryma AthenaEryma commented Mar 21, 2020

This commit causes negative TimeValues, other than -1 which is sometimes used as
a sentinel value, to be rejected during parsing.

Also introduces a hack to allow ILM to load policies which were written to the
cluster state with a negative min_age, treating those values as 0, which should
match the behavior of prior versions.

Fixes #54041

This commit causes negative TimeValues, other than -1 which is sometimes used as
a sentinel value, to be rejected during parsing.
@AthenaEryma AthenaEryma added :Core/Infra/Core Core issues without another label v8.0.0 v7.7.0 v6.8.9 labels Mar 21, 2020
@elasticmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-core-infra (:Core/Infra/Core)

@AthenaEryma AthenaEryma requested a review from dakrone March 23, 2020 23:55
@AthenaEryma AthenaEryma marked this pull request as ready for review March 23, 2020 23:56
@AthenaEryma AthenaEryma requested a review from rjernst March 23, 2020 23:57
@AthenaEryma AthenaEryma added the :Data Management/ILM+SLM DO NOT USE. Use ":StorageEngine/ILM" or ":Distributed Coordination/SLM" instead. label Mar 23, 2020
@elasticmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-core-features (:Core/Features/ILM+SLM)

Copy link
Copy Markdown
Member

@jasontedor jasontedor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left some comments.

henningandersen added a commit to henningandersen/elasticsearch that referenced this pull request Mar 24, 2020
Reindex would use timeValueNanos(System.nanoTime()). The intended use
for TimeValue is as a duration, not as absolute time. In particular,
this could result in negative TimeValue's, being unsupported in elastic#53913.
@henningandersen
Copy link
Copy Markdown
Contributor

Nice catch, @gwbrown . Unfortunately, reindex used timeValueNanos(System.nanoTime()) in a few places and I worry that this fix could break that (nanoTime is allowed to return negative values). I opened #54057 to address that, which I think is a prerequisite for this PR to be merged.

henningandersen added a commit that referenced this pull request Mar 24, 2020
Reindex would use timeValueNanos(System.nanoTime()). The intended use
for TimeValue is as a duration, not as absolute time. In particular,
this could result in negative TimeValue's, being unsupported in #53913.
Modified to use the bare long nano-second value.
henningandersen added a commit that referenced this pull request Mar 24, 2020
Reindex would use timeValueNanos(System.nanoTime()). The intended use
for TimeValue is as a duration, not as absolute time. In particular,
this could result in negative TimeValue's, being unsupported in #53913.
Modified to use the bare long nano-second value.
henningandersen added a commit to henningandersen/elasticsearch that referenced this pull request Mar 24, 2020
Reindex would use timeValueNanos(System.nanoTime()). The intended use
for TimeValue is as a duration, not as absolute time. In particular,
this could result in negative TimeValue's, being unsupported in elastic#53913.
Modified to use the bare long nano-second value.
Copy link
Copy Markdown
Member

@dakrone dakrone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM assuming CI is happy, thanks for addressing this Gordon

@AthenaEryma
Copy link
Copy Markdown
Contributor Author

AthenaEryma commented Mar 25, 2020

All the tests passed but I'm going to run one more time to be sure, as many of the failures previously were due to random values, which are only sometimes negative.

@elasticmachine test this please

AthenaEryma and others added 2 commits March 25, 2020 17:25
Co-Authored-By: Lee Hinman <dakrone@users.noreply.github.com>
@AthenaEryma AthenaEryma merged commit e9bc3e8 into elastic:master Mar 26, 2020
AthenaEryma added a commit to AthenaEryma/elasticsearch that referenced this pull request Mar 26, 2020
This commit causes negative TimeValues, other than -1 which is sometimes used as
a sentinel value, to be rejected during parsing.

Also introduces a hack to allow ILM to load policies which were written to the
cluster state with a negative min_age, treating those values as 0, which should
match the behavior of prior versions.
AthenaEryma added a commit to AthenaEryma/elasticsearch that referenced this pull request Mar 26, 2020
This commit causes negative TimeValues, other than -1 which is sometimes used as
a sentinel value, to be rejected during parsing.

Also introduces a hack to allow ILM to load policies which were written to the
cluster state with a negative min_age, treating those values as 0, which should
match the behavior of prior versions.
AthenaEryma added a commit that referenced this pull request Mar 26, 2020
This commit causes negative TimeValues, other than -1 which is sometimes used as
a sentinel value, to be rejected during parsing.

Also introduces a hack to allow ILM to load policies which were written to the
cluster state with a negative min_age, treating those values as 0, which should
match the behavior of prior versions.
AthenaEryma added a commit that referenced this pull request Mar 26, 2020
This commit causes negative TimeValues, other than -1 which is sometimes used as
a sentinel value, to be rejected during parsing.

Also introduces a hack to allow ILM to load policies which were written to the
cluster state with a negative min_age, treating those values as 0, which should
match the behavior of prior versions.
AthenaEryma added a commit to AthenaEryma/elasticsearch that referenced this pull request Mar 26, 2020
This commit causes negative TimeValues, other than -1 which is sometimes used as
a sentinel value, to be rejected during parsing.

Also introduces a hack to allow ILM to load policies which were written to the
cluster state with a negative min_age, treating those values as 0, which should
match the behavior of prior versions.
AthenaEryma added a commit that referenced this pull request Mar 27, 2020
* Disallow negative TimeValues (#53913)

This commit causes negative TimeValues, other than -1 which is sometimes used as
a sentinel value, to be rejected during parsing.

Also introduces a hack to allow ILM to load policies which were written to the
cluster state with a negative min_age, treating those values as 0, which should
match the behavior of prior versions.

* Re-apply IndexLifecycleRunner changes lost in merge

* Make SnapshotStatsTests generate more realistic times

* Fix negative TimeValue in deprecation test

* Re-apply CcrStatsResponseTests fix lost in merge
tlrx added a commit that referenced this pull request Apr 17, 2020
tlrx added a commit that referenced this pull request Apr 17, 2020
tlrx added a commit that referenced this pull request Apr 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>bug :Core/Infra/Core Core issues without another label :Data Management/ILM+SLM DO NOT USE. Use ":StorageEngine/ILM" or ":Distributed Coordination/SLM" instead. v6.8.9 v7.7.0 v7.8.0 v8.0.0-alpha1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Creating an ILM policy that has a negative min_age can cause problems loading cluster state

7 participants