Don't create tempdir for cli scripts#41913
Merged
rjernst merged 11 commits intoelastic:masterfrom May 10, 2019
Merged
Conversation
* Update TLS ciphers and protocols for JDK 11 (elastic#41385) This commit updates the default ciphers and TLS protocols that are used after the minimum supported JDK is JDK 11. The conditionals around TLSv1.3 and 256-bit cipher support have been removed. JDK 11 no longer requires an unlimited JCE policy file for 256 bit cipher support and TLSv1.3 is supported in JDK 11+. New cipher support has been introduced in the newer JDK versions as well. The ciphers are ordered with PFS ciphers being most preferred, then AEAD ciphers, and finally those with mainstream hardware support. * Fixes for TLSv1.3 on JDK11 * fix for JDK-8212885
…tic#41893) The class was called PatternSeenEventExcpectation. This commit is a straight class rename to correct the spelling.
Today if an exception is thrown when serializing a cluster state during
publication then the master enters a poisoned state where it cannot publish any
more cluster states, but nor does it stand down as master, yielding repeated
exceptions of the following form:
```
failed to commit cluster state version [12345]
org.elasticsearch.cluster.coordination.FailedToCommitClusterStateException: publishing failed
at org.elasticsearch.cluster.coordination.Coordinator.publish(Coordinator.java:1045) ~[elasticsearch-7.0.0.jar:7.0.0]
at org.elasticsearch.cluster.service.MasterService.publish(MasterService.java:252) [elasticsearch-7.0.0.jar:7.0.0]
at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:238) [elasticsearch-7.0.0.jar:7.0.0]
at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:142) [elasticsearch-7.0.0.jar:7.0.0]
at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) [elasticsearch-7.0.0.jar:7.0.0]
at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) [elasticsearch-7.0.0.jar:7.0.0]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:681) [elasticsearch-7.0.0.jar:7.0.0]
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:252) [elasticsearch-7.0.0.jar:7.0.0]
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:215) [elasticsearch-7.0.0.jar:7.0.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_144]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_144]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144]
Caused by: org.elasticsearch.cluster.coordination.CoordinationStateRejectedException: cannot start publishing next value before accepting previous one
at org.elasticsearch.cluster.coordination.CoordinationState.handleClientValue(CoordinationState.java:280) ~[elasticsearch-7.0.0.jar:7.0.0]
at org.elasticsearch.cluster.coordination.Coordinator.publish(Coordinator.java:1030) ~[elasticsearch-7.0.0.jar:7.0.0]
... 11 more
```
This is because it already created the publication request using
`CoordinationState#handleClientValue()` but then it fails before accepting it.
This commit addresses this by performing the serialization before calling
`handleClientValue()`.
Relates elastic#41090, which was the source of such a serialization exception.
The elasticsearch-cli helper script does not use the tempdir created by elasticsearch-env, yet the env script still creates it. This can lead to lots of temp directories being created when running cli scripts in an automated fashion. This commit passes a fake tmpdir to the env script to avoid creation. closes elastic#34445
Collaborator
|
Pinging @elastic/es-core-infra |
jasontedor
reviewed
May 7, 2019
| set -e -o pipefail | ||
|
|
||
| # set tempdir to non empty string so env script does not create it, as it is not used here | ||
| ES_TMPDIR="notused" |
Member
There was a problem hiding this comment.
This feels like a hack. Even if we proceed with this, a test would be nice because we're otherwise maintaining a hack without any tests. A non-hack would be ideal though. 😇
Member
Author
There was a problem hiding this comment.
I could alternatively just move the temp dir creation out of env and into bin/ES scripts? I don't have any good ideas for a test. The directory created by mktemp and equivalents is often obfuscated so I can't figure out if one was created.
Member
Author
|
@elasticmachine run elasticsearch-ci/1 |
1 similar comment
Member
Author
|
@elasticmachine run elasticsearch-ci/1 |
Member
Author
|
@elasticmachine run elasticsearch-ci/docbldesx |
rjernst
added a commit
that referenced
this pull request
May 10, 2019
The elasticsearch-cli helper script does not use the tempdir created by elasticsearch-env, yet the env script still creates it. This can lead to lots of temp directories being created when running cli scripts in an automated fashion. This commit passes a fake tmpdir to the env script to avoid creation. closes #34445
jasontedor
added a commit
to jasontedor/elasticsearch
that referenced
this pull request
May 10, 2019
* elastic/master: (84 commits) [ML] adds geo_centroid aggregation support to data frames (elastic#42088) Add documentation for calendar/fixed intervals (elastic#41919) Remove global checkpoint assertion in peer recovery (elastic#41987) Don't create tempdir for cli scripts (elastic#41913) Fix debian-8 update (elastic#42056) Cleanup plugin bin directories (elastic#41907) Prevent order being lost for _nodes API filters (elastic#42045) Change IndexAnalyzers default analyzer access (elastic#42011) Remove reference to fs.data.spins in docs Mute failing AsyncTwoPhaseIndexerTests Remove close method in PageCacheRecycler/Recycler (elastic#41917) [ML] adding pivot.max_search_page_size option for setting paging size (elastic#41920) Docs: Tweak list formatting Simplify handling of keyword field normalizers (elastic#42002) [ML] properly nesting objects in document source (elastic#41901) Remove extra `ms` from log message (elastic#42068) Increase the sample space for random inner hits name generator (elastic#42057) Recognise direct buffers in heap size docs (elastic#42070) shouldRollGeneration should execute under read lock (elastic#41696) Wait for active shard after close in mixed cluster (elastic#42029) ...
gurkankaymak
pushed a commit
to gurkankaymak/elasticsearch
that referenced
this pull request
May 27, 2019
The elasticsearch-cli helper script does not use the tempdir created by elasticsearch-env, yet the env script still creates it. This can lead to lots of temp directories being created when running cli scripts in an automated fashion. This commit passes a fake tmpdir to the env script to avoid creation. closes elastic#34445
rjernst
added a commit
to rjernst/elasticsearch
that referenced
this pull request
Jul 18, 2019
In elastic#41913 setting up the temp dir for ES was moved from the env script to individual cli scripts. However, moving it to the windows service cli was missed. This commit restores setting up the temp dir for the windows service control script.
rjernst
added a commit
that referenced
this pull request
Jul 21, 2019
In #41913 setting up the temp dir for ES was moved from the env script to individual cli scripts. However, moving it to the windows service cli was missed. This commit restores setting up the temp dir for the windows service control script.
rjernst
added a commit
to rjernst/elasticsearch
that referenced
this pull request
Jul 21, 2019
In elastic#41913 setting up the temp dir for ES was moved from the env script to individual cli scripts. However, moving it to the windows service cli was missed. This commit restores setting up the temp dir for the windows service control script.
rjernst
added a commit
to rjernst/elasticsearch
that referenced
this pull request
Jul 21, 2019
In elastic#41913 setting up the temp dir for ES was moved from the env script to individual cli scripts. However, moving it to the windows service cli was missed. This commit restores setting up the temp dir for the windows service control script.
rjernst
added a commit
to rjernst/elasticsearch
that referenced
this pull request
Jul 21, 2019
In elastic#41913 setting up the temp dir for ES was moved from the env script to individual cli scripts. However, moving it to the windows service cli was missed. This commit restores setting up the temp dir for the windows service control script.
rjernst
added a commit
that referenced
this pull request
Jul 21, 2019
In #41913 setting up the temp dir for ES was moved from the env script to individual cli scripts. However, moving it to the windows service cli was missed. This commit restores setting up the temp dir for the windows service control script.
rjernst
added a commit
that referenced
this pull request
Jul 21, 2019
In #41913 setting up the temp dir for ES was moved from the env script to individual cli scripts. However, moving it to the windows service cli was missed. This commit restores setting up the temp dir for the windows service control script.
rjernst
added a commit
that referenced
this pull request
Jul 21, 2019
In #41913 setting up the temp dir for ES was moved from the env script to individual cli scripts. However, moving it to the windows service cli was missed. This commit restores setting up the temp dir for the windows service control script.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The elasticsearch-cli helper script does not use the tempdir created by
elasticsearch-env, yet the env script still creates it. This can lead to
lots of temp directories being created when running cli scripts in an
automated fashion. This commit passes a fake tmpdir to the env script to
avoid creation.
closes #34445