:qa:os:destructiveBatsTest.upgrade FAILED
1..12
ok 1 [UPGRADE] install old version
ok 2 [UPGRADE] modify keystore
ok 3 [UPGRADE] start old version
ok 4 [UPGRADE] check elasticsearch version is old version
ok 5 [UPGRADE] index some documents into a few indexes
ok 6 [UPGRADE] verify that the documents are there
ok 7 [UPGRADE] stop old version
ok 8 [UPGRADE] install version under test
not ok 9 [UPGRADE] start version under test
# (from function `wait_for_elasticsearch_status' in file /var/lib/jenkins/workspace/elastic+elasticsearch+master+multijob+packaging-tests-unix-gcp/os/oraclelinux-7-packaging/qa/os/bats/utils/utils.bash, line 474,
#  from function `start_elasticsearch_service' in file /var/lib/jenkins/workspace/elastic+elasticsearch+master+multijob+packaging-tests-unix-gcp/os/oraclelinux-7-packaging/qa/os/bats/utils/utils.bash, line 295,
#  in test file /var/lib/jenkins/workspace/elastic+elasticsearch+master+multijob+packaging-tests-unix-gcp/os/oraclelinux-7-packaging/qa/os/bats/upgrade/80_upgrade.bats, line 113)
#   `start_elasticsearch_service yellow library' failed
# BWC test version: 7.1.0
# Making sure elasticsearch is up...
# --2020-07-05 09:18:41--  http://localhost:9200/_cluster/health
# Resolving localhost (localhost)... ::1, 127.0.0.1
# Connecting to localhost (localhost)|::1|:9200... connected.
# HTTP request sent, awaiting response... 200 OK
# Length: 385 [application/json]
# Saving to: ‘STDOUT’
# {"cluster_name":"elasticsearch","status":"red","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":0,"active_shards":0,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":4,"delayed_unassigned_shards":0,"number_of_pending_tasks":1,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":0.0}
#      0K                                                       100% 46.6M=0s
# 
# 2020-07-05 09:18:41 (46.6 MB/s) - written to stdout [385/385]
# 
# Trying to connect to elasticsearch and wait for expected status yellow for index library
# {
#   "cluster_name" : "elasticsearch",
#   "status" : "yellow",
#   "timed_out" : false,
#   "number_of_nodes" : 1,
#   "number_of_data_nodes" : 1,
#   "active_primary_shards" : 1,
#   "active_shards" : 1,
#   "relocating_shards" : 0,
#   "initializing_shards" : 0,
#   "unassigned_shards" : 1,
#   "delayed_unassigned_shards" : 0,
#   "number_of_pending_tasks" : 3,
#   "number_of_in_flight_fetch" : 0,
#   "task_max_waiting_in_queue_millis" : 35,
#   "active_shards_percent_as_number" : 25.0
# }
# Connected
# Checking that the cluster health matches the waited for status...
# unexpected status:  'red' wanted 'yellow'
# Here's the elasticsearch log:
# [2020-07-05T09:18:12,970][INFO ][o.e.e.NodeEnvironment    ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] using [1] data paths, mounts [[/ (/dev/sda1)]], net usable_space [304.5gb], net total_space [349.9gb], types [xfs]
# [2020-07-05T09:18:12,974][INFO ][o.e.e.NodeEnvironment    ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] heap size [989.8mb], compressed ordinary object pointers [true]
# [2020-07-05T09:18:12,976][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] node name [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556], node ID [qyzX7fyUSx67ui4CSahBWg], cluster name [elasticsearch]
# [2020-07-05T09:18:12,977][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] version[7.1.0], pid[15309], build[oss/rpm/606a173/2019-05-16T00:43:15.323135Z], OS[Linux/4.14.35-1902.303.5.3.el7uek.x86_64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/11.0.2/11.0.2+7]
# [2020-07-05T09:18:12,977][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] JVM home [/var/lib/jenkins/.java/openjdk-11.0.2-linux]
# [2020-07-05T09:18:12,979][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-10952062091795249794, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/elasticsearch, -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -Dio.netty.allocator.type=unpooled, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/etc/elasticsearch, -Des.distribution.flavor=oss, -Des.distribution.type=rpm, -Des.bundled_jdk=true]
# [2020-07-05T09:18:13,840][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [aggs-matrix-stats]
# [2020-07-05T09:18:13,841][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [analysis-common]
# [2020-07-05T09:18:13,841][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [ingest-common]
# [2020-07-05T09:18:13,841][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [ingest-geoip]
# [2020-07-05T09:18:13,842][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [ingest-user-agent]
# [2020-07-05T09:18:13,842][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [lang-expression]
# [2020-07-05T09:18:13,842][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [lang-mustache]
# [2020-07-05T09:18:13,843][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [lang-painless]
# [2020-07-05T09:18:13,843][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [mapper-extras]
# [2020-07-05T09:18:13,843][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [parent-join]
# [2020-07-05T09:18:13,844][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [percolator]
# [2020-07-05T09:18:13,844][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [rank-eval]
# [2020-07-05T09:18:13,844][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [reindex]
# [2020-07-05T09:18:13,845][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [repository-url]
# [2020-07-05T09:18:13,845][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [transport-netty4]
# [2020-07-05T09:18:13,846][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] no plugins loaded
# [2020-07-05T09:18:16,752][INFO ][o.e.d.DiscoveryModule    ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] using discovery type [zen] and seed hosts providers [settings]
# [2020-07-05T09:18:17,232][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] initialized
# [2020-07-05T09:18:17,233][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] starting ...
# [2020-07-05T09:18:17,383][INFO ][o.e.t.TransportService   ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
# [2020-07-05T09:18:17,395][WARN ][o.e.b.BootstrapChecks    ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
# [2020-07-05T09:18:17,414][INFO ][o.e.c.c.ClusterBootstrapService] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] no discovery configuration found, will perform best-effort cluster bootstrapping after [3s] unless existing master is discovered
# [2020-07-05T09:18:20,423][INFO ][o.e.c.c.Coordinator      ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] setting initial configuration to VotingConfiguration{qyzX7fyUSx67ui4CSahBWg}
# [2020-07-05T09:18:20,575][INFO ][o.e.c.s.MasterService    ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] elected-as-master ([1] nodes joined)[{elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556}{qyzX7fyUSx67ui4CSahBWg}{2Me0Xe1rRL-oxr7Y-gdKmQ}{127.0.0.1}{127.0.0.1:9300} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 1, version: 1, reason: master node changed {previous [], current [{elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556}{qyzX7fyUSx67ui4CSahBWg}{2Me0Xe1rRL-oxr7Y-gdKmQ}{127.0.0.1}{127.0.0.1:9300}]}
# [2020-07-05T09:18:20,599][INFO ][o.e.c.c.CoordinationState] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] cluster UUID set to [N2iM-pl0Sm-AscxIHpqSdA]
# [2020-07-05T09:18:20,613][INFO ][o.e.c.s.ClusterApplierService] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] master node changed {previous [], current [{elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556}{qyzX7fyUSx67ui4CSahBWg}{2Me0Xe1rRL-oxr7Y-gdKmQ}{127.0.0.1}{127.0.0.1:9300}]}, term: 1, version: 1, reason: Publication{term=1, version=1}
# [2020-07-05T09:18:20,638][INFO ][o.e.g.GatewayService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] recovered [0] indices into cluster_state
# [2020-07-05T09:18:20,642][INFO ][o.e.h.AbstractHttpServerTransport] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
# [2020-07-05T09:18:20,642][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] started
# [2020-07-05T09:18:21,991][INFO ][o.e.c.m.MetaDataCreateIndexService] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] [library] creating index, cause [auto(bulk api)], templates [], shards [1]/[1], mappings []
# [2020-07-05T09:18:22,416][INFO ][o.e.c.m.MetaDataMappingService] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] [library/xXw5ropCSGC2ZnCmiBCGJw] create_mapping [_doc]
# [2020-07-05T09:18:22,555][INFO ][o.e.c.m.MetaDataCreateIndexService] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] [library2] creating index, cause [auto(bulk api)], templates [], shards [1]/[1], mappings []
# [2020-07-05T09:18:22,647][INFO ][o.e.c.m.MetaDataMappingService] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] [library2/KTzwhwojT52gmset37TRhw] create_mapping [_doc]
# [2020-07-05T09:18:23,093][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] stopping ...
# [2020-07-05T09:18:23,171][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] stopped
# [2020-07-05T09:18:23,171][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] closing ...
# [2020-07-05T09:18:23,180][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] closed
# [2020-07-05T09:18:36,238][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] version[8.0.0-SNAPSHOT], pid[16876], build[oss/rpm/283cd8369d76c51e3f578dadf26cee7b8fc5f906/2020-07-05T09:07:21.960995Z], OS[Linux/4.14.35-1902.303.5.3.el7uek.x86_64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/11.0.2/11.0.2+7]
# [2020-07-05T09:18:36,243][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] JVM home [/var/lib/jenkins/.java/openjdk-11.0.2-linux]
# [2020-07-05T09:18:36,244][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.io.tmpdir=/tmp/elasticsearch-18271609915702022712, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/elasticsearch, -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=536870912, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/etc/elasticsearch, -Des.distribution.flavor=oss, -Des.distribution.type=rpm, -Des.bundled_jdk=true]
# [2020-07-05T09:18:36,245][WARN ][o.e.n.Node               ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] version [8.0.0-SNAPSHOT] is a pre-release version of Elasticsearch and is not suitable for production
# [2020-07-05T09:18:37,248][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [aggs-matrix-stats]
# [2020-07-05T09:18:37,248][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [analysis-common]
# [2020-07-05T09:18:37,248][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [geo]
# [2020-07-05T09:18:37,248][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [ingest-common]
# [2020-07-05T09:18:37,249][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [ingest-geoip]
# [2020-07-05T09:18:37,249][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [ingest-user-agent]
# [2020-07-05T09:18:37,249][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [kibana]
# [2020-07-05T09:18:37,249][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [lang-expression]
# [2020-07-05T09:18:37,249][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [lang-mustache]
# [2020-07-05T09:18:37,250][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [lang-painless]
# [2020-07-05T09:18:37,250][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [mapper-extras]
# [2020-07-05T09:18:37,250][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [parent-join]
# [2020-07-05T09:18:37,250][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [percolator]
# [2020-07-05T09:18:37,251][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [rank-eval]
# [2020-07-05T09:18:37,251][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [reindex]
# [2020-07-05T09:18:37,251][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [repository-url]
# [2020-07-05T09:18:37,251][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [systemd]
# [2020-07-05T09:18:37,251][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [tasks]
# [2020-07-05T09:18:37,252][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] loaded module [transport-netty4]
# [2020-07-05T09:18:37,252][INFO ][o.e.p.PluginsService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] no plugins loaded
# [2020-07-05T09:18:37,312][INFO ][o.e.e.NodeEnvironment    ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] using [1] data paths, mounts [[/ (/dev/sda1)]], net usable_space [304.5gb], net total_space [349.9gb], types [xfs]
# [2020-07-05T09:18:37,313][INFO ][o.e.e.NodeEnvironment    ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] heap size [989.8mb], compressed ordinary object pointers [true]
# [2020-07-05T09:18:37,318][INFO ][o.e.e.NodeEnvironment    ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] upgrading legacy data folders: [/var/lib/elasticsearch]
# [2020-07-05T09:18:37,385][INFO ][o.e.e.NodeEnvironment    ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] data folder upgrade: moved from [/var/lib/elasticsearch/nodes/0/indices] to [/var/lib/elasticsearch/indices]
# [2020-07-05T09:18:37,385][INFO ][o.e.e.NodeEnvironment    ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] data folder upgrade: moved from [/var/lib/elasticsearch/nodes/0/_state] to [/var/lib/elasticsearch/_state]
# [2020-07-05T09:18:37,395][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] node name [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556], node ID [qyzX7fyUSx67ui4CSahBWg], cluster name [elasticsearch]
# [2020-07-05T09:18:40,595][INFO ][o.e.d.DiscoveryModule    ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] using discovery type [zen] and seed hosts providers [settings]
# [2020-07-05T09:18:41,100][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] initialized
# [2020-07-05T09:18:41,101][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] starting ...
# [2020-07-05T09:18:41,273][INFO ][o.e.t.TransportService   ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
# [2020-07-05T09:18:41,561][WARN ][o.e.b.BootstrapChecks    ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
# [2020-07-05T09:18:41,563][INFO ][o.e.c.c.Coordinator      ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] cluster UUID [N2iM-pl0Sm-AscxIHpqSdA]
# [2020-07-05T09:18:41,576][INFO ][o.e.c.c.ClusterBootstrapService] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] no discovery configuration found, will perform best-effort cluster bootstrapping after [3s] unless existing master is discovered
# [2020-07-05T09:18:41,718][INFO ][o.e.c.s.MasterService    ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] elected-as-master ([1] nodes joined)[{elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556}{qyzX7fyUSx67ui4CSahBWg}{LoEzAaAeTOmv6Q28m1Jfzw}{127.0.0.1}{127.0.0.1:9300}{dimr} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 2, version: 9, delta: master node changed {previous [], current [{elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556}{qyzX7fyUSx67ui4CSahBWg}{LoEzAaAeTOmv6Q28m1Jfzw}{127.0.0.1}{127.0.0.1:9300}{dimr}]}
# [2020-07-05T09:18:41,774][INFO ][o.e.c.s.ClusterApplierService] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] master node changed {previous [], current [{elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556}{qyzX7fyUSx67ui4CSahBWg}{LoEzAaAeTOmv6Q28m1Jfzw}{127.0.0.1}{127.0.0.1:9300}{dimr}]}, term: 2, version: 9, reason: Publication{term=2, version=9}
# [2020-07-05T09:18:41,796][INFO ][o.e.h.AbstractHttpServerTransport] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
# [2020-07-05T09:18:41,797][INFO ][o.e.n.Node               ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] started
# [2020-07-05T09:18:41,843][INFO ][o.e.g.GatewayService     ] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] recovered [2] indices into cluster_state
# [2020-07-05T09:18:42,332][INFO ][o.e.c.r.a.AllocationService] [elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556] current.health="YELLOW" message="Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[library2][0]]])." previous.health="RED" reason="shards started [[library2][0]]"
# Here's the tail of the log at /var/log/messages:
# Jul  5 09:17:59 elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556 systemd: Reloading.
# Jul  5 09:18:05 elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556 systemd: Stopped Apply Kernel Variables.
# Jul  5 09:18:05 elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556 systemd: Stopping Apply Kernel Variables...
# Jul  5 09:18:05 elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556 systemd: Starting Apply Kernel Variables...
# Jul  5 09:18:05 elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556 systemd: Started Apply Kernel Variables.
# Jul  5 09:18:10 elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556 systemd: Reloading.
# Jul  5 09:18:10 elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556 systemd: Reloading.
# Jul  5 09:18:10 elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556 systemd: Started Elasticsearch.
# Jul  5 09:18:10 elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556 elasticsearch: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
# Jul  5 09:18:23 elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556 systemd: Stopping Elasticsearch...
# Jul  5 09:18:23 elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556 systemd: Stopped Elasticsearch.
# Jul  5 09:18:28 elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556 systemd: Stopped Apply Kernel Variables.
# Jul  5 09:18:28 elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556 systemd: Stopping Apply Kernel Variables...
# Jul  5 09:18:28 elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556 systemd: Starting Apply Kernel Variables...
# Jul  5 09:18:28 elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556 systemd: Started Apply Kernel Variables.
# Jul  5 09:18:32 elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556 systemd: Reloading.
# Jul  5 09:18:32 elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556 systemd: Reloading.
# Jul  5 09:18:32 elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556 systemd: Starting Elasticsearch...
# Jul  5 09:18:34 elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556 systemd-entrypoint: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
# Jul  5 09:18:41 elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556 systemd: Started Elasticsearch.
# Current java processes:
# jenkins     1597  4.1  1.9 14297652 632468 ?     Sl   09:06   0:30 java -Djava.util.logging.config.file=/var/lib/jenkins/logging.properties -javaagent:/opt/jolokia/jolokia-agent.jar=config=/opt/jolokia/jolokia.properties -Dorg.jenkinsci.plugins.gitclient.Git.timeOut=20 -jar /usr/local/share/jenkins/swarm-client-3.17.jar -description OracleLinux 7.8 - 4 CPU Cores / 52056 MB Memory -disableClientsUniqueId -executors 1 -fsroot /var/lib/jenkins -master https://elasticsearch-ci.elastic.co/ -maxRetryInterval 160 -retryBackOffStrategy linear -mode exclusive -name elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556 -username local-admin -passwordFile /var/lib/jenkins/.jenkins_password -name elasticsearch-ci-immutable-oraclelinux-7-p-1593939877159790556 -master https://elasticsearch-ci.elastic.co -username local-admin -labels oraclelinux-7-packaging -labels immutable -description MachineType:custom-16-32768,DiskSize:350
# jenkins     1874  3.5  1.4 12994748 461332 ?     Sl   09:06   0:25 java -server -Drunbld.version=1.6.12 -jar /usr/local/bin/runbld --redirect-stderr --last-good-commit elastic+elasticsearch+master+git+push /tmp/jenkins4336337036981893893.sh
# root        2356  0.0  0.0 242812  7700 ?        S    09:07   0:00 sudo -E env PATH=/var/lib/jenkins/.java/openjdk14/bin:/sbin:/bin:/usr/sbin:/usr/bin RUNTIME_JAVA_HOME=/var/lib/jenkins/.java/openjdk-11.0.2-linux --unset=JAVA_HOME SYSTEM_JAVA_HOME=/var/lib/jenkins/.java/openjdk-11.0.2-linux ./gradlew -g /var/lib/jenkins/.gradle --scan --parallel --continue --build-cache -Dorg.elasticsearch.build.cache.url=https://gradle-enterprise.elastic.co/cache/ -Dbwc.checkout.align=true destructivePackagingTest
# root        2358  0.7  0.3 3732996 114944 ?      Sl   09:07   0:05 java -Xmx64m -Xms64m -XX:+HeapDumpOnOutOfMemoryError -Xmx128m -Xms128m -Dorg.gradle.appname=gradlew -classpath /var/lib/jenkins/workspace/elastic+elasticsearch+master+multijob+packaging-tests-unix-gcp/os/oraclelinux-7-packaging/gradle/wrapper/gradle-wrapper.jar org.gradle.wrapper.GradleWrapperMain -g /var/lib/jenkins/.gradle --scan --parallel --continue --build-cache -Dorg.elasticsearch.build.cache.url=https://gradle-enterprise.elastic.co/cache/ -Dbwc.checkout.align=true destructivePackagingTest
# root        2413  115  5.3 13550056 1759532 ?    Ssl  09:07  13:18 /var/lib/jenkins/.java/openjdk-14.0.1-linux/bin/java -XX:+HeapDumpOnOutOfMemoryError -Xss2m --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.lang.invoke=ALL-UNNAMED --add-opens java.prefs/java.util.prefs=ALL-UNNAMED -Xmx3g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -cp /var/lib/jenkins/.gradle/wrapper/dists/gradle-6.5-all/2oz4ud9k3tuxjg84bbf55q0tn/gradle-6.5/lib/gradle-launcher-6.5.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 6.5
# elastic+   16876  371  4.2 8014788 1379476 ?     Ssl  09:18   0:37 /var/lib/jenkins/.java/openjdk-11.0.2-linux/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.locale.providers=SPI,COMPAT -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.io.tmpdir=/tmp/elasticsearch-18271609915702022712 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/lib/elasticsearch -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m -XX:MaxDirectMemorySize=536870912 -Des.path.home=/usr/share/elasticsearch -Des.path.conf=/etc/elasticsearch -Des.distribution.flavor=oss -Des.distribution.type=rpm -Des.bundled_jdk=true -cp /usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -p /var/run/elasticsearch/elasticsearch.pid --quiet
# Testing if ES ports are open:
# port 9200 on host 127.0.0.1 is open
# /var/lib/jenkins/workspace/elastic+elasticsearch+master+multijob+packaging-tests-unix-gcp/os/oraclelinux-7-packaging/qa/os/bats/utils/utils.bash: connect: Connection refused
# /var/lib/jenkins/workspace/elastic+elasticsearch+master+multijob+packaging-tests-unix-gcp/os/oraclelinux-7-packaging/qa/os/bats/utils/utils.bash: line 390: /dev/tcp/127.0.0.1/9201: Connection refused
# port 9201 on host 127.0.0.1 is not open
ok 10 [UPGRADE] check elasticsearch version is version under test
ok 11 [UPGRADE] verify that the documents are there after restart
ok 12 [UPGRADE] cleanup version under test