Elasticsearch version (bin/elasticsearch --version):

6.6.1

JVM version (java -version):

Java 8 (1.8.0_121), Java 9 (build 9+181), Java 10 (10.0.2), Java 11 (11.0.2)

OS version (uname -a if on a Unix-like system):

Ubuntu 18.04.1, but probably reproducible on any linux system

Description of the problem including expected versus actual behavior:

problem: If path to temp directory contains space, elasticsearch fails to start.
expected behavior: elastissearch starts and uses specified temp directory

Steps to reproduce:

Problem can be reproduced using both env variable ES_TMPDIR and jvm option java.io.tmpdir in jvm.options file

To reproduce with env variable:

1. unpack elastic tar package into /tmp/elasticsearch directory
2. create temp directory to be used by elasticsearch: mkdir -p "/tmp/has space/temp_for_es"
3. start ES specifying env variable ES_TMPDIR:

export ES_TMP_DIR='/tmp/temp_for_es' && /tmp/elasticsearch/bin/elasticsearch

4. startup fails with following error message:

Error: Could not find or load main class space.temp_for_es

To reproduce with jvm.options file:

1. unpack elastic tar package into /tmp/elasticsearch directory
2. create temp directory to be used by elasticsearch: mkdir -p "/tmp/has space/temp_for_es"
3. change value of property java.io.tmpdir in from -Djava.io.tmpdir=${ES_TMPDIR} to -Djava.io.tmpdir=/tmp/has space/temp_for_es
4. start ES:

/tmp/elasticsearch/bin/elasticsearch

4. startup fails with following error message:

Error: Could not find or load main class space.temp_for_es

Comment on the problem
The cause of the error is that the command line generated by the bin/elastissearch script does not correctly escape jvm options.

bin/elastissearch run with -x will display command line such as the following:

exec /opt/java/1.8/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.io.tmpdir=/tmp/has space/temp_for_es -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=logs/hs_err_pid%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:logs/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=32 -XX:GCLogFileSize=64m -Des.path.home=/tmp/elasticsearch -Des.path.conf=/tmp/elasticsearch/config -Des.distribution.flavor=default -Des.distribution.type=tar -cp '/tmp/elasticsearch/lib/*' org.elasticsearch.bootstrap.Elasticsearch

where we see that jvm option java.io.tmpDir is not escaped:
-Djava.io.tmpdir=/tmp/has space/temp_for_es.

This is interpreted by the java program as "/tmp/has" being the value of java.io.tmpdir option and space/temp_for_es the java class to be executed.