This repository was archived by the owner on Apr 2, 2026. It is now read-only.
This repository was archived by the owner on Apr 2, 2026. It is now read-only.
Crash when trying to check if a missing file exists #856
Description
There's a crash if you try to check Files.exists() or similar methods on a file that does not exists, and you have autoDetectRequesterPays set.
Environment details
- cloud nio
- OS type and version: all
- Java version: all
- version(s): current and probably previous
Steps to reproduce
- autoDetectRequesterPays = true
- userProject = a valid user project
- usePseudoDirectories = true
- Files.exists on a non existent path gs:// but a valid bucket path
Code example
public void testNotRequesterPays() throws IOException, URISyntaxException {
CloudStorageConfiguration config = CloudStorageConfiguration.builder()
.autoDetectRequesterPays(true)
.userProject(A_VALID_PROJECT_NAME)
.usePseudoDirectories(true)
.build();
CloudStorageFileSystemProvider.setDefaultCloudStorageConfiguration(config);
Files.exists(Paths.get(new URI("gs://hellbender/doesnteexist")));
}Stack trace
User project specified in the request is invalid.
com.google.cloud.storage.StorageException: User project specified in the request is invalid.
at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:233)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.list(HttpStorageRpc.java:376)
at com.google.cloud.storage.StorageImpl.lambda$listBlobs$11(StorageImpl.java:391)
at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105)
at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
at com.google.cloud.storage.Retrying.run(Retrying.java:51)
at com.google.cloud.storage.StorageImpl.listBlobs(StorageImpl.java:388)
at com.google.cloud.storage.StorageImpl.list(StorageImpl.java:359)
at com.google.cloud.storage.contrib.nio.CloudStoragePath.seemsLikeADirectoryAndUsePseudoDirectories(CloudStoragePath.java:118)
at com.google.cloud.storage.contrib.nio.CloudStorageFileSystemProvider.checkAccess(CloudStorageFileSystemProvider.java:743)
at java.nio.file.Files.exists(Files.java:2385)
at org.broadinstitute.hellbender.utils.gcs.BucketUtilsUnitTest.testNotRequesterPaysBucket(BucketUtilsUnitTest.java:341)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:133)
at org.testng.internal.TestInvoker.invokeMethod(TestInvoker.java:584)
at org.testng.internal.TestInvoker.invokeTestMethod(TestInvoker.java:172)
at org.testng.internal.MethodRunner.runInSequence(MethodRunner.java:46)
at org.testng.internal.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:804)
at org.testng.internal.TestInvoker.invokeTestMethods(TestInvoker.java:145)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:128)
at java.util.ArrayList.forEach(ArrayList.java:1259)
at org.testng.TestRunner.privateRun(TestRunner.java:770)
at org.testng.TestRunner.run(TestRunner.java:591)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:402)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:396)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:355)
at org.testng.SuiteRunner.run(SuiteRunner.java:304)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:96)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1180)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1102)
at org.testng.TestNG.runSuites(TestNG.java:1032)
at org.testng.TestNG.run(TestNG.java:1000)
at org.gradle.api.internal.tasks.testing.testng.TestNGTestClassProcessor.runTests(TestNGTestClassProcessor.java:141)
at org.gradle.api.internal.tasks.testing.testng.TestNGTestClassProcessor.stop(TestNGTestClassProcessor.java:90)
at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
at com.sun.proxy.$Proxy2.stop(Unknown Source)
at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133)
at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71)
at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
GET https://storage.googleapis.com/storage/v1/b/hellbender/o?maxResults=1&prefix=doesnteexist/&projection=full&userProject
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "User project specified in the request is invalid.",
"reason" : "invalid"
} ],
"message" : "User project specified in the request is invalid."
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:428)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:514)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:455)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:565)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.list(HttpStorageRpc.java:366)
... 57 more
This is a blocker for us since we check for the possible existence of several files to find the one we are looking for.