Environment details

1. gax-java/gax
2. OS type and version: distroless java17-debian12 on GKE
3. Java version: java 17
4. artifact version(s): 2.54.1

Steps to reproduce

1. Create a google sheet external table in BigQuery
2. Create Credentials with GoogleCredentialsProvider adding drive scope
3. Use the credentials created to query the table created from

Code example

val credentialsScope = List(
   "https://www.googleapis.com/auth/cloud-platform",
   "https://www.googleapis.com/auth/drive",
)

val credentials = GoogleCredentialsProvider
  .newBuilder
  .setScopesToApply(credentialsScope.asJava)
  .build
  .getCredentials

val bigquery = BigQueryOptions
  .newBuilder
  .setCredentials(credentials)
  .build
  .getService

val result = bigquery.query(QueryJobConfiguration.of(
  s"SELECT * FROM ${external_table_backed_by_google_sheet}"
))

Stack trace

Access Denied: BigQuery BigQuery: Permission denied while getting Drive credentials.

Any additional information below

Replacing the CredentialsProvider with the following fix the issue:

val provider: CredentialsProvider = () =>
  GoogleCredentials.getApplicationDefault().createScoped(scopes.asJava)

Looking at gax code, it seems like GoogleCredentialsProvider only apply the scopes if the underlying credentials require them ref. However, the doc for GoogleCredentialsProvider.Builder.setScopesToApply says ref:

Sets the scopes to apply to the credentials that are acquired from Application Default Credentials, before the credentials are sent to the service.

So I would expect the scopes to be apply regardless if the returned Credentials from GoogleCredentials.getApplicationDefault requires them to be set. Note that ComputeEngineCredentials never requires scopes to be apply, but support applying scopes.

So I think that whether the doc for GoogleCredentialsProvider.Builder.setScopesToApply should be modified to reflect what it really does or scopes provided via this method should always be applied.

Doc from GoogleCredentials.createScoped ref:

If the credentials support scopes, creates a copy of the identity with the specified scopes, invalidates the existing scoped access token; otherwise, return the same instance.

So it should always be safe to apply.