Skip to content

Pass --profile to CLI token source and add host-key read-fallback#1497

Merged
simonfaltum merged 2 commits intomainfrom
simonfaltum/token-cache-profile-fallback
Feb 24, 2026
Merged

Pass --profile to CLI token source and add host-key read-fallback#1497
simonfaltum merged 2 commits intomainfrom
simonfaltum/token-cache-profile-fallback

Conversation

@simonfaltum
Copy link
Member

@simonfaltum simonfaltum commented Feb 24, 2026

Why

Today databricks auth token --profile logfood fails even when a valid token exists — because the token was stored under the host URL key, not the profile name key. This happens when:

  1. The user authenticated with an older CLI version that only wrote host-based cache keys
  2. The SDK's CliTokenSource refreshed the token via --host (never --profile), updating only the host key

The read side does a single-key lookup with no fallback, so --profile never finds the host-keyed token.

This is the first step toward profile-first token caching (CLI#2): making the profile name the single authoritative cache key, with host-based keys as a legacy fallback only.

Changes

1. CliTokenSource passes --profile when available

buildCliCommands now checks cfg.Profile. When set (explicit profile via --profile, DATABRICKS_CONFIG_PROFILE, or code), it passes --profile <name> only — no other flags are forwarded. The profile is the identity; the CLI loads the host and other config from the profile.

When cfg.Profile is empty (implicit DEFAULT loading, env-var-only, direct config), the existing --host path is unchanged.

A --host fallback command is also built for compatibility with older CLIs. If the --profile command fails with "unknown flag: --profile", the SDK retries with the --host command and emits a warning asking the user to upgrade their CLI.

2. Read-fallback in Token() for migration

When the profile key returns ErrNotFound, Token() now checks the host-based key as a fallback. If found, it returns the token so the caller isn't blocked. The token is not persisted to the profile key — the host key may contain a token from a different profile with different scopes. A debug log message suggests running databricks auth login --profile <name> to properly migrate.

3. Stale comment update

Updated the comment in auth_u2m.go that described the cache as host-keyed only. It now reflects that the cache is keyed by profile name (explicit) or host (legacy/implicit default).

Test plan

  • buildCliCommands with profile set → --profile only, --host fallback built
  • buildCliCommands with profile set, no host → --profile only, no fallback
  • buildCliCommands with profile empty → --host used (unchanged)
  • Token() with --profile fails as unknown flag → retries with --host, succeeds
  • Token() with --profile fails with real auth error → no retry, error propagated
  • Token() profile key miss, host key exists → returns token, no write to cache
  • Token() profile key miss, host key also miss → ErrNotFound
  • Token() profile key exists → used directly, no fallback
  • Token() no profile set → host key used directly, no fallback attempted
  • dualWrite with two profiles on same host → each gets own profile key
  • make fmt, go test ./config/, go test ./credentials/u2m/..., make lint pass

@simonfaltum simonfaltum force-pushed the simonfaltum/token-cache-profile-fallback branch from 99f0811 to 13abe25 Compare February 24, 2026 11:09
@simonfaltum simonfaltum force-pushed the simonfaltum/token-cache-profile-fallback branch from 13abe25 to ca23773 Compare February 24, 2026 11:17
@simonfaltum simonfaltum marked this pull request as ready for review February 24, 2026 11:57
Copy link
Contributor

@hectorcast-db hectorcast-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM minus one comment.

Also, the PR description still says that we are passing the Override flags.

cmd := exec.CommandContext(ctx, c.cmd[0], c.cmd[1:]...)
tok, err := c.execCliCommand(ctx, c.cmd)
if err != nil && c.hostCmd != nil && isUnknownFlagError(err) {
return c.execCliCommand(ctx, c.hostCmd)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A warning here telling the user to update their CLI would be great.

When cfg.Profile is set, buildCliCommand now passes --profile as the
primary identifier to the CLI, forwarding --host/--account-id/
--workspace-id/--experimental-is-unified-host as override flags. When
cfg.Profile is empty (implicit DEFAULT or env-var-only), the existing
--host path is unchanged.

In PersistentAuth.Token(), when the profile key lookup returns
cache.ErrNotFound and the OAuthArgument implements HostCacheKeyProvider,
a fallback lookup by host key is attempted. If found, dualWrite migrates
the token to the profile key for future lookups. This allows legacy
host-only tokens to be transparently upgraded without re-authentication.

Signed-off-by: simon <simon.faltum@databricks.com>
@simonfaltum simonfaltum force-pushed the simonfaltum/token-cache-profile-fallback branch from 99c1901 to 78cffa9 Compare February 24, 2026 13:44
@github-actions
Copy link

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/sdk-go

Inputs:

  • PR number: 1497
  • Commit SHA: 7842596a72363966300b6f62e6c829ff86b7f35d

Checks will be approved automatically on success.

@simonfaltum simonfaltum added this pull request to the merge queue Feb 24, 2026
Merged via the queue into main with commit 406a17a Feb 24, 2026
15 checks passed
@simonfaltum simonfaltum deleted the simonfaltum/token-cache-profile-fallback branch February 24, 2026 14:06
deco-sdk-tagging bot added a commit that referenced this pull request Feb 24, 2026
## Release v0.112.0

### Bug Fixes

* Pass `--profile` to CLI token source when profile is set, and add read-fallback to migrate legacy host-keyed tokens to profile keys ([#1497](#1497)).

### API Changes
* Add `Parameters` field for [pipelines.StartUpdate](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#StartUpdate).
* Add `Parameters` field for [pipelines.UpdateInfo](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#UpdateInfo).
* [Breaking] Change `GetDownloadFullQueryResult` method for [w.Genie](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#GenieAPI) workspace-level service with new required argument order.
* [Breaking] Change `Name` field for [apps.Space](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/apps#Space) to be required.
* Change `Name` field for [apps.Space](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/apps#Space) to be required.
* [Breaking] Change `Id` and `UserId` fields for [dashboards.GenieConversation](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#GenieConversation) to no longer be required.
* [Breaking] Change `CreatedTimestamp` and `Title` fields for [dashboards.GenieConversationSummary](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#GenieConversationSummary) to no longer be required.
* [Breaking] Change `DownloadIdSignature` field for [dashboards.GenieGetDownloadFullQueryResultRequest](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#GenieGetDownloadFullQueryResultRequest) to be required.
* [Breaking] Change `Id` field for [dashboards.GenieMessage](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/dashboards#GenieMessage) to no longer be required.
github-merge-queue bot pushed a commit to databricks/cli that referenced this pull request Feb 24, 2026
## Why

When you run `databricks auth token --host <url>`, the CLI looks up the
token using the host URL as the cache key. But if you logged in with
`--profile`, the token was stored under the profile name, not the host
URL. So `--host` can't find it — even though the token exists and the
profile points to exactly that host.

This is the flip side of the bug fixed in
databricks/databricks-sdk-go#1497: that PR made `--profile` fall back to
the host key. This PR makes `--host` resolve to the profile key.

Together, both directions work: `--profile` finds host-keyed tokens, and
`--host` finds profile-keyed tokens.

## Changes

Two-line change in `loadToken` (`cmd/auth/token.go`).

The CLI already loads all profiles matching a given host and errors when
there are multiple matches (the ambiguity check). This PR adds an `else
if` branch: when exactly one profile matches, use that profile's name as
the cache key instead of the raw host URL.

```go
if len(matchingProfiles) > 1 {
    // existing: ambiguity error
} else if len(matchingProfiles) == 1 {
    args.profileName = matchingProfiles[0].Name
}
```

The matching is already host-type-aware from the existing code:
- Workspace hosts match by host URL
- Account/unified hosts match by host URL + account ID

No new matching logic was added.

**Side effect**: error messages now suggest `--profile <name>` instead
of `--host <url>` when the host resolves to a profile. This is better UX
— if we know the profile, we should suggest it.

## Test plan

Three new test cases added to `TestToken_loadToken`:

1. **Host with one matching profile resolves to profile key** — token
exists only under the profile key (not the host URL). `--host` finds it
by resolving to the profile first.
2. **Host with no matching profile uses host key** — no profile in
`.databrickscfg` matches the host. Falls back to looking up by host URL
directly. Unchanged behavior.
3. **Host with one matching profile, host-key-only token (migration)** —
profile exists in `.databrickscfg` but the token was stored under the
host URL only (legacy). CLI resolves `--host` to the profile name, then
the SDK's read-fallback (from
[#1497](databricks/databricks-sdk-go#1497))
finds the token under the host key.

All existing tests continue to pass. One existing test's expected error
message changed from `--host ... --account-id ...` to `--profile
expired` because the host now resolves to the `expired` profile.

### Manual testing

I tested this manually with my local `.databrickscfg`:

```
$ databricks auth token --host https://adb-2548836972759138.18.azuredatabricks.net/
```

This host matches my `logfood` profile. Before this change, it would
look up by host URL. After this change, it resolves to `logfood` and
looks up by profile key. The token is returned in both cases (because
`dualWrite` currently populates both keys), but the cache key used is
now the profile name.
github-merge-queue bot pushed a commit to databricks/databricks-sdk-py that referenced this pull request Feb 26, 2026
…CLIs (#1297)

## Why

Today `databricks auth token --profile <name>` fails to find a valid
token even when one exists — because the token was stored under the host
URL key (via `--host`), not the profile name key. The SDK's
`DatabricksCliTokenSource` always passed `--host` regardless of whether
a profile was configured, so the CLI never had a chance to do
profile-based lookup.

This is a port of
[databricks-sdk-go#1497](databricks/databricks-sdk-go#1497).

## Changes

### `CliTokenSource`
- Added `host_cmd` optional parameter: a fallback `--host` command for
CLIs too old to support `--profile`.
- Extracted `_exec_cli_command()` helper to run a single CLI invocation.
- Updated `refresh()` to try the primary command first; if it fails with
`"unknown flag: --profile"`, retry with `host_cmd` and emit a warning to
upgrade the CLI.

### `DatabricksCliTokenSource`
- Extracted `_build_host_args()` static method from the existing inline
`--host` argument construction.
- When `cfg.profile` is set: uses `--profile <name>` as the primary
command. If `cfg.host` is also present, builds a `--host` fallback via
`_build_host_args()` for compatibility with older CLIs.
- When `cfg.profile` is empty: unchanged — existing `--host` path is
used.

## Test plan

- [x] `--profile` set with host → primary cmd uses `--profile`, fallback
`--host` cmd is built
- [x] `--profile` set without host → primary cmd uses `--profile`, no
fallback
- [x] `--profile` not set → existing `--host` path unchanged
- [x] `refresh()`: `--profile` fails with "unknown flag: --profile" →
retries with `--host`, succeeds
- [x] `refresh()`: `--profile` fails with real auth error → no retry,
error propagated
- [x] `refresh()`: `--profile` fails, no `host_cmd` set → original error
raised
- [x] All 24 existing + new tests pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
github-merge-queue bot pushed a commit to databricks/databricks-sdk-java that referenced this pull request Feb 27, 2026
## Why

Today \`databricks auth token --profile <name>\` fails to find a valid
token even when one exists — because the token was stored under the host
URL key (via \`--host\`), not the profile name key. The SDK's
\`DatabricksCliCredentialsProvider\` always passed \`--host\` regardless
of whether a profile was configured, so the CLI never had a chance to do
profile-based lookup.

This is a port of
[databricks-sdk-py#1297](databricks/databricks-sdk-py#1297)
/
[databricks-sdk-go#1497](databricks/databricks-sdk-go#1497).

## Changes

### \`CliTokenSource\`
- Added \`fallbackCmd\` optional field: a fallback command for CLIs too
old to support \`--profile\`.
- Added inner \`CliCommandException extends IOException\`: its
\`getMessage()\` returns the clean stderr-based message (what users
see), while \`getFullOutput()\` exposes the combined stdout+stderr for
flag detection — avoiding verbose combined output in user-facing errors.
- Extracted \`execCliCommand()\` helper to run a single CLI invocation.
- Updated \`getToken()\` to try the primary command first; if it fails
with \`"unknown flag: --profile"\` (checked in full output across both
streams), retries with \`fallbackCmd\` and emits a warning to upgrade
the CLI.

### \`DatabricksCliCredentialsProvider\`
- Renamed \`buildCliCommand\` → \`buildHostArgs\` (reflects its role as
the \`--host\` legacy path).
- When \`cfg.profile\` is set: uses \`--profile <name>\` as the primary
command. If \`cfg.host\` is also present, builds a \`--host\` fallback
via \`buildHostArgs()\` for compatibility with older CLIs.
- When \`cfg.profile\` is empty: unchanged — existing \`--host\` path is
used.

## Test plan

- [x] \`--profile\` fails with \`"unknown flag: --profile"\` in stderr →
retries with \`--host\`, succeeds
- [x] \`--profile\` fails with \`"unknown flag: --profile"\` in stdout →
fallback still triggered
- [x] \`--profile\` fails with real auth error → no retry, error
propagated
- [x] \`fallbackCmd\` is \`null\` and \`--profile\` fails → original
error raised
- [x] All existing \`buildHostArgs\` tests (workspace, account, unified
host variants) pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants