Pass --force-refresh to CLI auth token command by mihaimitrea-db · Pull Request #752 · databricks/databricks-sdk-java

mihaimitrea-db · 2026-03-31T12:58:36Z

🥞 Stacked PR

Use this link to review incremental changes.

stack/cli-force-refresh [Files changed]
- stack/cli-attempt-chain [Files changed]

Summary

Append --force-refresh to the databricks auth token command when the installed CLI is >= v0.296.0, so the CLI bypasses its internal token cache and hands the SDK a freshly minted token every time.

Mirrors databricks/databricks-sdk-go#1628 and databricks/databricks-sdk-py#1378. Requires the version-detection infrastructure from the parent PR #751.

Why

The SDK already manages its own token caching via CachedTokenSource. When the SDK decides it needs a new token and shells out to databricks auth token, the CLI may return a token from its own cache that is about to expire (or that has already expired from the SDK's perspective). That produces unnecessary refresh failures and retry loops on top of a value that the SDK was confident was fresh.

The CLI added --force-refresh in databricks/cli#4767 (motivated by databricks/cli#4564) specifically to let callers bypass the CLI's cache. With the version-detection infrastructure from the parent PR already in place, opting in is a one-constant, one-branch change.

What changed

Interface changes

None. CliTokenSource is not part of the public API surface.

Behavioral changes

databricks auth token invocations now end with --force-refresh whenever the detected CLI is >= v0.296.0. Callers on older CLIs see no change.
On older CLIs, a WARNING is logged: "Databricks CLI <ver> does not support --force-refresh (requires >= v0.296.0). The CLI's token cache may provide stale tokens."

AzureCliCredentialsProvider is unaffected — it does not pass through DatabricksCliCredentialsProvider and does not opt into version-gated flag selection.

Internal changes

New constant DatabricksCliCredentialsProvider.CLI_VERSION_FOR_FORCE_REFRESH = new DatabricksCliVersion(0, 296, 0).
buildCliCommand is split into two helpers, matching the shape the Go and Python SDKs settled on after the same PR there:
- buildCoreCliCommand(cliPath, config, version) — holds the existing profile-vs-host decision (moved out of buildCliCommand).
- buildCliCommand(cliPath, config, version) — now a thin wrapper that calls buildCoreCliCommand, appends --force-refresh when version.atLeast(CLI_VERSION_FOR_FORCE_REFRESH), and otherwise logs the unsupported-version WARNING.

Future version-gated flags slot into the same pattern: add a CLI_VERSION_FOR_<flag> constant and an if version.atLeast(...) block in buildCliCommand.

How is this tested?

Additional testBuildCliCommand parameterized cases in DatabricksCliCredentialsProviderTest cover the full matrix:

--host + v0.296.0 → appends --force-refresh.
account --host + v0.296.0 → appends --force-refresh.
--profile + v0.296.0 → --profile + --force-refresh.
--profile + v0.207.1 → --profile only (too old for --force-refresh).
--host + v0.295.0 → --host only (too old for --force-refresh).
unknown version (detection failure) → --host only, no --force-refresh.
dev build → --host only, no --force-refresh.

All parent-PR tests continue to pass unchanged.

mihaimitrea-db · 2026-03-31T13:09:34Z

Range-diff: stack/cli-force-refresh (628f509 -> 09cdbc4)

NEXT_CHANGELOG.md

@@ -0,0 +1,10 @@
+diff --git a/NEXT_CHANGELOG.md b/NEXT_CHANGELOG.md
+--- a/NEXT_CHANGELOG.md
++++ b/NEXT_CHANGELOG.md
+ ### Documentation
+ 
+ ### Internal Changes
++* Generalize CLI token source into a progressive command list for forward-compatible flag support.
+ 
+ ### API Changes
+ * Add `createCatalog()`, `createSyncedTable()`, `deleteCatalog()`, `deleteSyncedTable()`, `getCatalog()` and `getSyncedTable()` methods for `workspaceClient.postgres()` service.
\ No newline at end of file

databricks-sdk-java/src/main/java/com/databricks/sdk/core/CliTokenSource.java

@@ -132,8 +132,7 @@
      this.env = env;
 -    this.fallbackCmd =
 -        fallbackCmd != null ? OSUtils.get(env).getCliExecutableCommand(fallbackCmd) : null;
--    this.forceCmd =
--        forceCmd != null ? OSUtils.get(env).getCliExecutableCommand(forceCmd) : null;
+-    this.forceCmd = forceCmd != null ? OSUtils.get(env).getCliExecutableCommand(forceCmd) : null;
 +  }
 +
 +  private static final String UNKNOWN_PROFILE_FLAG = "unknown flag: --profile";

databricks-sdk-java/src/main/java/com/databricks/sdk/core/DatabricksCliCredentialsProvider.java

@@ -21,8 +21,7 @@
 +    return result;
 +  }
 +
-+  List<CliTokenSource.CliCommand> buildAttempts(
-+      String cliPath, DatabricksConfig config) {
++  List<CliTokenSource.CliCommand> buildAttempts(String cliPath, DatabricksConfig config) {
 +    List<CliTokenSource.CliCommand> attempts = new ArrayList<>();
 +
 +    List<String> profileCmd;
@@ -54,9 +53,7 @@
 +          new CliTokenSource.CliCommand(
 +              buildHostArgs(cliPath, config), Collections.emptyList(), null));
 +    } else {
-+      attempts.add(
-+          new CliTokenSource.CliCommand(
-+              profileCmd, Collections.emptyList(), null));
++      attempts.add(new CliTokenSource.CliCommand(profileCmd, Collections.emptyList(), null));
 +    }
 +
 +    return attempts;
@@ -84,11 +81,7 @@
 -    return new CliTokenSource(
 -        profileCmd, "token_type", "access_token", "expiry", config.getEnv(), fallbackCmd, forceCmd);
 +    return CliTokenSource.fromAttempts(
-+        buildAttempts(cliPath, config),
-+        "token_type",
-+        "access_token",
-+        "expiry",
-+        config.getEnv());
++        buildAttempts(cliPath, config), "token_type", "access_token", "expiry", config.getEnv());
    }
  
    @Override
\ No newline at end of file

databricks-sdk-java/src/test/java/com/databricks/sdk/core/DatabricksCliCredentialsProviderTest.java

@@ -9,8 +9,7 @@
 +
 +  @Test
 +  void testBuildAttempts_WithProfileAndHost() {
-+    DatabricksConfig config =
-+        new DatabricksConfig().setHost(HOST).setProfile("my-profile");
++    DatabricksConfig config = new DatabricksConfig().setHost(HOST).setProfile("my-profile");
 +
 +    List<CliTokenSource.CliCommand> attempts = provider.buildAttempts(CLI_PATH, config);
 +
@@ -18,12 +17,9 @@
 +    assertEquals(
 +        Arrays.asList(CLI_PATH, "auth", "token", "--profile", "my-profile", "--force-refresh"),
 +        attempts.get(0).cmd);
-+    assertEquals(
-+        Arrays.asList(CLI_PATH, "auth", "token", "--profile", "my-profile"),
-+        attempts.get(1).cmd);
 +    assertEquals(
-+        Arrays.asList(CLI_PATH, "auth", "token", "--host", HOST),
-+        attempts.get(2).cmd);
++        Arrays.asList(CLI_PATH, "auth", "token", "--profile", "my-profile"), attempts.get(1).cmd);
++    assertEquals(Arrays.asList(CLI_PATH, "auth", "token", "--host", HOST), attempts.get(2).cmd);
 +  }
 +
 +  @Test
@@ -37,8 +33,7 @@
 +        Arrays.asList(CLI_PATH, "auth", "token", "--profile", "my-profile", "--force-refresh"),
 +        attempts.get(0).cmd);
 +    assertEquals(
-+        Arrays.asList(CLI_PATH, "auth", "token", "--profile", "my-profile"),
-+        attempts.get(1).cmd);
++        Arrays.asList(CLI_PATH, "auth", "token", "--profile", "my-profile"), attempts.get(1).cmd);
 +  }
 +
 +  @Test
@@ -51,23 +46,26 @@
 +    assertEquals(
 +        Arrays.asList(CLI_PATH, "auth", "token", "--host", HOST, "--force-refresh"),
 +        attempts.get(0).cmd);
-+    assertEquals(
-+        Arrays.asList(CLI_PATH, "auth", "token", "--host", HOST),
-+        attempts.get(1).cmd);
++    assertEquals(Arrays.asList(CLI_PATH, "auth", "token", "--host", HOST), attempts.get(1).cmd);
 +  }
 +
 +  @Test
 +  void testBuildAttempts_WithAccountHost() {
-+    DatabricksConfig config =
-+        new DatabricksConfig().setHost(ACCOUNT_HOST).setAccountId(ACCOUNT_ID);
++    DatabricksConfig config = new DatabricksConfig().setHost(ACCOUNT_HOST).setAccountId(ACCOUNT_ID);
 +
 +    List<CliTokenSource.CliCommand> attempts = provider.buildAttempts(CLI_PATH, config);
 +
 +    assertEquals(2, attempts.size());
 +    assertEquals(
 +        Arrays.asList(
-+            CLI_PATH, "auth", "token", "--host", ACCOUNT_HOST,
-+            "--account-id", ACCOUNT_ID, "--force-refresh"),
++            CLI_PATH,
++            "auth",
++            "token",
++            "--host",
++            ACCOUNT_HOST,
++            "--account-id",
++            ACCOUNT_ID,
++            "--force-refresh"),
 +        attempts.get(0).cmd);
 +    assertEquals(
 +        Arrays.asList(

_{Reproduce locally: git range-diff 048a903..628f509 5e8f476..09cdbc4 | Disable: git config gitstack.push-range-diff false}

mihaimitrea-db · 2026-03-31T13:22:24Z

Range-diff: stack/cli-force-refresh (09cdbc4 -> f88c52a)

databricks-sdk-java/src/main/java/com/databricks/sdk/core/CliTokenSource.java

@@ -26,9 +26,9 @@
 +   * command in the chain and an optional log message emitted on fallback.
 +   */
 +  static class CliCommand {
-+    private final List<String> cmd;
-+    private final List<String> fallbackTriggers;
-+    private final String fallbackMessage;
++    final List<String> cmd;
++    final List<String> fallbackTriggers;
++    final String fallbackMessage;
  
 -  private List<String> profileCmd;
 -  private String tokenTypeField;

_{Reproduce locally: git range-diff 5e8f476..09cdbc4 5e8f476..f88c52a | Disable: git config gitstack.push-range-diff false}

mihaimitrea-db · 2026-03-31T14:04:17Z

Range-diff: stack/cli-force-refresh (f88c52a -> 25a3779)

databricks-sdk-java/src/main/java/com/databricks/sdk/core/CliTokenSource.java

@@ -15,20 +15,19 @@
  public class CliTokenSource implements TokenSource {
    private static final Logger LOG = LoggerFactory.getLogger(CliTokenSource.class);
  
--  private static final String UNKNOWN_PROFILE_FLAG = "unknown flag: --profile";
--  private static final String UNKNOWN_FORCE_REFRESH_FLAG = "unknown flag: --force-refresh";
--
 -  // forceCmd is tried before profileCmd when non-null. If the CLI rejects
 -  // --force-refresh or --profile, execution falls through to profileCmd.
 -  private List<String> forceCmd;
 +  /**
-+   * Describes a CLI command with the error substrings that allow falling through to the next
-+   * command in the chain and an optional log message emitted on fallback.
++   * Describes a CLI command with an optional warning message emitted when falling through to the
++   * next command in the chain.
 +   */
 +  static class CliCommand {
 +    final List<String> cmd;
-+    final List<String> fallbackTriggers;
-+    final String fallbackMessage;
++
++    // Flags used by this command (e.g. "--force-refresh", "--profile"). Used to distinguish
++    // "unknown flag" errors (which trigger fallback) from real auth errors (which propagate).
++    final List<String> usedFlags;
  
 -  private List<String> profileCmd;
 -  private String tokenTypeField;
@@ -38,9 +37,11 @@
 -  // fallbackCmd is tried when profileCmd fails with "unknown flag: --profile",
 -  // indicating the CLI is too old to support --profile.
 -  private List<String> fallbackCmd;
-+    CliCommand(List<String> cmd, List<String> fallbackTriggers, String fallbackMessage) {
++    final String fallbackMessage;
++
++    CliCommand(List<String> cmd, List<String> usedFlags, String fallbackMessage) {
 +      this.cmd = cmd;
-+      this.fallbackTriggers = fallbackTriggers != null ? fallbackTriggers : Collections.emptyList();
++      this.usedFlags = usedFlags != null ? usedFlags : Collections.emptyList();
 +      this.fallbackMessage = fallbackMessage;
 +    }
 +  }
@@ -85,7 +86,7 @@
 +                a ->
 +                    new CliCommand(
 +                        OSUtils.get(env).getCliExecutableCommand(a.cmd),
-+                        a.fallbackTriggers,
++                        a.usedFlags,
 +                        a.fallbackMessage))
 +            .collect(Collectors.toList()),
 +        tokenTypeField,
@@ -108,7 +109,7 @@
 +                a ->
 +                    new CliCommand(
 +                        OSUtils.get(env).getCliExecutableCommand(a.cmd),
-+                        a.fallbackTriggers,
++                        a.usedFlags,
 +                        a.fallbackMessage))
 +            .collect(Collectors.toList()),
 +        tokenTypeField,
@@ -134,9 +135,6 @@
 -        fallbackCmd != null ? OSUtils.get(env).getCliExecutableCommand(fallbackCmd) : null;
 -    this.forceCmd = forceCmd != null ? OSUtils.get(env).getCliExecutableCommand(forceCmd) : null;
 +  }
-+
-+  private static final String UNKNOWN_PROFILE_FLAG = "unknown flag: --profile";
-+  private static final String UNKNOWN_FORCE_REFRESH_FLAG = "unknown flag: --force-refresh";
 +
 +  private static List<CliCommand> buildAttempts(
 +      List<String> forceCmd, List<String> profileCmd, List<String> fallbackCmd) {
@@ -146,7 +144,7 @@
 +      attempts.add(
 +          new CliCommand(
 +              forceCmd,
-+              Arrays.asList(UNKNOWN_FORCE_REFRESH_FLAG, UNKNOWN_PROFILE_FLAG),
++              Arrays.asList("--force-refresh", "--profile"),
 +              "Databricks CLI does not support --force-refresh flag. "
 +                  + "Falling back to regular token fetch. "
 +                  + "Please upgrade your CLI to the latest version."));
@@ -156,7 +154,7 @@
 +      attempts.add(
 +          new CliCommand(
 +              profileCmd,
-+              Collections.singletonList(UNKNOWN_PROFILE_FLAG),
++              Collections.singletonList("--profile"),
 +              "Databricks CLI does not support --profile flag. Falling back to --host. "
 +                  + "Please upgrade your CLI to the latest version."));
 +      attempts.add(new CliCommand(fallbackCmd, Collections.emptyList(), null));
@@ -168,6 +166,14 @@
    }
  
    /**
+         if (stderr.contains("not found")) {
+           throw new DatabricksException(stderr);
+         }
+-        // getMessage() returns the clean stderr-based message; getFullOutput() exposes
+-        // both streams so the caller can check for "unknown flag: --profile" in either.
+         throw new CliCommandException("cannot get access token: " + stderr, stdout + "\n" + stderr);
+       }
+       JsonNode jsonNode = new ObjectMapper().readTree(stdout);
      }
    }
  
@@ -179,7 +185,7 @@
    }
  
 -  private boolean isUnknownFlagError(String errorText, String flag) {
--    return errorText != null && errorText.contains(flag);
+-    return errorText != null && errorText.contains("unknown flag: " + flag);
 -  }
 -
 -  private Token execProfileCmdWithFallback() {
@@ -187,7 +193,7 @@
 -      return execCliCommand(this.profileCmd);
 -    } catch (IOException e) {
 -      String textToCheck = getErrorText(e);
--      if (fallbackCmd != null && isUnknownFlagError(textToCheck, UNKNOWN_PROFILE_FLAG)) {
+-      if (fallbackCmd != null && isUnknownFlagError(textToCheck, "--profile")) {
 -        LOG.warn(
 -            "Databricks CLI does not support --profile flag. Falling back to --host. "
 -                + "Please upgrade your CLI to the latest version.");
@@ -196,39 +202,46 @@
 -        } catch (IOException fallbackException) {
 -          throw new DatabricksException(fallbackException.getMessage(), fallbackException);
 -        }
--      }
--      throw new DatabricksException(e.getMessage(), e);
-+  private static boolean shouldFallback(CliCommand attempt, String errorText) {
++  private static boolean isUnknownFlagError(String errorText, List<String> flags) {
 +    if (errorText == null) {
 +      return false;
++    }
++    for (String flag : flags) {
++      if (errorText.contains("unknown flag: " + flag)) {
++        return true;
+       }
+-      throw new DatabricksException(e.getMessage(), e);
      }
-+    return attempt.fallbackTriggers.stream().anyMatch(errorText::contains);
++    return false;
    }
  
    @Override
    public Token getToken() {
 -    if (forceCmd == null) {
 -      return execProfileCmdWithFallback();
--    }
-+    IOException lastException = null;
++    if (attempts.isEmpty()) {
++      throw new DatabricksException("cannot get access token: no CLI commands configured");
+     }
  
 -    try {
 -      return execCliCommand(this.forceCmd);
 -    } catch (IOException e) {
 -      String textToCheck = getErrorText(e);
--      if (isUnknownFlagError(textToCheck, UNKNOWN_FORCE_REFRESH_FLAG)
--          || isUnknownFlagError(textToCheck, UNKNOWN_PROFILE_FLAG)) {
+-      if (isUnknownFlagError(textToCheck, "--force-refresh")
+-          || isUnknownFlagError(textToCheck, "--profile")) {
 -        LOG.warn(
 -            "Databricks CLI does not support --force-refresh flag. "
 -                + "Falling back to regular token fetch. "
 -                + "Please upgrade your CLI to the latest version.");
 -        return execProfileCmdWithFallback();
++    IOException lastException = null;
++
 +    for (int i = 0; i < attempts.size(); i++) {
 +      CliCommand attempt = attempts.get(i);
 +      try {
 +        return execCliCommand(attempt.cmd);
 +      } catch (IOException e) {
-+        if (i + 1 < attempts.size() && shouldFallback(attempt, getErrorText(e))) {
++        if (i + 1 < attempts.size() && isUnknownFlagError(getErrorText(e), attempt.usedFlags)) {
 +          if (attempt.fallbackMessage != null) {
 +            LOG.warn(attempt.fallbackMessage);
 +          }

databricks-sdk-java/src/main/java/com/databricks/sdk/core/DatabricksCliCredentialsProvider.java

@@ -1,17 +1,8 @@
 diff --git a/databricks-sdk-java/src/main/java/com/databricks/sdk/core/DatabricksCliCredentialsProvider.java b/databricks-sdk-java/src/main/java/com/databricks/sdk/core/DatabricksCliCredentialsProvider.java
 --- a/databricks-sdk-java/src/main/java/com/databricks/sdk/core/DatabricksCliCredentialsProvider.java
 +++ b/databricks-sdk-java/src/main/java/com/databricks/sdk/core/DatabricksCliCredentialsProvider.java
-     return cmd;
    }
  
-+  private static final String UNKNOWN_PROFILE_FLAG = "unknown flag: --profile";
-+  private static final String UNKNOWN_FORCE_REFRESH_FLAG = "unknown flag: --force-refresh";
-+
-   List<String> buildProfileArgs(String cliPath, DatabricksConfig config) {
-     return new ArrayList<>(
-         Arrays.asList(cliPath, "auth", "token", "--profile", config.getProfile()));
-   }
- 
    private static List<String> withForceRefresh(List<String> cmd) {
 -    List<String> forceCmd = new ArrayList<>(cmd);
 -    forceCmd.add("--force-refresh");
@@ -37,7 +28,7 @@
 +    attempts.add(
 +        new CliTokenSource.CliCommand(
 +            withForceRefresh(profileCmd),
-+            Arrays.asList(UNKNOWN_FORCE_REFRESH_FLAG, UNKNOWN_PROFILE_FLAG),
++            Arrays.asList("--force-refresh", "--profile"),
 +            "Databricks CLI does not support --force-refresh flag. "
 +                + "Falling back to regular token fetch. "
 +                + "Please upgrade your CLI to the latest version."));
@@ -46,7 +37,7 @@
 +      attempts.add(
 +          new CliTokenSource.CliCommand(
 +              profileCmd,
-+              Collections.singletonList(UNKNOWN_PROFILE_FLAG),
++              Collections.singletonList("--profile"),
 +              "Databricks CLI does not support --profile flag. Falling back to --host. "
 +                  + "Please upgrade your CLI to the latest version."));
 +      attempts.add(

_{Reproduce locally: git range-diff 5e8f476..f88c52a 6b8a57f..25a3779 | Disable: git config gitstack.push-range-diff false}

mihaimitrea-db · 2026-03-31T15:01:43Z

Range-diff: stack/cli-force-refresh (25a3779 -> 694521d)

databricks-sdk-java/src/main/java/com/databricks/sdk/core/CliTokenSource.java

@@ -1,10 +1,8 @@
 diff --git a/databricks-sdk-java/src/main/java/com/databricks/sdk/core/CliTokenSource.java b/databricks-sdk-java/src/main/java/com/databricks/sdk/core/CliTokenSource.java
 --- a/databricks-sdk-java/src/main/java/com/databricks/sdk/core/CliTokenSource.java
 +++ b/databricks-sdk-java/src/main/java/com/databricks/sdk/core/CliTokenSource.java
- import java.time.ZoneId;
  import java.time.format.DateTimeFormatter;
  import java.time.format.DateTimeParseException;
-+import java.util.ArrayList;
  import java.util.Arrays;
 +import java.util.Collections;
  import java.util.List;
@@ -61,47 +59,32 @@
    public CliTokenSource(
        List<String> cmd,
        String tokenTypeField,
-     this(cmd, tokenTypeField, accessTokenField, expiryField, env, null, null);
-   }
- 
-+  /** Constructs a two-attempt source with --profile to --host fallback. */
-   public CliTokenSource(
-       List<String> cmd,
-       String tokenTypeField,
-     this(cmd, tokenTypeField, accessTokenField, expiryField, env, fallbackCmd, null);
-   }
- 
-+  /** Constructs a source with optional force-refresh, profile, and host fallback chain. */
-   public CliTokenSource(
-       List<String> cmd,
-       String tokenTypeField,
-       Environment env,
-       List<String> fallbackCmd,
-       List<String> forceCmd) {
--    super();
--    this.profileCmd = OSUtils.get(env).getCliExecutableCommand(cmd);
+       String accessTokenField,
+       String expiryField,
+       Environment env) {
+-    this(cmd, tokenTypeField, accessTokenField, expiryField, env, null, null);
 +    this(
-+        buildAttempts(forceCmd, cmd, fallbackCmd).stream()
-+            .map(
-+                a ->
-+                    new CliCommand(
-+                        OSUtils.get(env).getCliExecutableCommand(a.cmd),
-+                        a.usedFlags,
-+                        a.fallbackMessage))
-+            .collect(Collectors.toList()),
++        Collections.singletonList(
++            new CliCommand(
++                OSUtils.get(env).getCliExecutableCommand(cmd), Collections.emptyList(), null)),
 +        tokenTypeField,
 +        accessTokenField,
 +        expiryField,
 +        env,
 +        true);
-+  }
-+
+   }
+ 
+-  public CliTokenSource(
+-      List<String> cmd,
 +  /** Creates a CliTokenSource from a pre-built attempt chain. */
 +  static CliTokenSource fromAttempts(
 +      List<CliCommand> attempts,
-+      String tokenTypeField,
-+      String accessTokenField,
-+      String expiryField,
+       String tokenTypeField,
+       String accessTokenField,
+       String expiryField,
+-      Environment env,
+-      List<String> fallbackCmd) {
+-    this(cmd, tokenTypeField, accessTokenField, expiryField, env, fallbackCmd, null);
 +      Environment env) {
 +    return new CliTokenSource(
 +        attempts.stream()
@@ -117,14 +100,20 @@
 +        expiryField,
 +        env,
 +        true);
-+  }
-+
+   }
+ 
+-  public CliTokenSource(
+-      List<String> cmd,
 +  private CliTokenSource(
 +      List<CliCommand> attempts,
-+      String tokenTypeField,
-+      String accessTokenField,
-+      String expiryField,
-+      Environment env,
+       String tokenTypeField,
+       String accessTokenField,
+       String expiryField,
+       Environment env,
+-      List<String> fallbackCmd,
+-      List<String> forceCmd) {
+-    super();
+-    this.profileCmd = OSUtils.get(env).getCliExecutableCommand(cmd);
 +      boolean alreadyResolved) {
 +    this.attempts = attempts;
      this.tokenTypeField = tokenTypeField;
@@ -134,35 +123,6 @@
 -    this.fallbackCmd =
 -        fallbackCmd != null ? OSUtils.get(env).getCliExecutableCommand(fallbackCmd) : null;
 -    this.forceCmd = forceCmd != null ? OSUtils.get(env).getCliExecutableCommand(forceCmd) : null;
-+  }
-+
-+  private static List<CliCommand> buildAttempts(
-+      List<String> forceCmd, List<String> profileCmd, List<String> fallbackCmd) {
-+    List<CliCommand> attempts = new ArrayList<>();
-+
-+    if (forceCmd != null) {
-+      attempts.add(
-+          new CliCommand(
-+              forceCmd,
-+              Arrays.asList("--force-refresh", "--profile"),
-+              "Databricks CLI does not support --force-refresh flag. "
-+                  + "Falling back to regular token fetch. "
-+                  + "Please upgrade your CLI to the latest version."));
-+    }
-+
-+    if (fallbackCmd != null) {
-+      attempts.add(
-+          new CliCommand(
-+              profileCmd,
-+              Collections.singletonList("--profile"),
-+              "Databricks CLI does not support --profile flag. Falling back to --host. "
-+                  + "Please upgrade your CLI to the latest version."));
-+      attempts.add(new CliCommand(fallbackCmd, Collections.emptyList(), null));
-+    } else {
-+      attempts.add(new CliCommand(profileCmd, Collections.emptyList(), null));
-+    }
-+
-+    return attempts;
    }
  
    /**

databricks-sdk-java/src/main/java/com/databricks/sdk/core/DatabricksCliCredentialsProvider.java

@@ -15,36 +15,36 @@
 +  List<CliTokenSource.CliCommand> buildAttempts(String cliPath, DatabricksConfig config) {
 +    List<CliTokenSource.CliCommand> attempts = new ArrayList<>();
 +
-+    List<String> profileCmd;
-+    boolean hasHostFallback = false;
++    boolean hasProfile = config.getProfile() != null;
 +
-+    if (config.getProfile() != null) {
-+      profileCmd = buildProfileArgs(cliPath, config);
-+      hasHostFallback = config.getHost() != null;
-+    } else {
-+      profileCmd = buildHostArgs(cliPath, config);
-+    }
++    if (hasProfile) {
++      List<String> profileCmd = buildProfileArgs(cliPath, config);
 +
-+    attempts.add(
-+        new CliTokenSource.CliCommand(
-+            withForceRefresh(profileCmd),
-+            Arrays.asList("--force-refresh", "--profile"),
-+            "Databricks CLI does not support --force-refresh flag. "
-+                + "Falling back to regular token fetch. "
-+                + "Please upgrade your CLI to the latest version."));
-+
-+    if (hasHostFallback) {
 +      attempts.add(
 +          new CliTokenSource.CliCommand(
-+              profileCmd,
-+              Collections.singletonList("--profile"),
-+              "Databricks CLI does not support --profile flag. Falling back to --host. "
++              withForceRefresh(profileCmd),
++              Arrays.asList("--force-refresh", "--profile"),
++              "Databricks CLI does not support --force-refresh flag. "
++                  + "Falling back to regular token fetch. "
 +                  + "Please upgrade your CLI to the latest version."));
++
++      if (config.getHost() != null) {
++        attempts.add(
++            new CliTokenSource.CliCommand(
++                profileCmd,
++                Collections.singletonList("--profile"),
++                "Databricks CLI does not support --profile flag. Falling back to --host. "
++                    + "Please upgrade your CLI to the latest version."));
++        attempts.add(
++            new CliTokenSource.CliCommand(
++                buildHostArgs(cliPath, config), Collections.emptyList(), null));
++      } else {
++        attempts.add(new CliTokenSource.CliCommand(profileCmd, Collections.emptyList(), null));
++      }
++    } else {
 +      attempts.add(
 +          new CliTokenSource.CliCommand(
 +              buildHostArgs(cliPath, config), Collections.emptyList(), null));
-+    } else {
-+      attempts.add(new CliTokenSource.CliCommand(profileCmd, Collections.emptyList(), null));
 +    }
 +
 +    return attempts;

databricks-sdk-java/src/test/java/com/databricks/sdk/core/CliTokenSourceTest.java

@@ -0,0 +1,40 @@
+diff --git a/databricks-sdk-java/src/test/java/com/databricks/sdk/core/CliTokenSourceTest.java b/databricks-sdk-java/src/test/java/com/databricks/sdk/core/CliTokenSourceTest.java
+--- a/databricks-sdk-java/src/test/java/com/databricks/sdk/core/CliTokenSourceTest.java
++++ b/databricks-sdk-java/src/test/java/com/databricks/sdk/core/CliTokenSourceTest.java
+ import java.time.format.DateTimeParseException;
+ import java.util.ArrayList;
+ import java.util.Arrays;
++import java.util.Collections;
+ import java.util.HashMap;
+ import java.util.List;
+ import java.util.Map;
+ 
+   private CliTokenSource makeTokenSource(
+       Environment env, List<String> primaryCmd, List<String> fallbackCmd, List<String> forceCmd) {
++    List<CliTokenSource.CliCommand> attempts = new ArrayList<>();
++
++    if (forceCmd != null) {
++      attempts.add(
++          new CliTokenSource.CliCommand(
++              forceCmd, Arrays.asList("--force-refresh", "--profile"), "force-refresh fallback"));
++    }
++
++    if (fallbackCmd != null) {
++      attempts.add(
++          new CliTokenSource.CliCommand(
++              primaryCmd, Collections.singletonList("--profile"), "profile fallback"));
++      attempts.add(new CliTokenSource.CliCommand(fallbackCmd, Collections.emptyList(), null));
++    } else {
++      attempts.add(new CliTokenSource.CliCommand(primaryCmd, Collections.emptyList(), null));
++    }
++
+     OSUtilities osUtils = mock(OSUtilities.class);
+     when(osUtils.getCliExecutableCommand(any())).thenAnswer(inv -> inv.getArgument(0));
+     try (MockedStatic<OSUtils> mockedOSUtils = mockStatic(OSUtils.class)) {
+       mockedOSUtils.when(() -> OSUtils.get(any())).thenReturn(osUtils);
+-      return new CliTokenSource(
+-          primaryCmd, "token_type", "access_token", "expiry", env, fallbackCmd, forceCmd);
++      return CliTokenSource.fromAttempts(attempts, "token_type", "access_token", "expiry", env);
+     }
+   }
+ 
\ No newline at end of file

databricks-sdk-java/src/test/java/com/databricks/sdk/core/DatabricksCliCredentialsProviderTest.java

@@ -1,6 +1,27 @@
 diff --git a/databricks-sdk-java/src/test/java/com/databricks/sdk/core/DatabricksCliCredentialsProviderTest.java b/databricks-sdk-java/src/test/java/com/databricks/sdk/core/DatabricksCliCredentialsProviderTest.java
 --- a/databricks-sdk-java/src/test/java/com/databricks/sdk/core/DatabricksCliCredentialsProviderTest.java
 +++ b/databricks-sdk-java/src/test/java/com/databricks/sdk/core/DatabricksCliCredentialsProviderTest.java
+ import static org.junit.jupiter.api.Assertions.*;
+ 
+ import java.util.Arrays;
++import java.util.Collections;
+ import java.util.List;
+ import org.junit.jupiter.api.Test;
+ 
+   private static final String ACCOUNT_ID = "test-account-123";
+   private static final String WORKSPACE_ID = "987654321";
+ 
++  private static final String FORCE_REFRESH_FALLBACK_MSG =
++      "Databricks CLI does not support --force-refresh flag. "
++          + "Falling back to regular token fetch. "
++          + "Please upgrade your CLI to the latest version.";
++  private static final String PROFILE_FALLBACK_MSG =
++      "Databricks CLI does not support --profile flag. Falling back to --host. "
++          + "Please upgrade your CLI to the latest version.";
++
+   private final DatabricksCliCredentialsProvider provider = new DatabricksCliCredentialsProvider();
+ 
+   @Test
  
      assertEquals(Arrays.asList(CLI_PATH, "auth", "token", "--profile", "my-profile"), cmd);
    }
@@ -14,12 +35,21 @@
 +    List<CliTokenSource.CliCommand> attempts = provider.buildAttempts(CLI_PATH, config);
 +
 +    assertEquals(3, attempts.size());
++
 +    assertEquals(
 +        Arrays.asList(CLI_PATH, "auth", "token", "--profile", "my-profile", "--force-refresh"),
 +        attempts.get(0).cmd);
++    assertEquals(Arrays.asList("--force-refresh", "--profile"), attempts.get(0).usedFlags);
++    assertEquals(FORCE_REFRESH_FALLBACK_MSG, attempts.get(0).fallbackMessage);
++
 +    assertEquals(
 +        Arrays.asList(CLI_PATH, "auth", "token", "--profile", "my-profile"), attempts.get(1).cmd);
++    assertEquals(Collections.singletonList("--profile"), attempts.get(1).usedFlags);
++    assertEquals(PROFILE_FALLBACK_MSG, attempts.get(1).fallbackMessage);
++
 +    assertEquals(Arrays.asList(CLI_PATH, "auth", "token", "--host", HOST), attempts.get(2).cmd);
++    assertEquals(Collections.emptyList(), attempts.get(2).usedFlags);
++    assertNull(attempts.get(2).fallbackMessage);
 +  }
 +
 +  @Test
@@ -29,11 +59,17 @@
 +    List<CliTokenSource.CliCommand> attempts = provider.buildAttempts(CLI_PATH, config);
 +
 +    assertEquals(2, attempts.size());
++
 +    assertEquals(
 +        Arrays.asList(CLI_PATH, "auth", "token", "--profile", "my-profile", "--force-refresh"),
 +        attempts.get(0).cmd);
++    assertEquals(Arrays.asList("--force-refresh", "--profile"), attempts.get(0).usedFlags);
++    assertEquals(FORCE_REFRESH_FALLBACK_MSG, attempts.get(0).fallbackMessage);
++
 +    assertEquals(
 +        Arrays.asList(CLI_PATH, "auth", "token", "--profile", "my-profile"), attempts.get(1).cmd);
++    assertEquals(Collections.emptyList(), attempts.get(1).usedFlags);
++    assertNull(attempts.get(1).fallbackMessage);
 +  }
 +
 +  @Test
@@ -42,11 +78,11 @@
 +
 +    List<CliTokenSource.CliCommand> attempts = provider.buildAttempts(CLI_PATH, config);
 +
-+    assertEquals(2, attempts.size());
-+    assertEquals(
-+        Arrays.asList(CLI_PATH, "auth", "token", "--host", HOST, "--force-refresh"),
-+        attempts.get(0).cmd);
-+    assertEquals(Arrays.asList(CLI_PATH, "auth", "token", "--host", HOST), attempts.get(1).cmd);
++    assertEquals(1, attempts.size());
++
++    assertEquals(Arrays.asList(CLI_PATH, "auth", "token", "--host", HOST), attempts.get(0).cmd);
++    assertEquals(Collections.emptyList(), attempts.get(0).usedFlags);
++    assertNull(attempts.get(0).fallbackMessage);
 +  }
 +
 +  @Test
@@ -55,21 +91,13 @@
 +
 +    List<CliTokenSource.CliCommand> attempts = provider.buildAttempts(CLI_PATH, config);
 +
-+    assertEquals(2, attempts.size());
++    assertEquals(1, attempts.size());
++
 +    assertEquals(
 +        Arrays.asList(
-+            CLI_PATH,
-+            "auth",
-+            "token",
-+            "--host",
-+            ACCOUNT_HOST,
-+            "--account-id",
-+            ACCOUNT_ID,
-+            "--force-refresh"),
++            CLI_PATH, "auth", "token", "--host", ACCOUNT_HOST, "--account-id", ACCOUNT_ID),
 +        attempts.get(0).cmd);
-+    assertEquals(
-+        Arrays.asList(
-+            CLI_PATH, "auth", "token", "--host", ACCOUNT_HOST, "--account-id", ACCOUNT_ID),
-+        attempts.get(1).cmd);
++    assertEquals(Collections.emptyList(), attempts.get(0).usedFlags);
++    assertNull(attempts.get(0).fallbackMessage);
 +  }
  }
\ No newline at end of file

_{Reproduce locally: git range-diff 6b8a57f..25a3779 6b8a57f..694521d | Disable: git config gitstack.push-range-diff false}

mihaimitrea-db · 2026-03-31T16:03:22Z

Range-diff: stack/cli-force-refresh (694521d -> 0f6baf6)

NEXT_CHANGELOG.md

@@ -4,7 +4,7 @@
  ### Documentation
  
  ### Internal Changes
-+* Generalize CLI token source into a progressive command list for forward-compatible flag support.
++* Generalized CLI token source into a progressive command attempt list, replacing the fixed three-field approach with an extensible chain.
  
  ### API Changes
  * Add `createCatalog()`, `createSyncedTable()`, `deleteCatalog()`, `deleteSyncedTable()`, `getCatalog()` and `getSyncedTable()` methods for `workspaceClient.postgres()` service.
\ No newline at end of file

databricks-sdk-java/src/main/java/com/databricks/sdk/core/CliTokenSource.java

@@ -13,9 +13,13 @@
  public class CliTokenSource implements TokenSource {
    private static final Logger LOG = LoggerFactory.getLogger(CliTokenSource.class);
  
--  // forceCmd is tried before profileCmd when non-null. If the CLI rejects
--  // --force-refresh or --profile, execution falls through to profileCmd.
--  private List<String> forceCmd;
+-  private List<String> cmd;
+-  private List<String> fallbackCmd;
+-  private List<String> secondFallbackCmd;
+-  private String tokenTypeField;
+-  private String accessTokenField;
+-  private String expiryField;
+-  private Environment env;
 +  /**
 +   * Describes a CLI command with an optional warning message emitted when falling through to the
 +   * next command in the chain.
@@ -26,15 +30,7 @@
 +    // Flags used by this command (e.g. "--force-refresh", "--profile"). Used to distinguish
 +    // "unknown flag" errors (which trigger fallback) from real auth errors (which propagate).
 +    final List<String> usedFlags;
- 
--  private List<String> profileCmd;
--  private String tokenTypeField;
--  private String accessTokenField;
--  private String expiryField;
--  private Environment env;
--  // fallbackCmd is tried when profileCmd fails with "unknown flag: --profile",
--  // indicating the CLI is too old to support --profile.
--  private List<String> fallbackCmd;
++
 +    final String fallbackMessage;
 +
 +    CliCommand(List<String> cmd, List<String> usedFlags, String fallbackMessage) {
@@ -63,66 +59,60 @@
        String expiryField,
        Environment env) {
 -    this(cmd, tokenTypeField, accessTokenField, expiryField, env, null, null);
-+    this(
-+        Collections.singletonList(
-+            new CliCommand(
-+                OSUtils.get(env).getCliExecutableCommand(cmd), Collections.emptyList(), null)),
-+        tokenTypeField,
-+        accessTokenField,
-+        expiryField,
-+        env,
-+        true);
++    this(cmd, null, tokenTypeField, accessTokenField, expiryField, env);
    }
  
 -  public CliTokenSource(
--      List<String> cmd,
 +  /** Creates a CliTokenSource from a pre-built attempt chain. */
 +  static CliTokenSource fromAttempts(
 +      List<CliCommand> attempts,
-       String tokenTypeField,
-       String accessTokenField,
-       String expiryField,
--      Environment env,
--      List<String> fallbackCmd) {
--    this(cmd, tokenTypeField, accessTokenField, expiryField, env, fallbackCmd, null);
++      String tokenTypeField,
++      String accessTokenField,
++      String expiryField,
 +      Environment env) {
-+    return new CliTokenSource(
-+        attempts.stream()
-+            .map(
-+                a ->
-+                    new CliCommand(
-+                        OSUtils.get(env).getCliExecutableCommand(a.cmd),
-+                        a.usedFlags,
-+                        a.fallbackMessage))
-+            .collect(Collectors.toList()),
-+        tokenTypeField,
-+        accessTokenField,
-+        expiryField,
-+        env,
-+        true);
-   }
- 
--  public CliTokenSource(
--      List<String> cmd,
++    return new CliTokenSource(null, attempts, tokenTypeField, accessTokenField, expiryField, env);
++  }
++
 +  private CliTokenSource(
+       List<String> cmd,
 +      List<CliCommand> attempts,
        String tokenTypeField,
        String accessTokenField,
        String expiryField,
-       Environment env,
+-      Environment env,
 -      List<String> fallbackCmd,
--      List<String> forceCmd) {
--    super();
--    this.profileCmd = OSUtils.get(env).getCliExecutableCommand(cmd);
-+      boolean alreadyResolved) {
-+    this.attempts = attempts;
+-      List<String> secondFallbackCmd) {
+-    this.cmd = OSUtils.get(env).getCliExecutableCommand(cmd);
++      Environment env) {
++    if (attempts != null) {
++      this.attempts =
++          attempts.stream()
++              .map(
++                  a ->
++                      new CliCommand(
++                          OSUtils.get(env).getCliExecutableCommand(a.cmd),
++                          a.usedFlags,
++                          a.fallbackMessage))
++              .collect(Collectors.toList());
++    } else {
++      this.attempts =
++          Collections.singletonList(
++              new CliCommand(
++                  OSUtils.get(env).getCliExecutableCommand(cmd), Collections.emptyList(), null));
++    }
++    if (this.attempts.isEmpty()) {
++      throw new DatabricksException("cannot get access token: no CLI commands configured");
++    }
      this.tokenTypeField = tokenTypeField;
      this.accessTokenField = accessTokenField;
      this.expiryField = expiryField;
      this.env = env;
 -    this.fallbackCmd =
 -        fallbackCmd != null ? OSUtils.get(env).getCliExecutableCommand(fallbackCmd) : null;
--    this.forceCmd = forceCmd != null ? OSUtils.get(env).getCliExecutableCommand(forceCmd) : null;
+-    this.secondFallbackCmd =
+-        secondFallbackCmd != null
+-            ? OSUtils.get(env).getCliExecutableCommand(secondFallbackCmd)
+-            : null;
    }
  
    /**
@@ -144,24 +134,8 @@
          : e.getMessage();
    }
  
--  private boolean isUnknownFlagError(String errorText, String flag) {
--    return errorText != null && errorText.contains("unknown flag: " + flag);
--  }
--
--  private Token execProfileCmdWithFallback() {
--    try {
--      return execCliCommand(this.profileCmd);
--    } catch (IOException e) {
--      String textToCheck = getErrorText(e);
--      if (fallbackCmd != null && isUnknownFlagError(textToCheck, "--profile")) {
--        LOG.warn(
--            "Databricks CLI does not support --profile flag. Falling back to --host. "
--                + "Please upgrade your CLI to the latest version.");
--        try {
--          return execCliCommand(this.fallbackCmd);
--        } catch (IOException fallbackException) {
--          throw new DatabricksException(fallbackException.getMessage(), fallbackException);
--        }
+-  private boolean isUnknownFlagError(String errorText) {
+-    return errorText != null && errorText.contains("unknown flag:");
 +  private static boolean isUnknownFlagError(String errorText, List<String> flags) {
 +    if (errorText == null) {
 +      return false;
@@ -169,33 +143,36 @@
 +    for (String flag : flags) {
 +      if (errorText.contains("unknown flag: " + flag)) {
 +        return true;
-       }
--      throw new DatabricksException(e.getMessage(), e);
-     }
++      }
++    }
 +    return false;
    }
  
    @Override
    public Token getToken() {
--    if (forceCmd == null) {
--      return execProfileCmdWithFallback();
-+    if (attempts.isEmpty()) {
-+      throw new DatabricksException("cannot get access token: no CLI commands configured");
-     }
+-    try {
+-      return execCliCommand(this.cmd);
+-    } catch (IOException e) {
+-      if (fallbackCmd != null && isUnknownFlagError(getErrorText(e))) {
+-        LOG.warn(
+-            "CLI does not support some flags used by this SDK. "
+-                + "Falling back to a compatible command. "
+-                + "Please upgrade your CLI to the latest version.");
+-      } else {
+-        throw new DatabricksException(e.getMessage(), e);
+-      }
+-    }
++    IOException lastException = null;
  
 -    try {
--      return execCliCommand(this.forceCmd);
+-      return execCliCommand(this.fallbackCmd);
 -    } catch (IOException e) {
--      String textToCheck = getErrorText(e);
--      if (isUnknownFlagError(textToCheck, "--force-refresh")
--          || isUnknownFlagError(textToCheck, "--profile")) {
+-      if (secondFallbackCmd != null && isUnknownFlagError(getErrorText(e))) {
 -        LOG.warn(
--            "Databricks CLI does not support --force-refresh flag. "
--                + "Falling back to regular token fetch. "
+-            "CLI does not support some flags used by this SDK. "
+-                + "Falling back to a compatible command. "
 -                + "Please upgrade your CLI to the latest version.");
--        return execProfileCmdWithFallback();
-+    IOException lastException = null;
-+
+-      } else {
 +    for (int i = 0; i < attempts.size(); i++) {
 +      CliCommand attempt = attempts.get(i);
 +      try {
@@ -208,11 +185,15 @@
 +          lastException = e;
 +          continue;
 +        }
-+        throw new DatabricksException(e.getMessage(), e);
+         throw new DatabricksException(e.getMessage(), e);
        }
+     }
+ 
+-    try {
+-      return execCliCommand(this.secondFallbackCmd);
+-    } catch (IOException e) {
 -      throw new DatabricksException(e.getMessage(), e);
-     }
-+
+-    }
 +    throw new DatabricksException(lastException.getMessage(), lastException);
    }
  }
\ No newline at end of file

databricks-sdk-java/src/main/java/com/databricks/sdk/core/DatabricksCliCredentialsProvider.java

@@ -1,21 +1,14 @@
 diff --git a/databricks-sdk-java/src/main/java/com/databricks/sdk/core/DatabricksCliCredentialsProvider.java b/databricks-sdk-java/src/main/java/com/databricks/sdk/core/DatabricksCliCredentialsProvider.java
 --- a/databricks-sdk-java/src/main/java/com/databricks/sdk/core/DatabricksCliCredentialsProvider.java
 +++ b/databricks-sdk-java/src/main/java/com/databricks/sdk/core/DatabricksCliCredentialsProvider.java
+     return forceCmd;
    }
  
-   private static List<String> withForceRefresh(List<String> cmd) {
--    List<String> forceCmd = new ArrayList<>(cmd);
--    forceCmd.add("--force-refresh");
--    return forceCmd;
-+    List<String> result = new ArrayList<>(cmd);
-+    result.add("--force-refresh");
-+    return result;
-+  }
-+
 +  List<CliTokenSource.CliCommand> buildAttempts(String cliPath, DatabricksConfig config) {
 +    List<CliTokenSource.CliCommand> attempts = new ArrayList<>();
 +
 +    boolean hasProfile = config.getProfile() != null;
++    boolean hasHost = config.getHost() != null;
 +
 +    if (hasProfile) {
 +      List<String> profileCmd = buildProfileArgs(cliPath, config);
@@ -28,49 +21,52 @@
 +                  + "Falling back to regular token fetch. "
 +                  + "Please upgrade your CLI to the latest version."));
 +
-+      if (config.getHost() != null) {
-+        attempts.add(
-+            new CliTokenSource.CliCommand(
-+                profileCmd,
-+                Collections.singletonList("--profile"),
-+                "Databricks CLI does not support --profile flag. Falling back to --host. "
-+                    + "Please upgrade your CLI to the latest version."));
-+        attempts.add(
-+            new CliTokenSource.CliCommand(
-+                buildHostArgs(cliPath, config), Collections.emptyList(), null));
-+      } else {
-+        attempts.add(new CliTokenSource.CliCommand(profileCmd, Collections.emptyList(), null));
-+      }
-+    } else {
 +      attempts.add(
 +          new CliTokenSource.CliCommand(
++              profileCmd,
++              Collections.singletonList("--profile"),
++              "Databricks CLI does not support --profile flag. Falling back to --host. "
++                  + "Please upgrade your CLI to the latest version."));
++    }
++
++    if (hasHost) {
++      attempts.add(
++          new CliTokenSource.CliCommand(
 +              buildHostArgs(cliPath, config), Collections.emptyList(), null));
 +    }
 +
 +    return attempts;
-   }
- 
++  }
++
    private CliTokenSource getDatabricksCliTokenSource(DatabricksConfig config) {
+     String cliPath = config.getDatabricksCliPath();
+     if (cliPath == null) {
        return null;
      }
  
--    List<String> profileCmd;
+-    List<String> cmd;
 -    List<String> fallbackCmd = null;
--    List<String> forceCmd;
+-    List<String> secondFallbackCmd = null;
 -
 -    if (config.getProfile() != null) {
--      profileCmd = buildProfileArgs(cliPath, config);
--      forceCmd = withForceRefresh(profileCmd);
+-      List<String> profileArgs = buildProfileArgs(cliPath, config);
+-      cmd = withForceRefresh(profileArgs);
+-      fallbackCmd = profileArgs;
 -      if (config.getHost() != null) {
--        fallbackCmd = buildHostArgs(cliPath, config);
+-        secondFallbackCmd = buildHostArgs(cliPath, config);
 -      }
 -    } else {
--      profileCmd = buildHostArgs(cliPath, config);
--      forceCmd = withForceRefresh(profileCmd);
+-      cmd = buildHostArgs(cliPath, config);
 -    }
 -
 -    return new CliTokenSource(
--        profileCmd, "token_type", "access_token", "expiry", config.getEnv(), fallbackCmd, forceCmd);
+-        cmd,
+-        "token_type",
+-        "access_token",
+-        "expiry",
+-        config.getEnv(),
+-        fallbackCmd,
+-        secondFallbackCmd);
 +    return CliTokenSource.fromAttempts(
 +        buildAttempts(cliPath, config), "token_type", "access_token", "expiry", config.getEnv());
    }

databricks-sdk-java/src/test/java/com/databricks/sdk/core/CliTokenSourceTest.java

@@ -10,22 +10,29 @@
  import java.util.Map;
  
    private CliTokenSource makeTokenSource(
-       Environment env, List<String> primaryCmd, List<String> fallbackCmd, List<String> forceCmd) {
+       Environment env, List<String> cmd, List<String> fallbackCmd, List<String> secondFallbackCmd) {
 +    List<CliTokenSource.CliCommand> attempts = new ArrayList<>();
 +
-+    if (forceCmd != null) {
-+      attempts.add(
-+          new CliTokenSource.CliCommand(
-+              forceCmd, Arrays.asList("--force-refresh", "--profile"), "force-refresh fallback"));
-+    }
++    attempts.add(
++        new CliTokenSource.CliCommand(
++            cmd,
++            fallbackCmd != null
++                ? Arrays.asList("--force-refresh", "--profile")
++                : Collections.emptyList(),
++            fallbackCmd != null ? "fallback" : null));
 +
 +    if (fallbackCmd != null) {
 +      attempts.add(
 +          new CliTokenSource.CliCommand(
-+              primaryCmd, Collections.singletonList("--profile"), "profile fallback"));
-+      attempts.add(new CliTokenSource.CliCommand(fallbackCmd, Collections.emptyList(), null));
-+    } else {
-+      attempts.add(new CliTokenSource.CliCommand(primaryCmd, Collections.emptyList(), null));
++              fallbackCmd,
++              secondFallbackCmd != null
++                  ? Collections.singletonList("--profile")
++                  : Collections.emptyList(),
++              secondFallbackCmd != null ? "second fallback" : null));
++    }
++
++    if (secondFallbackCmd != null) {
++      attempts.add(new CliTokenSource.CliCommand(secondFallbackCmd, Collections.emptyList(), null));
 +    }
 +
      OSUtilities osUtils = mock(OSUtilities.class);
@@ -33,7 +40,7 @@
      try (MockedStatic<OSUtils> mockedOSUtils = mockStatic(OSUtils.class)) {
        mockedOSUtils.when(() -> OSUtils.get(any())).thenReturn(osUtils);
 -      return new CliTokenSource(
--          primaryCmd, "token_type", "access_token", "expiry", env, fallbackCmd, forceCmd);
+-          cmd, "token_type", "access_token", "expiry", env, fallbackCmd, secondFallbackCmd);
 +      return CliTokenSource.fromAttempts(attempts, "token_type", "access_token", "expiry", env);
      }
    }

databricks-sdk-java/src/test/java/com/databricks/sdk/core/DatabricksCliCredentialsProviderTest.java

@@ -68,8 +68,8 @@
 +
 +    assertEquals(
 +        Arrays.asList(CLI_PATH, "auth", "token", "--profile", "my-profile"), attempts.get(1).cmd);
-+    assertEquals(Collections.emptyList(), attempts.get(1).usedFlags);
-+    assertNull(attempts.get(1).fallbackMessage);
++    assertEquals(Collections.singletonList("--profile"), attempts.get(1).usedFlags);
++    assertEquals(PROFILE_FALLBACK_MSG, attempts.get(1).fallbackMessage);
 +  }
 +
 +  @Test

_{Reproduce locally: git range-diff 6b8a57f..694521d 61686da..0f6baf6 | Disable: git config gitstack.push-range-diff false}

mihaimitrea-db · 2026-04-01T08:26:36Z

Range-diff: stack/cli-force-refresh (0ef99dc -> 0dd48d1)

databricks-sdk-java/src/main/java/com/databricks/sdk/core/CliTokenSource.java

@@ -6,6 +6,7 @@
  import java.util.Arrays;
 +import java.util.Collections;
  import java.util.List;
++import java.util.concurrent.atomic.AtomicInteger;
 +import java.util.stream.Collectors;
  import org.apache.commons.io.IOUtils;
  import org.slf4j.Logger;
@@ -45,7 +46,15 @@
      }
    }
  
-+  private final List<CliCommand> attempts;
++  private final List<CliCommand> commands;
++
++  // Index of the CLI command known to work, or -1 if not yet resolved. Once
++  // resolved it never changes — older CLIs don't gain new flags. We use
++  // AtomicInteger instead of synchronization because probing must be retryable
++  // on transient errors: concurrent callers may redundantly probe, but all
++  // converge to the same index.
++  private final AtomicInteger activeCommandIndex = new AtomicInteger(-1);
++
 +  private final String tokenTypeField;
 +  private final String accessTokenField;
 +  private final String expiryField;
@@ -63,19 +72,19 @@
    }
  
 -  public CliTokenSource(
-+  /** Creates a CliTokenSource from a pre-built attempt chain. */
-+  static CliTokenSource fromAttempts(
-+      List<CliCommand> attempts,
++  /** Creates a CliTokenSource from a pre-built command chain. */
++  static CliTokenSource fromCommands(
++      List<CliCommand> commands,
 +      String tokenTypeField,
 +      String accessTokenField,
 +      String expiryField,
 +      Environment env) {
-+    return new CliTokenSource(null, attempts, tokenTypeField, accessTokenField, expiryField, env);
++    return new CliTokenSource(null, commands, tokenTypeField, accessTokenField, expiryField, env);
 +  }
 +
 +  private CliTokenSource(
        List<String> cmd,
-+      List<CliCommand> attempts,
++      List<CliCommand> commands,
        String tokenTypeField,
        String accessTokenField,
        String expiryField,
@@ -84,9 +93,9 @@
 -      List<String> secondFallbackCmd) {
 -    this.cmd = OSUtils.get(env).getCliExecutableCommand(cmd);
 +      Environment env) {
-+    if (attempts != null) {
-+      this.attempts =
-+          attempts.stream()
++    if (commands != null && !commands.isEmpty()) {
++      this.commands =
++          commands.stream()
 +              .map(
 +                  a ->
 +                      new CliCommand(
@@ -94,13 +103,15 @@
 +                          a.usedFlags,
 +                          a.fallbackMessage))
 +              .collect(Collectors.toList());
-+    } else {
-+      this.attempts =
++    } else if (cmd != null) {
++      if (commands != null && commands.isEmpty()) {
++        LOG.warn("No CLI commands configured. Falling back to the default command.");
++      }
++      this.commands =
 +          Collections.singletonList(
 +              new CliCommand(
 +                  OSUtils.get(env).getCliExecutableCommand(cmd), Collections.emptyList(), null));
-+    }
-+    if (this.attempts.isEmpty()) {
++    } else {
 +      throw new DatabricksException("cannot get access token: no CLI commands configured");
 +    }
      this.tokenTypeField = tokenTypeField;
@@ -121,6 +132,9 @@
          }
 -        // getMessage() returns the clean stderr-based message; getFullOutput() exposes
 -        // both streams so the caller can check for "unknown flag: --profile" in either.
++        // getMessage() carries the clean stderr message for user-facing errors;
++        // getFullOutput() includes both streams so isUnknownFlagError can detect
++        // "unknown flag:" regardless of which stream the CLI wrote it to.
          throw new CliCommandException("cannot get access token: " + stderr, stdout + "\n" + stderr);
        }
        JsonNode jsonNode = new ObjectMapper().readTree(stdout);
@@ -159,10 +173,16 @@
 -                + "Falling back to a compatible command. "
 -                + "Please upgrade your CLI to the latest version.");
 -      } else {
--        throw new DatabricksException(e.getMessage(), e);
--      }
--    }
-+    IOException lastException = null;
++    int idx = activeCommandIndex.get();
++    if (idx >= 0) {
++      try {
++        return execCliCommand(commands.get(idx).cmd);
++      } catch (IOException e) {
+         throw new DatabricksException(e.getMessage(), e);
+       }
+     }
++    return probeAndExec();
++  }
  
 -    try {
 -      return execCliCommand(this.fallbackCmd);
@@ -173,16 +193,24 @@
 -                + "Falling back to a compatible command. "
 -                + "Please upgrade your CLI to the latest version.");
 -      } else {
-+    for (int i = 0; i < attempts.size(); i++) {
-+      CliCommand attempt = attempts.get(i);
++  /**
++   * Walks the command list from most-featured to simplest, looking for a CLI command that succeeds.
++   * When a command fails with "unknown flag" for one of its {@link CliCommand#usedFlags}, it logs a
++   * warning and tries the next. On success, {@link #activeCommandIndex} is stored so future calls
++   * skip probing.
++   */
++  private Token probeAndExec() {
++    for (int i = 0; i < commands.size(); i++) {
++      CliCommand command = commands.get(i);
 +      try {
-+        return execCliCommand(attempt.cmd);
++        Token token = execCliCommand(command.cmd);
++        activeCommandIndex.set(i);
++        return token;
 +      } catch (IOException e) {
-+        if (i + 1 < attempts.size() && isUnknownFlagError(getErrorText(e), attempt.usedFlags)) {
-+          if (attempt.fallbackMessage != null) {
-+            LOG.warn(attempt.fallbackMessage);
++        if (i + 1 < commands.size() && isUnknownFlagError(getErrorText(e), command.usedFlags)) {
++          if (command.fallbackMessage != null) {
++            LOG.warn(command.fallbackMessage);
 +          }
-+          lastException = e;
 +          continue;
 +        }
          throw new DatabricksException(e.getMessage(), e);
@@ -194,6 +222,6 @@
 -    } catch (IOException e) {
 -      throw new DatabricksException(e.getMessage(), e);
 -    }
-+    throw new DatabricksException(lastException.getMessage(), lastException);
++    throw new DatabricksException("cannot get access token: all CLI commands failed");
    }
  }
\ No newline at end of file

databricks-sdk-java/src/main/java/com/databricks/sdk/core/DatabricksCliCredentialsProvider.java

@@ -4,8 +4,8 @@
      return forceCmd;
    }
  
-+  List<CliTokenSource.CliCommand> buildAttempts(String cliPath, DatabricksConfig config) {
-+    List<CliTokenSource.CliCommand> attempts = new ArrayList<>();
++  List<CliTokenSource.CliCommand> buildCommands(String cliPath, DatabricksConfig config) {
++    List<CliTokenSource.CliCommand> commands = new ArrayList<>();
 +
 +    boolean hasProfile = config.getProfile() != null;
 +    boolean hasHost = config.getHost() != null;
@@ -13,7 +13,7 @@
 +    if (hasProfile) {
 +      List<String> profileCmd = buildProfileArgs(cliPath, config);
 +
-+      attempts.add(
++      commands.add(
 +          new CliTokenSource.CliCommand(
 +              withForceRefresh(profileCmd),
 +              Arrays.asList("--force-refresh", "--profile"),
@@ -21,7 +21,7 @@
 +                  + "Falling back to regular token fetch. "
 +                  + "Please upgrade your CLI to the latest version."));
 +
-+      attempts.add(
++      commands.add(
 +          new CliTokenSource.CliCommand(
 +              profileCmd,
 +              Collections.singletonList("--profile"),
@@ -30,12 +30,12 @@
 +    }
 +
 +    if (hasHost) {
-+      attempts.add(
++      commands.add(
 +          new CliTokenSource.CliCommand(
 +              buildHostArgs(cliPath, config), Collections.emptyList(), null));
 +    }
 +
-+    return attempts;
++    return commands;
 +  }
 +
    private CliTokenSource getDatabricksCliTokenSource(DatabricksConfig config) {
@@ -67,8 +67,8 @@
 -        config.getEnv(),
 -        fallbackCmd,
 -        secondFallbackCmd);
-+    return CliTokenSource.fromAttempts(
-+        buildAttempts(cliPath, config), "token_type", "access_token", "expiry", config.getEnv());
++    return CliTokenSource.fromCommands(
++        buildCommands(cliPath, config), "token_type", "access_token", "expiry", config.getEnv());
    }
  
    @Override
\ No newline at end of file

databricks-sdk-java/src/test/java/com/databricks/sdk/core/CliTokenSourceTest.java

@@ -11,9 +11,9 @@
  
    private CliTokenSource makeTokenSource(
        Environment env, List<String> cmd, List<String> fallbackCmd, List<String> secondFallbackCmd) {
-+    List<CliTokenSource.CliCommand> attempts = new ArrayList<>();
++    List<CliTokenSource.CliCommand> commands = new ArrayList<>();
 +
-+    attempts.add(
++    commands.add(
 +        new CliTokenSource.CliCommand(
 +            cmd,
 +            fallbackCmd != null
@@ -22,7 +22,7 @@
 +            fallbackCmd != null ? "fallback" : null));
 +
 +    if (fallbackCmd != null) {
-+      attempts.add(
++      commands.add(
 +          new CliTokenSource.CliCommand(
 +              fallbackCmd,
 +              secondFallbackCmd != null
@@ -32,7 +32,7 @@
 +    }
 +
 +    if (secondFallbackCmd != null) {
-+      attempts.add(new CliTokenSource.CliCommand(secondFallbackCmd, Collections.emptyList(), null));
++      commands.add(new CliTokenSource.CliCommand(secondFallbackCmd, Collections.emptyList(), null));
 +    }
 +
      OSUtilities osUtils = mock(OSUtilities.class);
@@ -41,7 +41,55 @@
        mockedOSUtils.when(() -> OSUtils.get(any())).thenReturn(osUtils);
 -      return new CliTokenSource(
 -          cmd, "token_type", "access_token", "expiry", env, fallbackCmd, secondFallbackCmd);
-+      return CliTokenSource.fromAttempts(attempts, "token_type", "access_token", "expiry", env);
++      return CliTokenSource.fromCommands(commands, "token_type", "access_token", "expiry", env);
      }
    }
- 
\ No newline at end of file
+ 
+       assertEquals(2, mocked.constructed().size());
+     }
+   }
++
++  @Test
++  public void testActiveCommandIndexPersists() {
++    Environment env = mock(Environment.class);
++    when(env.getEnv()).thenReturn(new HashMap<>());
++
++    CliTokenSource tokenSource = makeTokenSource(env, FORCE_CMD, PROFILE_CMD);
++
++    AtomicInteger callCount = new AtomicInteger(0);
++    try (MockedConstruction<ProcessBuilder> mocked =
++        mockConstruction(
++            ProcessBuilder.class,
++            (pb, context) -> {
++              int call = callCount.getAndIncrement();
++              if (call == 0) {
++                Process failProcess = mock(Process.class);
++                when(failProcess.getInputStream())
++                    .thenReturn(new ByteArrayInputStream(new byte[0]));
++                when(failProcess.getErrorStream())
++                    .thenReturn(
++                        new ByteArrayInputStream(
++                            "Error: unknown flag: --force-refresh".getBytes()));
++                when(failProcess.waitFor()).thenReturn(1);
++                when(pb.start()).thenReturn(failProcess);
++              } else {
++                Process successProcess = mock(Process.class);
++                when(successProcess.getInputStream())
++                    .thenReturn(
++                        new ByteArrayInputStream(validTokenJson("profile-token").getBytes()));
++                when(successProcess.getErrorStream())
++                    .thenReturn(new ByteArrayInputStream(new byte[0]));
++                when(successProcess.waitFor()).thenReturn(0);
++                when(pb.start()).thenReturn(successProcess);
++              }
++            })) {
++      Token first = tokenSource.getToken();
++      assertEquals("profile-token", first.getAccessToken());
++      assertEquals(2, mocked.constructed().size());
++
++      Token second = tokenSource.getToken();
++      assertEquals("profile-token", second.getAccessToken());
++      assertEquals(3, mocked.constructed().size());
++    }
++  }
+ }
\ No newline at end of file

databricks-sdk-java/src/test/java/com/databricks/sdk/core/DatabricksCliCredentialsProviderTest.java

@@ -26,78 +26,78 @@
      assertEquals(Arrays.asList(CLI_PATH, "auth", "token", "--profile", "my-profile"), cmd);
    }
 +
-+  // ---- Attempt chain construction tests ----
++  // ---- Command chain construction tests ----
 +
 +  @Test
 +  void testBuildAttempts_WithProfileAndHost() {
 +    DatabricksConfig config = new DatabricksConfig().setHost(HOST).setProfile("my-profile");
 +
-+    List<CliTokenSource.CliCommand> attempts = provider.buildAttempts(CLI_PATH, config);
++    List<CliTokenSource.CliCommand> commands = provider.buildCommands(CLI_PATH, config);
 +
-+    assertEquals(3, attempts.size());
++    assertEquals(3, commands.size());
 +
 +    assertEquals(
 +        Arrays.asList(CLI_PATH, "auth", "token", "--profile", "my-profile", "--force-refresh"),
-+        attempts.get(0).cmd);
-+    assertEquals(Arrays.asList("--force-refresh", "--profile"), attempts.get(0).usedFlags);
-+    assertEquals(FORCE_REFRESH_FALLBACK_MSG, attempts.get(0).fallbackMessage);
++        commands.get(0).cmd);
++    assertEquals(Arrays.asList("--force-refresh", "--profile"), commands.get(0).usedFlags);
++    assertEquals(FORCE_REFRESH_FALLBACK_MSG, commands.get(0).fallbackMessage);
 +
 +    assertEquals(
-+        Arrays.asList(CLI_PATH, "auth", "token", "--profile", "my-profile"), attempts.get(1).cmd);
-+    assertEquals(Collections.singletonList("--profile"), attempts.get(1).usedFlags);
-+    assertEquals(PROFILE_FALLBACK_MSG, attempts.get(1).fallbackMessage);
++        Arrays.asList(CLI_PATH, "auth", "token", "--profile", "my-profile"), commands.get(1).cmd);
++    assertEquals(Collections.singletonList("--profile"), commands.get(1).usedFlags);
++    assertEquals(PROFILE_FALLBACK_MSG, commands.get(1).fallbackMessage);
 +
-+    assertEquals(Arrays.asList(CLI_PATH, "auth", "token", "--host", HOST), attempts.get(2).cmd);
-+    assertEquals(Collections.emptyList(), attempts.get(2).usedFlags);
-+    assertNull(attempts.get(2).fallbackMessage);
++    assertEquals(Arrays.asList(CLI_PATH, "auth", "token", "--host", HOST), commands.get(2).cmd);
++    assertEquals(Collections.emptyList(), commands.get(2).usedFlags);
++    assertNull(commands.get(2).fallbackMessage);
 +  }
 +
 +  @Test
 +  void testBuildAttempts_WithProfileOnly() {
 +    DatabricksConfig config = new DatabricksConfig().setProfile("my-profile");
 +
-+    List<CliTokenSource.CliCommand> attempts = provider.buildAttempts(CLI_PATH, config);
++    List<CliTokenSource.CliCommand> commands = provider.buildCommands(CLI_PATH, config);
 +
-+    assertEquals(2, attempts.size());
++    assertEquals(2, commands.size());
 +
 +    assertEquals(
 +        Arrays.asList(CLI_PATH, "auth", "token", "--profile", "my-profile", "--force-refresh"),
-+        attempts.get(0).cmd);
-+    assertEquals(Arrays.asList("--force-refresh", "--profile"), attempts.get(0).usedFlags);
-+    assertEquals(FORCE_REFRESH_FALLBACK_MSG, attempts.get(0).fallbackMessage);
++        commands.get(0).cmd);
++    assertEquals(Arrays.asList("--force-refresh", "--profile"), commands.get(0).usedFlags);
++    assertEquals(FORCE_REFRESH_FALLBACK_MSG, commands.get(0).fallbackMessage);
 +
 +    assertEquals(
-+        Arrays.asList(CLI_PATH, "auth", "token", "--profile", "my-profile"), attempts.get(1).cmd);
-+    assertEquals(Collections.singletonList("--profile"), attempts.get(1).usedFlags);
-+    assertEquals(PROFILE_FALLBACK_MSG, attempts.get(1).fallbackMessage);
++        Arrays.asList(CLI_PATH, "auth", "token", "--profile", "my-profile"), commands.get(1).cmd);
++    assertEquals(Collections.singletonList("--profile"), commands.get(1).usedFlags);
++    assertEquals(PROFILE_FALLBACK_MSG, commands.get(1).fallbackMessage);
 +  }
 +
 +  @Test
 +  void testBuildAttempts_WithHostOnly() {
 +    DatabricksConfig config = new DatabricksConfig().setHost(HOST);
 +
-+    List<CliTokenSource.CliCommand> attempts = provider.buildAttempts(CLI_PATH, config);
++    List<CliTokenSource.CliCommand> commands = provider.buildCommands(CLI_PATH, config);
 +
-+    assertEquals(1, attempts.size());
++    assertEquals(1, commands.size());
 +
-+    assertEquals(Arrays.asList(CLI_PATH, "auth", "token", "--host", HOST), attempts.get(0).cmd);
-+    assertEquals(Collections.emptyList(), attempts.get(0).usedFlags);
-+    assertNull(attempts.get(0).fallbackMessage);
++    assertEquals(Arrays.asList(CLI_PATH, "auth", "token", "--host", HOST), commands.get(0).cmd);
++    assertEquals(Collections.emptyList(), commands.get(0).usedFlags);
++    assertNull(commands.get(0).fallbackMessage);
 +  }
 +
 +  @Test
 +  void testBuildAttempts_WithAccountHost() {
 +    DatabricksConfig config = new DatabricksConfig().setHost(ACCOUNT_HOST).setAccountId(ACCOUNT_ID);
 +
-+    List<CliTokenSource.CliCommand> attempts = provider.buildAttempts(CLI_PATH, config);
++    List<CliTokenSource.CliCommand> commands = provider.buildCommands(CLI_PATH, config);
 +
-+    assertEquals(1, attempts.size());
++    assertEquals(1, commands.size());
 +
 +    assertEquals(
 +        Arrays.asList(
 +            CLI_PATH, "auth", "token", "--host", ACCOUNT_HOST, "--account-id", ACCOUNT_ID),
-+        attempts.get(0).cmd);
-+    assertEquals(Collections.emptyList(), attempts.get(0).usedFlags);
-+    assertNull(attempts.get(0).fallbackMessage);
++        commands.get(0).cmd);
++    assertEquals(Collections.emptyList(), commands.get(0).usedFlags);
++    assertNull(commands.get(0).fallbackMessage);
 +  }
  }
\ No newline at end of file

_{Reproduce locally: git range-diff 1c68e85..0ef99dc 1c68e85..0dd48d1 | Disable: git config gitstack.push-range-diff false}

simonfaltum

Swarm Review Summary (Isaac + Cursor, 2 rounds)

Verdict: Not ready yet. 4 Major, 11 Nit, 4 Suggestion. No Critical.

The design mirrors the Go/Python SDK implementations well, DatabricksCliVersion is clean, and the buildCliCommand test matrix is solid. The concerns are concentrated in probeCliVersion + parseCliVersion + getCliVersion:

The four Majors

Parse failures are cached as successful UNKNOWN (line 263). Violates the "don't cache failures" contract, a transiently corrupt CLI response pins the SDK to the conservative fallback for the rest of the JVM lifetime.
asInt() silently coerces non-integer fields to 0 (line 329). {"Major":"v0","Minor":296,"Patch":0} parses as v0.296.0 and enables --force-refresh. Strictly worse failure mode than UNKNOWN.
Subprocess stream draining: latent pipe-buffer deadlock (line 286). waitFor before draining stdout/stderr is the textbook Java ProcessBuilder gotcha. Works today because output is ~100 bytes; breaks the first time a CLI release adds a stderr banner.
No tests for getCliVersion caching or probeCliVersion subprocess paths. The central correctness claim of this infrastructure, "cache successes, retry failures", is completely untested. Subprocess argument construction, OS wrapping, exit handling, timeout, and parse-failure handling also have zero coverage.

What went well

DatabricksCliVersion is clean, well-commented, with correct compareTo/equals/hashCode, and handles sentinels in toString.
Cache-by-cliPath (not by DatabricksConfig) is the right key.
"Don't cache failures" is the right policy and is explicitly documented, the bug is that the implementation doesn't match the policy for the parse-failure path.
Explicit dev-build (0, 0, 0) sentinel with isDefaultDevBuild() is thoughtful.
Removing fallbackCmd from CliTokenSource is a real simplification.

Reviewer alignment

Isaac and Cursor independently converged on: the subprocess deadlock, test-coverage gap, log noise (3 WARNs per failed probe), package-private constants, VERSION_CACHE reset hook, and concurrent-probe dedup. That's strong signal on the areas above.

Isaac found the --host null regression in buildCoreCliCommand; Cursor verified it is gated by configure()'s early-return in the production path, so it's downgraded to Nit as a local-invariant concern (still worth a guard or a javadoc note).

Cursor found the cached-UNKNOWN bug and the asInt() coercion issue, the two most consequential correctness findings, during cross-review.

Full synthesis: inline comments below.

simonfaltum · 2026-05-04T10:41:37Z

+
+    ProcessBuilder pb = new ProcessBuilder(cmd);
+    pb.environment().putAll(env.getEnv());
+    Process process = pb.start();


[Nit] Child stdin is never closed, interactive prompts would hang until timeout.

ProcessBuilder leaves the child's stdin as an open pipe to the parent. If the CLI ever reads from stdin (future interactive prompt, TTY-triggered behavior, some flag we didn't pass), the child blocks on read() and we eat the full 5-second timeout for every configure() call. Today databricks version doesn't read stdin, so this doesn't fire in practice, but it's a one-line fix to harden against it.

Suggestion:

Process process = pb.start(); process.getOutputStream().close();

Or, before start(): pb.redirectInput(ProcessBuilder.Redirect.from(new File(isWindows ? "NUL" : "/dev/null"))).

Found by: Cursor.

simonfaltum · 2026-05-04T10:41:38Z

+      LOG.warn(
+          "Databricks CLI {} does not support --force-refresh (requires >= {}). "
+              + "The CLI's token cache may provide stale tokens.",
+          version,
+          CLI_VERSION_FOR_FORCE_REFRESH);


[Nit] Warning message has no actionable fix for the user.

"Databricks CLI v0.xxx does not support --force-refresh (requires >= v0.296.0). The CLI's token cache may provide stale tokens." - a user who sees this has no idea that the fix is to upgrade the CLI.

Suggestion: append an upgrade hint, e.g. "Upgrade via brew upgrade databricks or https://docs.databricks.com/dev-tools/cli/install.html to silence this warning." Match the phrasing used in the Go SDK's equivalent message for cross-SDK consistency.

Found by: Cursor.

simonfaltum · 2026-05-04T10:41:38Z

+        process.destroyForcibly();
+        throw new IOException(
+            "timed out after "
+                + VERSION_PROBE_TIMEOUT_SECONDS
+                + "s waiting for `databricks version`");


[Suggestion] Capture and surface stderr on the timeout path for diagnostics.

On timeout, the IOException message is "timed out after 5s waiting for databricks version" with no clue why the CLI hung. Even partial stderr ("Initializing telemetry...", "Locking config dir...", "Checking for updates...") would help users diagnose AV scanners, lock contention, proxy issues, or first-run setup.

Suggestion: best-effort readStream(process.getErrorStream()) after destroyForcibly().waitFor(<short>) and append to the exception message. Wrap in its own try/catch, the stream may already be closed. This folds naturally into the concurrent-drain fix for the Major finding above.

Found by: Isaac.

simonfaltum · 2026-05-04T10:41:38Z

+    try {
+      DatabricksCliVersion version = probeCliVersion(cliPath, env);
+      VERSION_CACHE.put(cliPath, version);
+      return version;


[Suggestion] Log a DEBUG line on successful version detection.

When a user files a bug saying "force-refresh isn't being passed", the first diagnostic question is "what version did the probe detect?" Currently there's no way to answer without running under a debugger. One DEBUG line gives us free diagnosability.

Suggestion: LOG.debug("Detected Databricks CLI {} at {}", version, cliPath); after the successful probeCliVersion call.

Found by: Cursor.

simonfaltum · 2026-05-04T10:41:38Z

+  List<String> buildCliCommand(
+      String cliPath, DatabricksConfig config, DatabricksCliVersion version) {
+    List<String> cmd = buildCoreCliCommand(cliPath, config, version);
+    if (version.atLeast(CLI_VERSION_FOR_FORCE_REFRESH)) {
+      cmd.add("--force-refresh");
+    } else if (version.equals(DatabricksCliVersion.UNKNOWN) || version.isDefaultDevBuild()) {
+      // Detection failed or no version metadata — we can't prove the CLI lacks --force-refresh,
+      // just failed to confirm it. The version probe already logged the underlying cause.
+      LOG.warn(
+          "Could not confirm --force-refresh support for Databricks CLI {} (requires >= {}). "
+              + "The CLI's token cache may provide stale tokens.",
+          version,
+          CLI_VERSION_FOR_FORCE_REFRESH);
+    } else {
+      LOG.warn(
+          "Databricks CLI {} does not support --force-refresh (requires >= {}). "
+              + "The CLI's token cache may provide stale tokens.",
+          version,
+          CLI_VERSION_FOR_FORCE_REFRESH);
+    }
+    return cmd;
+  }


[Suggestion] Defensive copy before mutating the list from buildCoreCliCommand.

buildCliCommand mutates the list returned by buildCoreCliCommand via cmd.add("--force-refresh"). This works today only because buildCoreCliCommand and buildHostArgs happen to return ArrayList. A future refactor returning List.of(...) or Collections.unmodifiableList(...) will silently break this with UnsupportedOperationException at a call site that looks innocent.

Suggestion: either List<String> cmd = new ArrayList<>(buildCoreCliCommand(...)); at the top of this method, or document the mutability contract in javadoc on buildCoreCliCommand. The copy is more robust for a trivial cost.

Found by: Isaac.

`--profile` on `databricks auth token` is a global Cobra flag, so old CLIs (< v0.207.1) silently accept it and fail later with `cannot fetch credentials` instead of `unknown flag: --profile`. The previous error-based fallback never matched, leaving the `--host` fallback as dead code. This commit replaces the runtime fallback chain with version-based capability detection: * `CliVersion` carries a (major, minor, patch) triple plus an `UNKNOWN` sentinel and a default-dev-build (0,0,0) check. * `DatabricksCliCredentialsProvider` runs `databricks version --output json` once per CLI path (cached on success only, with a 5s timeout) and gates `--profile` on >= v0.207.1; everything else falls back to `--host` with a precise warning. * `CliTokenSource` is simplified to a single `cmd`; the `fallbackCmd` parameter and the runtime "unknown flag" retry loop are removed. Mirrors the equivalent refactors in the Go and Python SDKs: * databricks/databricks-sdk-go#1605 * databricks/databricks-sdk-py#1377 Co-authored-by: Isaac

The SDK manages its own token caching via `CachedTokenSource`. When the SDK shells out to `databricks auth token`, the CLI may return a token from *its* own cache that is about to expire (or has already expired from the SDK's perspective), producing unnecessary refresh failures and retry loops. The CLI added `--force-refresh` in v0.296.0 (databricks/cli#4767) to let callers bypass its cache. With the version-detection infrastructure from the parent PR already in place, opting in is a one-constant, one-branch change: * Introduce `CLI_VERSION_FOR_FORCE_REFRESH = v0.296.0`. * Split `buildCliCommand` into the existing profile/host decision (now `buildCoreCliCommand`) and a thin wrapper that appends `--force-refresh` when supported and otherwise logs a precise warning. Future capability-gated flags slot into the same wrapper. Mirrors: * databricks/databricks-sdk-go#1628 * databricks/databricks-sdk-py#1378 Co-authored-by: Isaac

github-actions · 2026-05-04T14:47:43Z

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/sdk-java

Inputs:

PR number: 752
Commit SHA: 6bbf406a7c8d7ef8b275e83d3881134da46d2856

Checks will be approved automatically on success.

mihaimitrea-db mentioned this pull request Mar 31, 2026

Fix CLI token source --profile fallback with version detection #751

Open

mihaimitrea-db temporarily deployed to test-trigger-is March 31, 2026 12:58 — with GitHub Actions Inactive

mihaimitrea-db temporarily deployed to test-trigger-is March 31, 2026 12:59 — with GitHub Actions Inactive

mihaimitrea-db force-pushed the mihaimitrea-db/stack/cli-attempt-chain branch from 628f509 to 09cdbc4 Compare March 31, 2026 13:09

mihaimitrea-db temporarily deployed to test-trigger-is March 31, 2026 13:09 — with GitHub Actions Inactive

mihaimitrea-db temporarily deployed to test-trigger-is March 31, 2026 13:10 — with GitHub Actions Inactive

mihaimitrea-db force-pushed the mihaimitrea-db/stack/cli-attempt-chain branch from 09cdbc4 to f88c52a Compare March 31, 2026 13:22

mihaimitrea-db temporarily deployed to test-trigger-is March 31, 2026 13:22 — with GitHub Actions Inactive

mihaimitrea-db temporarily deployed to test-trigger-is March 31, 2026 13:24 — with GitHub Actions Inactive

mihaimitrea-db force-pushed the mihaimitrea-db/stack/cli-attempt-chain branch from f88c52a to 25a3779 Compare March 31, 2026 14:04

mihaimitrea-db temporarily deployed to test-trigger-is March 31, 2026 14:04 — with GitHub Actions Inactive

mihaimitrea-db temporarily deployed to test-trigger-is March 31, 2026 14:05 — with GitHub Actions Inactive

mihaimitrea-db force-pushed the mihaimitrea-db/stack/cli-attempt-chain branch from 25a3779 to 694521d Compare March 31, 2026 15:01

mihaimitrea-db temporarily deployed to test-trigger-is March 31, 2026 15:01 — with GitHub Actions Inactive

mihaimitrea-db temporarily deployed to test-trigger-is March 31, 2026 15:03 — with GitHub Actions Inactive

mihaimitrea-db force-pushed the mihaimitrea-db/stack/cli-attempt-chain branch from 694521d to 0f6baf6 Compare March 31, 2026 16:03

mihaimitrea-db temporarily deployed to test-trigger-is March 31, 2026 16:03 — with GitHub Actions Inactive

mihaimitrea-db self-assigned this Apr 1, 2026

mihaimitrea-db force-pushed the mihaimitrea-db/stack/cli-attempt-chain branch from 0f6baf6 to 0ef99dc Compare April 1, 2026 07:59

mihaimitrea-db temporarily deployed to test-trigger-is April 1, 2026 07:59 — with GitHub Actions Inactive

mihaimitrea-db had a problem deploying to test-trigger-is April 1, 2026 08:00 — with GitHub Actions Failure

mihaimitrea-db force-pushed the mihaimitrea-db/stack/cli-attempt-chain branch from 0ef99dc to 0dd48d1 Compare April 1, 2026 08:26

mihaimitrea-db temporarily deployed to test-trigger-is April 1, 2026 08:26 — with GitHub Actions Inactive

mihaimitrea-db marked this pull request as ready for review April 1, 2026 09:29

mihaimitrea-db removed the request for review from renaudhartert-db April 14, 2026 13:59

mihaimitrea-db force-pushed the mihaimitrea-db/stack/cli-attempt-chain branch from 0dd48d1 to a69ceff Compare April 30, 2026 13:28

mihaimitrea-db temporarily deployed to test-trigger-is April 30, 2026 13:28 — with GitHub Actions Inactive

mihaimitrea-db temporarily deployed to test-trigger-is April 30, 2026 13:30 — with GitHub Actions Inactive

mihaimitrea-db force-pushed the mihaimitrea-db/stack/cli-attempt-chain branch from a69ceff to 68844a4 Compare April 30, 2026 13:37

mihaimitrea-db temporarily deployed to test-trigger-is April 30, 2026 13:37 — with GitHub Actions Inactive

mihaimitrea-db temporarily deployed to test-trigger-is April 30, 2026 13:39 — with GitHub Actions Inactive

mihaimitrea-db changed the title ~~Refactor CliTokenSource to use an ordered attempt chain~~ Pass --force-refresh to CLI auth token command Apr 30, 2026

mihaimitrea-db force-pushed the mihaimitrea-db/stack/cli-attempt-chain branch from 68844a4 to 5ee2187 Compare April 30, 2026 14:51

mihaimitrea-db temporarily deployed to test-trigger-is April 30, 2026 14:51 — with GitHub Actions Inactive

mihaimitrea-db temporarily deployed to test-trigger-is April 30, 2026 14:52 — with GitHub Actions Inactive

mihaimitrea-db requested a review from simonfaltum April 30, 2026 15:41

simonfaltum reviewed May 4, 2026

View reviewed changes

mihaimitrea-db force-pushed the mihaimitrea-db/stack/cli-attempt-chain branch from 5ee2187 to b68965e Compare May 4, 2026 13:30

mihaimitrea-db temporarily deployed to test-trigger-is May 4, 2026 13:30 — with GitHub Actions Inactive

mihaimitrea-db temporarily deployed to test-trigger-is May 4, 2026 13:32 — with GitHub Actions Inactive

mihaimitrea-db force-pushed the mihaimitrea-db/stack/cli-attempt-chain branch from b68965e to 9b49187 Compare May 4, 2026 13:57

mihaimitrea-db temporarily deployed to test-trigger-is May 4, 2026 13:57 — with GitHub Actions Inactive

mihaimitrea-db temporarily deployed to test-trigger-is May 4, 2026 13:58 — with GitHub Actions Inactive

mihaimitrea-db force-pushed the mihaimitrea-db/stack/cli-attempt-chain branch from 9b49187 to dbc2062 Compare May 4, 2026 14:27

mihaimitrea-db temporarily deployed to test-trigger-is May 4, 2026 14:27 — with GitHub Actions Inactive

mihaimitrea-db added 2 commits May 4, 2026 14:46

mihaimitrea-db force-pushed the mihaimitrea-db/stack/cli-attempt-chain branch from dbc2062 to 6bbf406 Compare May 4, 2026 14:47

mihaimitrea-db temporarily deployed to test-trigger-is May 4, 2026 14:47 — with GitHub Actions Inactive

mihaimitrea-db deployed to test-trigger-is May 4, 2026 14:47 — with GitHub Actions Active

mihaimitrea-db requested a review from simonfaltum May 4, 2026 15:23

Conversation

mihaimitrea-db commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🥞 Stacked PR

Summary

Why

What changed

Interface changes

Behavioral changes

Internal changes

How is this tested?

Uh oh!

mihaimitrea-db commented Mar 31, 2026

Uh oh!

mihaimitrea-db commented Mar 31, 2026

Uh oh!

mihaimitrea-db commented Mar 31, 2026

Uh oh!

mihaimitrea-db commented Mar 31, 2026

Uh oh!

mihaimitrea-db commented Mar 31, 2026

Uh oh!

mihaimitrea-db commented Apr 1, 2026

Uh oh!

simonfaltum left a comment

Choose a reason for hiding this comment

Swarm Review Summary (Isaac + Cursor, 2 rounds)

The four Majors

What went well

Reviewer alignment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

simonfaltum May 4, 2026

Choose a reason for hiding this comment

Uh oh!

simonfaltum May 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

simonfaltum May 4, 2026

Choose a reason for hiding this comment

Uh oh!

simonfaltum May 4, 2026

Choose a reason for hiding this comment

Uh oh!

simonfaltum May 4, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented May 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mihaimitrea-db commented Mar 31, 2026 •

edited

Loading