acc: CloudSlow only narrows Cloud = true tests

.agents/rules/testing.md

@@ -56,9 +56,9 @@ When writing tests, don't include an explanation in each test case in your respo
 
 ## Acceptance Tests
 
-**RULE: `Cloud`/`CloudSlow` add a cloud run; they never remove the local run.** Every test under `acceptance/` runs locally against the fake server in `libs/testserver`. `Cloud = true` in a `test.toml` or `out.test.toml` means "this test *also* runs against a real workspace when `CLOUD_ENV` is set" — it never means "this test does not run locally". Never tell the user a test doesn't run locally because it is `Cloud = true`.
+**RULE: `Cloud = true` adds a cloud run; it never removes the local run.** Every test under `acceptance/` runs locally against the fake server in `libs/testserver`. `Cloud = true` in a `test.toml` or `out.test.toml` means "this test *also* runs against a real workspace when `CLOUD_ENV` is set" — it never means "this test does not run locally". Never tell the user a test doesn't run locally because it is `Cloud = true`. `CloudSlow` does *not* enable a cloud run on its own; it only narrows an existing `Cloud = true` run (see below).
 
-The whole `Cloud*` family lives inside an `if isRunningOnCloud` branch in `getSkipReason` (`acceptance/acceptance_test.go`), so it can only ever subtract from the cloud run: `CloudSlow` drops it under `-short`, and `CloudEnvs` / `RequiresUnityCatalog` / `RequiresCluster` / `RequiresWarehouse` narrow it to environments that have the prerequisite. To find what skips a test *locally*, look at a different set: `GOOS`, `RunsOnDbr`, and `DATABRICKS_TEST_SKIPLOCAL` (which cloud CI runs set precisely because those tests already ran locally).
+The whole `Cloud*` family lives inside an `if isRunningOnCloud` branch in `getSkipReason` (`acceptance/acceptance_test.go`), so it can only ever subtract from the cloud run: `CloudSlow` (only meaningful when `Cloud = true`) drops it under `-short`, and `CloudEnvs` / `RequiresUnityCatalog` / `RequiresCluster` / `RequiresWarehouse` narrow it to environments that have the prerequisite. To find what skips a test *locally*, look at a different set: `GOOS`, `RunsOnDbr`, and `DATABRICKS_TEST_SKIPLOCAL` (which cloud CI runs set precisely because those tests already ran locally).
 
 `Cloud` is inherited, so a parent `test.toml` can opt a whole subtree in; a leaf `test.toml` with no `Cloud` line is not evidence of anything. Read the generated `out.test.toml` for a test's effective settings.
 
@@ -127,14 +127,14 @@ Ignore = ["databricks.yml"]   # parsed as EnvMatrix.Ignore, not top-level Ignore
 
 ### Reference
 
-- Tests live in `acceptance/` with a nested directory structure. All of them run locally; those with `Cloud`/`CloudSlow` set also run against a real workspace.
+- Tests live in `acceptance/` with a nested directory structure. All of them run locally; those with `Cloud = true` set also run against a real workspace.
 - Each test directory contains `databricks.yml`, `script`, and `output.txt`.
 - Source files: `test.toml`, `script`, `script.prepare`, `databricks.yml`, etc.
 - Tests are configured via `test.toml`. Config schema and explanation is in `acceptance/internal/config.go`. Certain options are also dumped to `out.test.toml` so that inherited values are visible on PRs.
 - Run a single test: `go test ./acceptance -run TestAccept/bundle/<path>/<to>/<folder>`
 - Run a specific variant by appending `EnvMatrix` values to the test name: `go test ./acceptance -run 'TestAccept/.../DATABRICKS_BUNDLE_ENGINE=direct'`. When there are multiple `EnvMatrix` variables, they appear in alphabetical order.
 - Useful flags: `-v` for verbose output, `-tail` to follow test output (requires `-v`), `-logrequests` to log all HTTP requests/responses (requires `-v`).
-- Run tests on cloud: `deco env run -i -n aws-prod-ucws -- <go test command>` (requires `deco` tool and access to test env). This is an *additional* pass over the same test directories, restricted to those with `Cloud`/`CloudSlow` set; it does not replace the local run.
+- Run tests on cloud: `deco env run -i -n aws-prod-ucws -- <go test command>` (requires `deco` tool and access to test env). This is an *additional* pass over the same test directories, restricted to those with `Cloud = true` set; it does not replace the local run.
 - `script.prepare` files from parent directories are concatenated into the test script. Use them for shared bash helpers.
 
 ### Built-in shell helpers

acceptance/README.md

@@ -6,7 +6,7 @@ Acceptance tests are blackbox tests that are run against compiled binary.
 
 A test can *additionally* opt into running against a real workspace by setting `Cloud = true` in its `test.toml`. That is an extra run, not a different one: `Cloud = true` never means "this test does not run locally". Related settings:
 
-- `CloudSlow = true` implies `Cloud = true`, but the cloud run is skipped when `-short` is passed.
+- `CloudSlow = true` only narrows a `Cloud = true` test: its cloud run is skipped when `-short` is passed. On its own (without `Cloud = true`) it does not enable the cloud run.
 - `CloudEnvs` and `RequiresUnityCatalog` / `RequiresCluster` / `RequiresWarehouse` only narrow the cloud run further. They are ignored locally.
 - `Cloud` is inherited from parent `test.toml` files, so a whole subtree can be opted in at once. The root `acceptance/test.toml` defaults it to `Cloud = false`, i.e. local only.
 - Each test's effective settings are visible in its generated `out.test.toml`.

acceptance/acceptance_test.go

@@ -499,12 +499,6 @@ func testAccept(t *testing.T, inprocessMode bool, singleTest string) int {
 				t.Fatalf("Invalid config %s: %s", configPath, err)
 			}
 
-			// Apply default: CloudSlow implies Cloud. Do this before generating
-			// the materialized config so the implication is visible in out.test.toml.
-			if isTruePtr(config.CloudSlow) {
-				config.Cloud = config.CloudSlow
-			}
-
 			// Generate materialized config for this test.
 			// We do this before skipping the test, so the configs are generated for all tests.
 			materializedConfig := internal.GenerateMaterializedConfig(&config)
@@ -686,20 +680,13 @@ func getSkipReason(config *internal.TestConfig, configPath, dir, skipLocalMode s
 			return fmt.Sprintf("Disabled via CloudEnvs.%s setting in %s (CLOUD_ENV=%s)", cloudEnvBase, configPath, cloudEnv)
 		}
 
-		if isTruePtr(config.CloudSlow) {
-			if testing.Short() {
-				return fmt.Sprintf("Disabled via CloudSlow setting in %s (CLOUD_ENV=%s, Short=%v)", configPath, cloudEnv, testing.Short())
-			}
+		if !isTruePtr(config.Cloud) {
+			return fmt.Sprintf("Disabled via Cloud setting in %s (CLOUD_ENV=%s)", configPath, cloudEnv)
 		}
 
-		isCloudEnabled := isTruePtr(config.Cloud) || isTruePtr(config.CloudSlow)
-		if !isCloudEnabled {
-			return fmt.Sprintf("Disabled via Cloud/CloudSlow setting in %s (CLOUD_ENV=%s, Cloud=%v, CloudSlow=%v)",
-				configPath,
-				cloudEnv,
-				isTruePtr(config.Cloud),
-				isTruePtr(config.CloudSlow),
-			)
+		// CloudSlow only narrows an already-enabled cloud run: skip it under -short.
+		if isTruePtr(config.CloudSlow) && testing.Short() {
+			return fmt.Sprintf("Disabled via CloudSlow setting in %s (CLOUD_ENV=%s, Short=%v)", configPath, cloudEnv, testing.Short())
 		}
 
 		if isTruePtr(config.RequiresUnityCatalog) && os.Getenv("TEST_METASTORE_ID") == "" {

acceptance/bundle/integration_whl/test.toml

@@ -1,3 +1,4 @@
+Cloud = true
 CloudSlow = true
 
 # Workspace file system does not allow initializing python envs on it.

acceptance/bundle/resources/clusters/run/spark_python_task/test.toml

@@ -1,3 +1,4 @@
+Cloud = true
 CloudSlow = true
 RecordRequests = false
 RunsOnDbr = true

acceptance/bundle/resources/permissions/factcheck/test.toml

@@ -1,3 +1,4 @@
+Cloud = true
 CloudSlow = true
 RecordRequests = false
 RunsOnDbr = true

acceptance/bundle/resources/sql_warehouses/lifecycle-started-edit/test.toml

@@ -1,7 +1,9 @@
 RecordRequests = true
 
 # Starting warehouses is slow, so run on cloud nightly (CloudSlow) instead of every PR
-# to confirm the real Edit-restarts behaviour without per-PR timeouts.
+# to confirm the real Edit-restarts behaviour without per-PR timeouts. The parent
+# disables Cloud, so re-enable it here alongside CloudSlow.
+Cloud = true
 CloudSlow = true
 
 Ignore = [".databricks", "databricks.yml"]

acceptance/bundle/resources/vector_search_indexes/drift/deleted_remotely/test.toml

@@ -1,3 +1 @@
 Cloud = false
-# The parent sets CloudSlow=true; disable it here too, otherwise CloudSlow would imply Cloud=true.
-CloudSlow = false

acceptance/bundle/resources/vector_search_indexes/drift/orphaned_endpoint/test.toml

@@ -1,3 +1 @@
 Cloud = false
-# The parent sets CloudSlow=true; disable it here too, otherwise CloudSlow would imply Cloud=true.
-CloudSlow = false

acceptance/bundle/resources/vector_search_indexes/recreate/pending_deletion/test.toml

@@ -1,4 +1,2 @@
 # The two-phase deletion race is simulated by the testserver, so this only runs locally.
 Cloud = false
-# The parent sets CloudSlow=true; disable it here too, otherwise CloudSlow would imply Cloud=true.
-CloudSlow = false

acceptance/bundle/resources/vector_search_indexes/recreate/with_endpoint/test.toml

@@ -1,5 +1,3 @@
 Cloud = false
-# The parent sets CloudSlow=true; disable it here too, otherwise CloudSlow would imply Cloud=true.
-CloudSlow = false
 
 Badness = "Recreating a vector_search_endpoint does not cascade to its dependent vector_search_indexes. The first deploy recreates the endpoint and leaves the index attached to a stale endpoint UUID; a second plan/deploy is required to reconcile the index. Follow-up will add a generic per-resource recreate cascade rule to the framework."

acceptance/bundle/run/app-with-job/test.toml

@@ -5,6 +5,7 @@
 # - 10-20 seconds to deploy the bundle
 # This test includes 2 application starts, and it is taking aroung 420 seconds to complete the entire test
 #
+Cloud = true
 CloudSlow = true
 
 Ignore = [

acceptance/internal/config.go

@@ -52,8 +52,8 @@ type TestConfig struct {
 	// Does not affect the local run, which happens either way.
 	Cloud *bool
 
-	// Like Cloud, but the cloud run is skipped when -short is passed.
-	// This also sets -tail when -v is passed. Implies Cloud.
+	// Only meaningful alongside Cloud=true: the cloud run is skipped when -short is passed.
+	// This also sets -tail when -v is passed. It does not enable the cloud run on its own.
 	CloudSlow *bool
 
 	// If true and Cloud=true, run the cloud part of this test only if unity catalog is available in the cloud environment

acceptance/test.toml

@@ -1,6 +1,7 @@
 # Default settings that apply to all tests unless overriden by test.toml files in inner directories.
-# Tests always run locally; set Cloud/CloudSlow to also run on a real workspace.
+# Tests always run locally; set Cloud = true to also run on a real workspace.
 # Cloud = false (the inherited default) therefore means local only.
+# CloudSlow only narrows a Cloud = true run (skipped under -short); it does not enable cloud on its own.
 Cloud = false
 
 # default timeouts