Repository navigation
docs(storage): add Rapid Cache samples - #14886
sahusneha2004 wants to merge 2 commits into
Conversation
Adds create, get, list, update and disable Rapid Cache samples for Storage Control, with an integration test. Bumps libraries-bom to 26.90.0 (storage-control 2.75.0 with RapidCache). b/482946642 [Generated-by: AI]
There was a problem hiding this comment.
Code Review
This pull request introduces code snippets and integration tests for managing Rapid Cache in Google Cloud Storage Control v2, including creation, retrieval, listing, updating, and disabling operations, alongside updating the necessary Maven dependencies. The review feedback focuses on improving the robustness and readability of the integration tests by moving the Rapid Cache disabling logic into the tearDown method, declaring cacheId as an instance field, and simplifying the test flow by removing the complex manual try-catch-finally block.
| private String bucketName; | ||
| private int outputOffset; |
There was a problem hiding this comment.
Declare cacheId as an instance field so that it can be accessed in the tearDown method for robust cleanup, even if the test fails before reaching the manual cleanup block.
| private String bucketName; | |
| private int outputOffset; | |
| private String bucketName; | |
| private String cacheId; | |
| private int outputOffset; |
| public void tearDown() throws InterruptedException { | ||
| if (bucketName == null) { | ||
| return; | ||
| } | ||
| // Best-effort cleanup. A bucket with a Rapid Cache cannot be deleted until the cache has been | ||
| // fully removed, which can take a long time after it is disabled. Any leftover bucket is | ||
| // removed by the stale bucket cleanup. | ||
| long deadline = System.nanoTime() + TimeUnit.SECONDS.toNanos(BUCKET_DELETE_TIMEOUT_SECONDS); | ||
| while (true) { | ||
| try { | ||
| storage.delete(bucketName); | ||
| return; | ||
| } catch (StorageException e) { | ||
| if (e.getCode() != 409 || System.nanoTime() > deadline) { | ||
| System.err.printf("Leaving bucket %s for cleanup: %s%n", bucketName, e.getMessage()); | ||
| return; | ||
| } | ||
| TimeUnit.SECONDS.sleep(BUCKET_DELETE_INTERVAL_SECONDS); | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
Update the tearDown method to automatically disable the Rapid Cache if cacheId is set. This ensures that resources are cleaned up reliably even if assertions or other operations in the test fail.
@After
public void tearDown() throws InterruptedException {
if (bucketName == null) {
return;
}
if (cacheId != null) {
try {
RapidCacheDisable.rapidCacheDisable(bucketName, cacheId);
} catch (Exception e) {
System.err.printf("Failed to disable Rapid Cache %s: %s%n", cacheId, e.getMessage());
}
}
// Best-effort cleanup. A bucket with a Rapid Cache cannot be deleted until the cache has been
// fully removed, which can take a long time after it is disabled. Any leftover bucket is
// removed by the stale bucket cleanup.
long deadline = System.nanoTime() + TimeUnit.SECONDS.toNanos(BUCKET_DELETE_TIMEOUT_SECONDS);
while (true) {
try {
storage.delete(bucketName);
return;
} catch (StorageException e) {
if (e.getCode() != 409 || System.nanoTime() > deadline) {
System.err.printf("Leaving bucket %s for cleanup: %s%n", bucketName, e.getMessage());
return;
}
TimeUnit.SECONDS.sleep(BUCKET_DELETE_INTERVAL_SECONDS);
}
}
}References
- In test cleanup or resource deletion logic, catching a broad Exception is acceptable to ensure that the cleanup process is not halted by individual failures.
| public void createGetListUpdateDisable() throws Exception { | ||
| // Test create Rapid Cache | ||
| RapidCacheCreate.rapidCacheCreate(bucketName, ZONE.getZone()); | ||
| String out = newOutput(); | ||
| assertThat(out).contains(CREATED_PREFIX); | ||
| String cacheName = | ||
| out.substring(out.indexOf(CREATED_PREFIX) + CREATED_PREFIX.length()).trim().split("\\s")[0]; | ||
| String cacheId = RapidCacheName.parse(cacheName).getRapidCache(); | ||
| assertThat(cacheId).isNotEmpty(); | ||
|
|
||
| Throwable failure = null; | ||
| try { | ||
| // Test get Rapid Cache | ||
| RapidCacheGet.rapidCacheGet(bucketName, cacheId); | ||
| out = newOutput(); | ||
| assertThat(out).contains(cacheName); | ||
| assertThat(out).contains("Zone: " + ZONE.getZone()); | ||
| assertThat(out).contains("State: "); | ||
|
|
||
| // Test list Rapid Caches | ||
| RapidCacheList.rapidCacheList(bucketName); | ||
| assertThat(newOutput()).contains(cacheName); | ||
|
|
||
| // Test update Rapid Cache | ||
| RapidCacheUpdate.rapidCacheUpdate(bucketName, cacheId, 172800); | ||
| out = newOutput(); | ||
| assertThat(out).contains("Updated Rapid Cache: " + cacheName); | ||
| assertThat(out).contains("TTL: 172800 seconds"); | ||
| } catch (Throwable t) { | ||
| failure = t; | ||
| throw t; | ||
| } finally { | ||
| // Disable the Rapid Cache. This also cleans up the cache so that the bucket can be deleted | ||
| // once the cache has been fully removed. | ||
| newOutput(); | ||
| try { | ||
| RapidCacheDisable.rapidCacheDisable(bucketName, cacheId); | ||
| } catch (Exception e) { | ||
| if (failure == null) { | ||
| throw e; | ||
| } | ||
| // Keep the original test failure as the primary error. | ||
| failure.addSuppressed(e); | ||
| } | ||
| } | ||
|
|
||
| // Test disable Rapid Cache | ||
| String disableOut = newOutput(); | ||
| assertThat(disableOut).contains("Disabling Rapid Cache: " + cacheName); | ||
| assertThat(disableOut).contains("Operation: "); | ||
|
|
||
| // The cache moves to the "disabled" state shortly after the request. | ||
| for (int i = 0; i < DISABLE_POLL_ATTEMPTS; i++) { | ||
| RapidCacheGet.rapidCacheGet(bucketName, cacheId); | ||
| out = newOutput(); | ||
| if (out.contains("State: disabled")) { | ||
| break; | ||
| } | ||
| TimeUnit.SECONDS.sleep(DISABLE_POLL_INTERVAL_SECONDS); | ||
| } | ||
| assertThat(out).contains("State: disabled"); | ||
| } |
There was a problem hiding this comment.
Simplify the test method by removing the complex manual try-catch-finally block. Since cleanup is now handled in tearDown, we can write a clean, linear test flow. We also set cacheId = null at the end of the test to prevent double-disabling in tearDown.
@Test
public void createGetListUpdateDisable() throws Exception {
// Test create Rapid Cache
RapidCacheCreate.rapidCacheCreate(bucketName, ZONE.getZone());
String out = newOutput();
assertThat(out).contains(CREATED_PREFIX);
String cacheName =
out.substring(out.indexOf(CREATED_PREFIX) + CREATED_PREFIX.length()).trim().split("\\s")[0];
cacheId = RapidCacheName.parse(cacheName).getRapidCache();
assertThat(cacheId).isNotEmpty();
// Test get Rapid Cache
RapidCacheGet.rapidCacheGet(bucketName, cacheId);
out = newOutput();
assertThat(out).contains(cacheName);
assertThat(out).contains("Zone: " + ZONE.getZone());
assertThat(out).contains("State: ");
// Test list Rapid Caches
RapidCacheList.rapidCacheList(bucketName);
assertThat(newOutput()).contains(cacheName);
// Test update Rapid Cache
RapidCacheUpdate.rapidCacheUpdate(bucketName, cacheId, 172800);
out = newOutput();
assertThat(out).contains("Updated Rapid Cache: " + cacheName);
assertThat(out).contains("TTL: 172800 seconds");
// Test disable Rapid Cache
RapidCacheDisable.rapidCacheDisable(bucketName, cacheId);
String disableOut = newOutput();
assertThat(disableOut).contains("Disabling Rapid Cache: " + cacheName);
assertThat(disableOut).contains("Operation: ");
// The cache moves to the "disabled" state shortly after the request.
for (int i = 0; i < DISABLE_POLL_ATTEMPTS; i++) {
RapidCacheGet.rapidCacheGet(bucketName, cacheId);
out = newOutput();
if (out.contains("State: disabled")) {
break;
}
TimeUnit.SECONDS.sleep(DISABLE_POLL_INTERVAL_SECONDS);
}
assertThat(out).contains("State: disabled");
cacheId = null; // Prevent double-disabling in tearDown
}References
- In integration tests, prioritize consistency and readability with existing test patterns (such as sequential client teardown) over complex nested try-finally blocks, unless a repository-wide cleanup is being performed.
Set the explicit cache name (ID = zone), cache_type rapid-cache-ultra and ttl 24h on create; wait for the disable LRO; tests assert running state, cache type, list membership and the updated ttl, and bound the disable wait. Aligns with #14529. b/482946642 [Generated-by: AI]
nidhiii-27
left a comment
There was a problem hiding this comment.
Please regenerate java-storage/README.md via librarian generate — this is critical so merging the PR does not break the post-submit Librarian - Generate libraries check on main.
| <groupId>com.google.cloud</groupId> | ||
| <artifactId>google-cloud-storage</artifactId> | ||
| <version>2.64.0</version> | ||
| <version>2.75.0</version> |
There was a problem hiding this comment.
Please remove changes in this file, we do not modify these files directly, they are autgenerated.
| <groupId>com.google.cloud</groupId> | ||
| <artifactId>libraries-bom</artifactId> | ||
| <version>26.78.0</version> | ||
| <version>26.90.0</version> |
There was a problem hiding this comment.
Please remove changes from this file, we do not edit these files manually.
|
|
||
| // The zone in which to create the Rapid Cache. The zone must be in the same region as the | ||
| // bucket. | ||
| // String zoneName = "us-central1-a"; |
There was a problem hiding this comment.
Please rename this variable to cacheId as followed across other samples (RapidCacheGet, RapidCacheUpdate)
| .setZone(zoneName) | ||
| .setCacheType("rapid-cache-ultra") | ||
| // How long a cache entry can live (between 1 hour and 7 days). 86400s = 24 hours. | ||
| .setTtl(Duration.newBuilder().setSeconds(86400).build()) |
There was a problem hiding this comment.
In AnywhereCacheCreate.java, AnywhereCacheCreate only sets the required fields (.setName(...).setZone(zoneName)) and leaves TTL configuration to AnywhereCacheUpdate. Let's follow the same here.
| ExecutorService executor = Executors.newSingleThreadExecutor(); | ||
| try { | ||
| Future<?> disable = | ||
| executor.submit( | ||
| () -> { | ||
| RapidCacheDisable.rapidCacheDisable(bucketName, cacheId); | ||
| return null; | ||
| }); | ||
| try { | ||
| disable.get(DISABLE_TIMEOUT_MINUTES, TimeUnit.MINUTES); | ||
| cacheDisabled = true; | ||
| assertThat(newOutput()).contains("Disabled Rapid Cache: " + cacheName); | ||
| } catch (TimeoutException | ExecutionException e) { | ||
| // A TimeoutException means our wait expired. A CancellationException means the client | ||
| // stopped polling the operation (the default LRO polling timeout is 5 minutes). In both | ||
| // cases the operation is still running on the server, and the cache must already be in | ||
| // the "disabled" state. Any other error fails the test. | ||
| if (e instanceof ExecutionException && !(e.getCause() instanceof CancellationException)) { | ||
| throw e; | ||
| } | ||
| // Cancelling the local wait does not cancel the operation on the server. | ||
| disable.cancel(true); | ||
| cacheDisabled = true; | ||
| newOutput(); | ||
| RapidCacheGet.rapidCacheGet(bucketName, cacheId); | ||
| assertThat(newOutput()).contains("State: disabled"); | ||
| } | ||
| } finally { | ||
| executor.shutdownNow(); | ||
| } |
There was a problem hiding this comment.
Instead of spawning a single-threaded ExecutorService and waiting for DISABLE_TIMEOUT_MINUTES (10 minutes), we can simply call RapidCacheDisable.rapidCacheDisable(bucketName, cacheId) directly and catch ExecutionException when caused by CancellationException
| ExecutorService executor = Executors.newSingleThreadExecutor(); | |
| try { | |
| Future<?> disable = | |
| executor.submit( | |
| () -> { | |
| RapidCacheDisable.rapidCacheDisable(bucketName, cacheId); | |
| return null; | |
| }); | |
| try { | |
| disable.get(DISABLE_TIMEOUT_MINUTES, TimeUnit.MINUTES); | |
| cacheDisabled = true; | |
| assertThat(newOutput()).contains("Disabled Rapid Cache: " + cacheName); | |
| } catch (TimeoutException | ExecutionException e) { | |
| // A TimeoutException means our wait expired. A CancellationException means the client | |
| // stopped polling the operation (the default LRO polling timeout is 5 minutes). In both | |
| // cases the operation is still running on the server, and the cache must already be in | |
| // the "disabled" state. Any other error fails the test. | |
| if (e instanceof ExecutionException && !(e.getCause() instanceof CancellationException)) { | |
| throw e; | |
| } | |
| // Cancelling the local wait does not cancel the operation on the server. | |
| disable.cancel(true); | |
| cacheDisabled = true; | |
| newOutput(); | |
| RapidCacheGet.rapidCacheGet(bucketName, cacheId); | |
| assertThat(newOutput()).contains("State: disabled"); | |
| } | |
| } finally { | |
| executor.shutdownNow(); | |
| } | |
| // Test disable Rapid Cache. The client's default LRO polling timeout is 5 minutes, whereas full | |
| // cache removal can take over an hour, though the cache enters the "disabled" state within | |
| // seconds. | |
| try { | |
| RapidCacheDisable.rapidCacheDisable(bucketName, cacheId); | |
| cacheDisabled = true; | |
| assertThat(newOutput()).contains("Disabled Rapid Cache: " + cacheName); | |
| } catch (ExecutionException e) { | |
| if (!(e.getCause() instanceof CancellationException)) { | |
| throw e; | |
| } | |
| cacheDisabled = true; | |
| newOutput(); | |
| RapidCacheGet.rapidCacheGet(bucketName, cacheId); | |
| assertThat(newOutput()).contains("State: disabled"); | |
| } |
| private String bucketName; | ||
| // The Rapid Cache ID must be the same as the zone. | ||
| private final String cacheId = ZONE.getZone(); | ||
| private boolean cacheDisabled; |
There was a problem hiding this comment.
nit: Boolean variables should use positive names to avoid inverted logic like !cacheDisabled as per go/java-practices/naming#positive-booleans and go/tott-387
| private boolean cacheDisabled; | |
| private boolean cacheActive; |
Adds Storage Control Rapid Cache samples for Java:
RapidCacheCreatestorage_control_create_rapid_cacheRapidCacheGetstorage_control_get_rapid_cacheRapidCacheListstorage_control_list_rapid_cachesRapidCacheUpdatestorage_control_update_rapid_cacheRapidCacheDisablestorage_control_disable_rapid_cacheNotes:
cache_type = "rapid-cache-ultra"and does not set a name; the server assigns the cache ID (currently equal to the zone).ttl = 172800swith update mask["ttl"].disabledstate within seconds, but the operation only completes once the cache is fully removed (can take more than 30 minutes).libraries-bom26.78.0 → 26.90.0 (google-cloud-storage-control 2.75.0, which includes RapidCache), and the matching google-cloud-storage versions (tests jar andinstall-without-bom) 2.64.0 → 2.75.0.Testing:
ITRapidCacheTestruns create → get → list → update, then disable infinally, and polls get until the state isdisabled. It uses a regional HNS + UBLA bucket in the zone's region. Zone defaults tous-central1-aand can be overridden withRAPID_CACHE_ZONE.ITAnywhereCacheTest, the test is skipped in CI (JOB_TYPEset) because of the long operation times. It passed when run locally against an allowlisted project.b/482946642
[Generated-by: AI]
Update: aligned with #14529
The samples now match the API usage in the reference integration test (
ITRapidCacheTestin #14529). Where this section differs from the notes above, this section is current.Samples:
RapidCacheCreate: sets the name explicitly toprojects/_/buckets/{bucket}/rapidCaches/{zone}(the backend requires the cache ID to equal the zone), pluszone,cache_type = "rapid-cache-ultra"andttl = 86400s(24h). Prints the name, cache type and state.RapidCacheGet: also prints the cache type.RapidCacheDisable: now waits for the LRO (disableRapidCacheAsync(request).get()) and prints the result, the same as the reference. A comment explains that the cache is disabled within seconds, but full removal can take longer than the client's default polling timeout (5 minutes).Test (
ITRapidCacheTest):State: runningand the cache type. List: checks that the cache is listed. Update: checksTTL: 172800 seconds.State: disabledand does not fail.JOB_TYPE) and zone default/override are unchanged.