I noticed this while working with the integration tests. @SpringBootTest + @Testcontainers. Some builds on CI were failing with timeout exception when trying to do repository.save() and CB error was
om.couchbase.client.core.error.AmbiguousTimeoutException: UpsertRequest, Reason: TIMEOUT
{"cancelled":true,"completed":true,"coreId":"0x3b57a1700000003","idempotent":false
,"reason":"TIMEOUT","requestId":161,"requestType":"UpsertRequest","retried":19
,"retryReasons":["BUCKET_NOT_AVAILABLE"], ...
I've added buckets printout right before save operation like this:
var couchbaseClientFactory = context.getApplicationContext().getBean(CouchbaseClientFactory.class);
couchbaseClientFactory.getCluster().buckets().getAllBuckets()
.forEach((bucket, settings) -> log.info("Test Bucket: [{}]; Settings: [{}]; Healthy: [{}]", bucket, settings, settings.healthy()));
and it showed that the bucket is there and is healthy. But save() fails.
Following through the code from the client factory down to where the bucket opens, it looks like opening a bucket is an async operation that is not guaranteed to finish (or rather guaranteed not to finish) when the method exits:
SimpleCouchbaseClientFactory:
this.bucket = cluster.get().bucket(bucketName);
Cluster:
return bucketCache.computeIfAbsent(bucketName, n -> new Bucket(asyncCluster.bucket(n)));
AsyncCluster:
return bucketCache.computeIfAbsent(bucketName, n -> {
core.openBucket(n);
return new AsyncBucket(n, core, environment.get());
});
Core:
configurationProvider
.openBucket(name)
.subscribe(v -> {}, t -> {}, () -> eventBus.publish(new BucketOpenedEvent(Duration.ofNanos(System.nanoTime() - start), coreContext, name)));
.subscribe is async here so the client factory may very well get a bucket instance that has not been opened yet.
I tried couchbaseClientFactory.getBucket().waitUntilReady(...) but it does not seem to work here and I don't see any code in waitUntilReady() that would ensure that bucket is open.
A workaround I came up with is the following:
couchbaseClientFactory.getCluster().core().configurationProvider().openBucket(BUCKET_NAME).block(WAIT_TIME);
It's not very nice and I block on Mono but it works for tests where context starts in less than a second and I need to make sure bucket is available when test cases start executing.
It'd be good to ensure in SimpleCouchbaseClientFactory that the bucket is indeed opened when context starts.
I noticed this while working with the integration tests.
@SpringBootTest+@Testcontainers. Some builds on CI were failing with timeout exception when trying to dorepository.save()and CB error wasI've added buckets printout right before save operation like this:
and it showed that the bucket is there and is healthy. But
save()fails.Following through the code from the client factory down to where the bucket opens, it looks like opening a bucket is an async operation that is not guaranteed to finish (or rather guaranteed not to finish) when the method exits:
.subscribeis async here so the client factory may very well get a bucket instance that has not been opened yet.I tried
couchbaseClientFactory.getBucket().waitUntilReady(...)but it does not seem to work here and I don't see any code inwaitUntilReady()that would ensure that bucket is open.A workaround I came up with is the following:
It's not very nice and I block on
Monobut it works for tests where context starts in less than a second and I need to make sure bucket is available when test cases start executing.It'd be good to ensure in
SimpleCouchbaseClientFactorythat the bucket is indeed opened when context starts.