Skip to content

CXXCBC-462: Fix hanging when specifying a custom metadata collection via the public API & expose errors#532

Merged
DemetrisChr merged 4 commits intocouchbase:mainfrom
DemetrisChr:CXXCBC-462-connect-custom-metadata
Mar 5, 2024
Merged

CXXCBC-462: Fix hanging when specifying a custom metadata collection via the public API & expose errors#532
DemetrisChr merged 4 commits intocouchbase:mainfrom
DemetrisChr:CXXCBC-462-connect-custom-metadata

Conversation

@DemetrisChr
Copy link
Copy Markdown
Contributor

@DemetrisChr DemetrisChr commented Mar 4, 2024

Motivation

Specifying a custom metadata collection when calling cluster::connect results in the SDK to hang as the constructor of core::transactions::transactions, which is called in the callback of core.open (running on an IO thread), attempts to open the bucket of the custom metadata (requires an available IO thread) and blocks until that is completed (which means that it does not release its IO thread to be used by open_bucket). In addition, the constructor throws exceptions when an error occurs in the constructor which are not being handled by either the Public API or the wrappers.

Changes

  • Add a core::transactions::transactions::create method to construct the core transactions object, which takes a callback and can be chained with other asynchronous operations. It also returns an appropriate error code via the callback when an error occurs.
  • In cluster::connect, handle any errors that occur during transactions::create by closing the cluster and surfacing the error via the cluster_connect_handler
  • Added test to ensure that cluster::connect returns bucket_not_found when the specified metadata collection's bucket does not exist

Results

  • Test passes
  • FIT tests which require a custom metadata collection (e.g. CleanUpInMultipleBucketsTest) no longer hang when the performer runs with 1 IO thread

@DemetrisChr DemetrisChr requested review from avsej and thejcfactor March 4, 2024 13:03
Comment thread core/transactions.hxx Outdated
Comment thread core/transactions.hxx
@DemetrisChr DemetrisChr force-pushed the CXXCBC-462-connect-custom-metadata branch from dfaf04f to 3a0611c Compare March 4, 2024 15:19
avsej
avsej previously approved these changes Mar 5, 2024
@thejcfactor
Copy link
Copy Markdown
Contributor

This question is not related to the changes specific to this PR, but it is related. Why do we "implicitly" create the transactions object when we create a cluster object in the public API? I don't get the sense that is what we need to do in order to support ExtSDKIntegration (ref Transaction instantiation). To me, it would seem better to lazily create the transactions object when users call cluster::transactions(). Is there something I am missing?

@DemetrisChr
Copy link
Copy Markdown
Contributor Author

This question is not related to the changes specific to this PR, but it is related. Why do we "implicitly" create the transactions object when we create a cluster object in the public API? I don't get the sense that is what we need to do in order to support ExtSDKIntegration (ref Transaction instantiation). To me, it would seem better to lazily create the transactions object when users call cluster::transactions(). Is there something I am missing?

The spec doesn't specify this behaviour. It only says "The life-cycle of that Transactions object is now bound to the Cluster it's created from." and it appears to leave the details up to the implementation. However, I agree that it would be better to lazily create the transactions object to avoid spinning up all the transactions infrastructure for users who are not using transactions. A problem here is that transactions::create currently returns an error because of this bucket_open which would be inconvenient if it was returned via transactions() I think (since we return errors rather than throwing in the Public API).

I'll need to investigate further if we should actually be doing this bucket_open here or if we can just do the transactions create asynchronously and surface any errors elsewhere (e.g. on txn.run). If we end up doing this, we should be able to easily move the creation to transactions().

@DemetrisChr DemetrisChr requested a review from avsej March 5, 2024 15:10
@avsej
Copy link
Copy Markdown
Member

avsej commented Mar 5, 2024

To me, it would seem better to lazily create the transactions object when users call cluster::transactions(). Is there something I am missing?

I agree, this is what we do with buckets, scopes and collections. Lets capture this as a ticket and discuss it separately. Maybe it should go one step further, and not just lazily create transaction object, but defer running background tasks and opening a bucket until first transaction will be scheduled.

@DemetrisChr DemetrisChr merged commit e09c6c1 into couchbase:main Mar 5, 2024
@DemetrisChr DemetrisChr deleted the CXXCBC-462-connect-custom-metadata branch March 5, 2024 18:40
@DemetrisChr
Copy link
Copy Markdown
Contributor Author

To me, it would seem better to lazily create the transactions object when users call cluster::transactions(). Is there something I am missing?

I agree, this is what we do with buckets, scopes and collections. Lets capture this as a ticket and discuss it separately. Maybe it should go one step further, and not just lazily create transaction object, but defer running background tasks and opening a bucket until first transaction will be scheduled.

I've filed a ticket for this: CXXCBC-474

thejcfactor pushed a commit to thejcfactor/couchbase-cxx-client that referenced this pull request Mar 6, 2024
…via the public API & expose errors (couchbase#532)

Co-authored-by: Sergey Avseyev <sergey.avseyev@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants