Exactly-Once Semantics for golem:rdbms#1741
Conversation
# Conflicts: # golem-worker-executor/tests/common/mod.rs
|
From your TODO list, don't worry about backward compatibility. We decided to break many things in the 1.3 release so it won't be compatible with any older worker anyway. |
# Conflicts: # cli/golem-cli/src/model/text/worker.rs # golem-api-grpc/proto/golem/worker/public_oplog.proto # golem-common/src/base_model.rs # golem-common/src/model/oplog.rs # golem-common/src/model/public_oplog/mod.rs # golem-common/src/model/public_oplog/protobuf.rs # golem-test-framework/src/dsl/debug_render.rs # golem-worker-executor/src/durable_host/mod.rs # golem-worker-executor/src/model/public_oplog/mod.rs # golem-worker-executor/src/model/public_oplog/wit.rs # golem-worker-executor/src/services/oplog/tests.rs # golem-worker-executor/src/worker/status.rs # golem-worker-executor/tests/common/mod.rs # openapi/golem-service.yaml # openapi/golem-worker-service.yaml # test-components/rdbms-service.wasm
That kind of callback interface is not really possible with these WIT interfaces (I mean what you write is technically probably possible but very inconvenient). Instead the version of this exposed as host function should be refactored similarly to what I did with I don't want this to be implemented before this PR is merged though. Because we are going to break compatibility at least once more in 1.4 according to the current plans, I am giving this another review now with the intent to have it in 1.3 as we planned. (And modify it later if needed) |
fixes: #1514
/claim #1514
new oplog entries:
BeginRemoteTransaction- Begins a transaction operation - entry is added to oplog after transaction beginPreCommitRemoteTransaction- Pre-Commit of the transaction, indicating that the transaction will be committed - entry is added to oplog before transaction commit is executedPreRollbackRemoteTransaction- Pre-Rollback of the transaction, indicating that the transaction will be rolled back - entry is added to oplog before transaction rollback is executedCommittedRemoteTransaction- Committed transaction operation, indicating that the transaction was committed - entry is added to oplog after successful transaction commitRolledBackRemoteTransaction- Rolled back transaction operation, indicating that the transaction was rolled back - entry is added to oplog after successful transaction rollbackBeginRemoteTransactionis first oplog entry related to DB transactionPreCommitRemoteTransactionandCommittedRemoteTransactionentries will be around DB commitPreRollbackRemoteTransactionandRolledBackRemoteTransactionentries will be around DB rollbackCommittedRemoteTransactionorRolledBackRemoteTransactionoplog entry is last oplog entry related to transactionTODO:
TestOplogwhere append of oplog entries failing based on patterns in worker name. Oplog functions do not haveResultas return type, in case of error, it just ends with panic, (PrimaryOplog), in this PR new oplog functions were added, where add of entry have in responseResult type- discuss Oplog function withResultas return type, find better way how to control when specific action should failoplog snippet
[ { "type": "BeginRemoteTransaction", "timestamp": "2025-06-04T17:51:36.626Z", "transactionId": "740" }, { "type": "ImportedFunctionInvoked", "timestamp": "2025-06-04T17:51:36.638Z", "functionName": "rdbms::postgres::db-transaction::execute", "request": { ... }, "response": {... }, "wrappedFunctionType": { "type": "WriteRemoteTransaction", "index": 16 } }, { "type": "Error", "timestamp": "2025-06-04T17:51:36.662Z", "error": "error while executing at wasm backtrace:\n 0: 0x3fc08 - wit-component:shim!indirect-golem:rdbms/postgres@0.0.1-[method]db-transaction.commit\n 1: 0x2a6a1 - <unknown>!<wasm function 231>\n 2: 0x2af54 - <unknown>!<wasm function 238>: Runtime error: worker rdbms-service-postgres-763b5c94-2128-4fca-b130-682ffaa8ec10-Fail1OnPreCommitRemoteTransaction failed on PreCommitRemoteTransaction 1 times" }, { "type": "Jump", "timestamp": "2025-06-04T17:51:36.842Z", "jump": { "start": 16, "end": 21 } }, { "type": "BeginRemoteTransaction", "timestamp": "2025-06-04T17:51:36.848Z", "transactionId": "744" }, { "type": "ImportedFunctionInvoked", "timestamp": "2025-06-04T17:51:36.857Z", "functionName": "rdbms::postgres::db-transaction::execute", "request": {... }, "response": {... }, "wrappedFunctionType": { "type": "WriteRemoteTransaction", "index": 22 } }, { "type": "PreCommitRemoteTransaction", "timestamp": "2025-06-04T17:51:36.868Z", "beginIndex": 22 }, { "type": "CommitedRemoteTransaction", "timestamp": "2025-06-04T17:51:36.888Z", "beginIndex": 22 } ]where transaction failed on commit, on appending
PreCommitRemoteTransactionentry to oplog, which means transaction was not commited, and was restarted