Skip to content

mining: add submitBlock to IPC Mining interface#34644

Open
w0xlt wants to merge 3 commits intobitcoin:masterfrom
w0xlt:ipc-submit-block
Open

mining: add submitBlock to IPC Mining interface#34644
w0xlt wants to merge 3 commits intobitcoin:masterfrom
w0xlt:ipc-submit-block

Conversation

@w0xlt
Copy link
Contributor

@w0xlt w0xlt commented Feb 21, 2026

This PR adds a submitBlock method to the IPC Mining interface, equivalent to the submitblock RPC. It accepts a serialized block over IPC, validates/processes it via the normal block-processing path.

The method uses the same result shape as checkBlock: bool + reason/debug out-params. It reports duplicate, inconclusive, and invalid-block rejection details, and initializes reason/debug on every call.

Closes #34626

@DrahtBot
Copy link
Contributor

DrahtBot commented Feb 21, 2026

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type Reviewers
Concept ACK Sjors, enirox001
Stale ACK optout21

If your review is incorrectly listed, please copy-paste <!--meta-tag:bot-skip--> into the comment that the bot should ignore.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #34727 (test: Add IPC wake-up test and reuse mining context by enirox001)
  • #34020 (mining: add getTransactions(ByWitnessID) IPC methods by Sjors)
  • #33922 (mining: add getMemoryLoad() and track template non-mempool memory footprint by Sjors)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

Copy link
Member

@Sjors Sjors left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK

But let's make it more similar to the submitSolution() IPC rather than the getblocktemplate RPC, in terms of behavior and implementation (which can be much smaller).

*
* Like the submitblock RPC, this may add the coinbase witness nonce if
* missing, if the block already has a witness commitment and segwit is
* active for the previous block (via UpdateUncommittedBlockStructures).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

7a2d88c: I would prefer to mimic the behavior of submitSolution() here, and not magically modify the block.

It was useful for backwards compatibility when SegWit was newly introduced, but since IPC is new and SegWit has been active for a decade, there's no need for it. And I find the behavior confusing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the UpdateUncommittedBlockStructures call - the block is now passed as std::make_shared<const CBlock>(block_in) directly (the const enforces this at the type level).

The doc comment now notes: "unlike the submitblock RPC, this method does NOT add the coinbase witness automatically."

Added a test that submits a block with a witness commitment but no coinbase witness nonce and confirms it's rejected with bad-witness-nonce-size.

NodeContext& m_node;
};

class SubmitBlockStateCatcher final : public CValidationInterface
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

7a2d88c: I'm surprised this is needed, given that submitSolution can do without.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

submitSolution returns just a bool (and passes nullptr for new_block), so it doesn't need to distinguish failure reasons.

submitBlock returns (reason, debug, result) and to populate reason and debug with the specific rejection string (e.g. bad-version(...), bad-witness-nonce-size), we need the BlockValidationState, which is only available through the BlockChecked validation interface callback.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I confused checkBlock with submitSolution.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, checkBlock calls TestBlockValidity() which returns BlockValidationState directly as its return value.

submitBlock calls ProcessNewBlock() which only returns a bool. The BlockValidationState with the specific rejection reason is only available asynchronously through the CValidationInterface::BlockChecked callback.


/**
* Process and relay a fully assembled block.
* Process a fully assembled block.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

b72108f nit: changes in the wrong commit?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Thanks.

balance = self.miniwallet.get_balance()
coinbase = await self.build_coinbase_test(template, ctx, self.miniwallet)
coinbase.vout[0].nValue = COIN
block.vtx[0] = coinbase
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

b72108f: let's add at least one real transaction to the block.

It's also useful, see my earlier comment, to test:

  1. a non-segwit block
  2. some invalid encodings, like having a segwit op_return output, but missing the coinbase witness

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Thanks.

self.miniwallet.rescan_utxos()
assert_equal(self.miniwallet.get_balance(), balance + 1)

self.log.debug("submitBlock duplicate should fail")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

b72108f: it might be useful to add a test for the interaction between submitSolution and submitBlock(). In either order this should result in a duplicate.

Calling submitSolution after submitBlock() shouldn't result in a crash or missing object error, because we hold on to BlockTemplate as long as the client has a reference to it. The sv2-tp implementation waits 10 seconds after a new block before releasing stale templates.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added two interaction tests. Thanks.

return state.IsValid();
}

bool submitBlock(const CBlock& block_in, std::string& reason, std::string& debug) override
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once submitSolution reconstructs the CBlock it should be able to use the same code path as submitBlock.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's true in principle, but submitSolution currently returns a plain bool (and true for duplicates), while submitBlock returns (reason, debug, result) and false for duplicates.

Sharing the code path would change submitSolution's return semantics, which would be a breaking change for existing IPC clients. Might be worth doing as a follow-up if the IPC schema for submitSolution is updated to also return reason/debug strings.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding fields to .capnp is not a breaking change for clients. It makes sense to relay the failure reason.

I'm not saying that submitSolution should call submitBlock directly, it can be a common helper. So it shouldn't be necessary to change the submitSolution signature, that can be left to a followup.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the commit 9e12fb4 with the common helper.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the submitSolution changes in #34672.
Preferred to open a separate PR to keep the this one focused on submitBlock.

@optout21
Copy link
Contributor

optout21 commented Mar 3, 2026

utACK 9e12fb4

New method added to the mining IPC interface. New method, so no behavior change or compatibility issues. The implementation builds on submitSolution method and "submitblock" RPC, so it is rather small. Tests are added, including for typical error cases. Follow-up in #34672.

@DrahtBot DrahtBot requested a review from Sjors March 3, 2026 12:09
@DrahtBot DrahtBot mentioned this pull request Mar 4, 2026
Copy link
Contributor

@enirox001 enirox001 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK

I reviewed the code and ran some boundary tests. Left some comments below along with a couple of suggestions for test coverage and code cleanup

NodeContext& m_node;
};

class SubmitBlockStateCatcher final : public CValidationInterface
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "refactor: extract ProcessBlock helper for submitSolution and submitBlock" (9e12fb4):

SubmitBlockStateCatcher is essentially a 1:1 copy of submitblock_StateCatcher in src/rpc/mining.cpp. Since commit this commit already refactored how this is used locally, does it make sense to deduplicate it entirely and move it to a shared location so that it can be used in src/rpc/mining.cpp as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be done in a follow-up PR. It's a refactor that increases scope. Moving the class to a shared header would touch src/rpc/mining.cpp which is unrelated to this PR's purpose (adding IPC submitBlock).

And also it is only 10 lines of boilerplate. Deduplicating it would require a new shared header, an include in both files, and coordination between the RPC and IPC layers that currently don't depend on each other. The "abstraction cost" exceeds the "duplication cost" for something this small.

self.sync_all()

asyncio.run(capnp.run(async_routine()))

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit suggestion:

In commit "test: add IPC submitBlock functional test" (354538d):

The current tests check what happens when you submit the same valid block twice (returning "duplicate"). It would be great to also test what happens when you submit the same invalid block twice.

The IPC test should return "duplicate-invalid" the second time, proving that the node correctly remembered the block was bad and rejected it.

Here is a quick test snippet you can drop in to cover this

index 032a696353..f5cc9c3d56 100755
--- a/test/functional/interface_ipc_mining.py
+++ b/test/functional/interface_ipc_mining.py
@@ -530,6 +530,43 @@ class IPCMiningTest(BitcoinTestFramework):
 
         asyncio.run(capnp.run(async_routine()))
 
+    def run_submit_block_duplicate_invalid_test(self):
+        """Test submitting the exact same invalid block twice to check StateCatcher behavior."""
+        self.log.info("Running submitBlock duplicate-invalid test")
+
+        async def async_routine():
+            ctx, mining = await self.make_mining_ctx()
+
+            async with destroying((await mining.createNewBlock(ctx, self.default_block_create_options)).result, ctx) as
 template:
+                block = await mining_get_block(template, ctx)
+                coinbase = await self.build_coinbase_test(template, ctx, self.miniwallet)
+                coinbase.vout[0].nValue = COIN
+                block.vtx[0] = coinbase
+                block.hashMerkleRoot = block.calc_merkle_root()
+
+                # Make the block definitively invalid (bad locktime)
+                block.vtx[0].nLockTime = 2**32 - 1
+                block.hashMerkleRoot = block.calc_merkle_root()
+                block.solve()
+                serialized_block = block.serialize()
+
+                self.log.debug("Submit invalid block first time")
+                result1 = await mining.submitBlock(ctx, serialized_block)
+                assert_equal(result1.result, False)
+                assert_equal(result1.reason, "bad-txns-nonfinal")
+
+
+                self.log.info(f"First submission reason was: '{result1.reason}'")
+
+                self.log.debug("Submit the exact same invalid block a second time")
+                result2 = await mining.submitBlock(ctx, serialized_block)
+                assert_equal(result2.result, False)
+
+                self.log.info(f"Second submission reason was: '{result2.reason}'")
+                assert_equal(result2.reason, "duplicate-invalid")
+
+        asyncio.run(capnp.run(async_routine()))
+
     def run_test(self):
         self.miniwallet = MiniWallet(self.nodes[0])
         self.default_block_create_options = self.capnp_modules['mining'].BlockCreateOptions()
@@ -540,6 +577,7 @@ class IPCMiningTest(BitcoinTestFramework):
         self.run_submit_block_witness_test()
         self.run_submit_block_then_solution_test()
         self.run_submit_solution_then_block_test()
+        self.run_submit_block_duplicate_invalid_test()
         self.run_ipc_option_override_test() 

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Included the test. Thanks.

self.sync_all()

asyncio.run(capnp.run(async_routine()))

Copy link
Contributor

@enirox001 enirox001 Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "test: add IPC submitBlock functional test" (354538d):

I was testing the boundary conditions of the new IPC interface and noticed that sending malformed or truncated block data causes the entire node to crash.

Unlike the standard submitblock RPC which safely catches deserialization errors and returns a -22 Block decode failed error. The IPC implementation seems to let the deserialization exception bubble up, resulting in a SIGABRT (exit code -6).

Since a sv2 client could potentially send bad wire data, this looks like a DoS vector that should be caught at the boundary.

You can reproduce the crash by adding this test to test/functional/interface_ipc_mining.py:

def run_submit_block_malformed_test(self):
        """Test that submitting truncated/garbage bytes does not crash the node."""
        self.log.info("Running submitBlock malformed data test")

        async def async_routine():
            ctx, mining = await self.make_mining_ctx()

            async with destroying((await mining.createNewBlock(ctx, self.default_block_create_options)).result, ctx) as template:
                block = await mining_get_block(template, ctx)
                
                # Truncate the last 15 bytes to create an invalid serialization
                bad_bytes = block.serialize()[:-15]

                self.log.debug("Submitting truncated block bytes via IPC")
                
                try:
                    result = await mining.submitBlock(ctx, bad_bytes)
                    assert_equal(result.result, False)
                except Exception as e:
                    self.log.info(f"IPC exception caught in test: {e}")
                
                # The node should handle the bad stream gracefully and remain alive
                assert_equal(self.nodes[0].is_node_stopped(), False)

        asyncio.run(capnp.run(async_routine()))

Running this results in the peer disconnecting and the node crashing:

AssertionError: [node 0] Node returned unexpected exit code (-6) vs ((0,)) when stopping
capnp.lib.capnp.KjException: capnp/rpc.c++:2778: disconnected: Peer disconnected.

I suspect the C++ handler unpacking the Data blob into a CBlock needs to be wrapped in a try...catch block to handle the std::ios_base::failure and return a safe rejection reason (e.g., "inconclusive" or a specific decode error), similar to how src/rpc/mining.cpp handles it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran the test but I wasn't able to reproduce the error you mentioned.

As I understand it, Proxy.Context already handles it (via kj::runCatchingExceptions()), catching std::exception and sends it back to the client as kj::Exception rather than crashing the node.

I added the suggested functional test for this (truncated block bytes via submitBlock, asserting the node stays alive) so we can verify this on CI. Could you share which platform/configuration you tested on and which capnp version you were using? Is it possible there's a platform-specific edge case ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is my environment

  • OS: Fedora Linux 43
  • Cap'n Proto: 1.0.1
  • Build: cmake -B build -DENABLE_IPC=ON -DCapnProto_DIR="/usr/lib64/cmake/CapnProto"

The kj::runCatchingExceptions() safety net might be failing locally because Fedora's capnproto package could be compiled with flags that alter standard exception handling. This could cause it to bypass the catch block and crash.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't find any information about flags that alter standard exception handling. Standard C++ try/catch for std::exception is fundamental. Distro packages don't break this.

Even if Cap'n Proto were built with -fno-exceptions, it would use a completely different error path.

The information provided so far are vague and insufficient to reproduce the error.

Anyway, what you are reporting - assuming it is reproducible - affects the IPC as overall (it would affect checkBlock and any other IPC method that deserializes Data → C++ types, not just submitBlock) and should be handled in a specific PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

​You make a fair point. If this crash also happens when sending bad data to checkBlock or other IPC methods, then it is a broader issue with the IPC layer itself. It definitely shouldn't hold up this specific PR.

@Sjors
Copy link
Member

Sjors commented Mar 5, 2026

I opened 2140-dev/bitcoin-capnp-types#12 for easier testing from Rust, cc @plebhash.

It would be good to rebase this now that #34422 landed, to prevent confusion while testing from Rust.

Sjors added a commit to Sjors/bitcoin-capnp-types that referenced this pull request Mar 5, 2026
w0xlt added 3 commits March 5, 2026 11:18
Add a submitBlock method to the Mining IPC interface, similar to the
submitblock RPC. This accepts a fully assembled block, validates it, and
if accepted as new, processes it into chainstate.

Unlike the submitblock RPC, this method does NOT auto-add the coinbase
witness nonce (via UpdateUncommittedBlockStructures). Since this is a new
interface and SegWit has been active for years, callers must provide a
fully-formed block including the coinbase witness nonce when a witness
commitment is present.

This is needed for Stratum v2 Job Declarator Server (JDS), where accepted
solutions may correspond to jobs not tied to a Bitcoin Core BlockTemplate.
JDS receives PushSolution fields and reconstructs full blocks; without an
IPC submitBlock method, final submission requires the submitblock RPC.

The method returns detailed status (reason/debug strings) matching the
checkBlock pattern, giving callers enough information to handle
validation failures.
Test the new Mining.submitBlock IPC method:
- Invalid block (bad version) returns failure with reason
- Valid block (with a real mempool tx) is accepted and propagates
- Duplicate block returns failure with "duplicate" reason
- Witness commitment without coinbase witness nonce is rejected
  (bad-witness-nonce-size), confirming no auto-fix behavior
- submitBlock then submitSolution: duplicate is accepted (submitSolution
  returns true for already-known blocks)
- submitSolution then submitBlock interaction (duplicate)

The test bootstraps a candidate block with createNewBlock(), mutates and
serializes it, then submits it through Mining.submitBlock instead of
BlockTemplate.submitSolution. This exercises the complete-block submission
path and approximates the Sv2 JDS use case.
Move SubmitBlockStateCatcher and the ProcessNewBlock + validation
interface registration logic into a shared ProcessBlock() helper.
Both submitSolution and submitBlock now use this common code path.

No behavior change.
@w0xlt w0xlt force-pushed the ipc-submit-block branch from 9e12fb4 to 6cc8353 Compare March 5, 2026 20:22
@w0xlt
Copy link
Contributor Author

w0xlt commented Mar 5, 2026

It would be good to rebase this

Done. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Sv2 needs a submitBlock IPC method

5 participants