Skip to content

Conversation

@mzumsande
Copy link
Contributor

In c711ca1 logic was introduced that -reindex and -reindex-chainstate will delete the snapshot chainstate.
This doesn't work currently, instead of deleting the snapshot chainstate the node crashes with an assert (this can be triggered by applying the added test commit on master).
Fix this, and another bug that would prevent the new active chainstate from having a mempool after -reindex has deleted the snapshot (also covered by the test).

This didn't work for two reasons:
1.) GetSnapshotCoinsDBPath() was used to retrieve the path.
    This requires coins_views to exist, but the initialisation only happens later
    (in CompleteChainstateInitialization) so the node hits an assert in
    CCoinsViewDB& CoinsDB() and crashes.

2.) The snapshot was already activated, so it has the mempool attached.
    Therefore, the mempool needs to be transferred back to the ibd
    chainstate before deleting the snapshot chainstate.
@DrahtBot
Copy link
Contributor

DrahtBot commented Mar 25, 2024

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK fjahr, BrandonOdiwuor, hernanmarino, byaye

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

@fjahr fjahr mentioned this pull request Mar 25, 2024
32 tasks
Copy link
Contributor

@BrandonOdiwuor BrandonOdiwuor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK

Copy link
Contributor

@fjahr fjahr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tACK cfdef5c

Good catch! I could reproduce the crash and that the change here fixes this.

My suggested test extension can be pulled in here if you retouch or I can add it to one of my outstanding assumeutxo PRs as a small follow-up.

assert_equal(n1.getblockchaininfo()["blocks"], SNAPSHOT_BASE_HEIGHT)

self.log.info(f"Check that restarting with -reindex will delete the snapshot chainstate")
self.restart_node(1, extra_args=['-reindex=1', *self.extra_args[1]])
Copy link
Contributor

@fjahr fjahr Mar 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only covers -reindex but not -reindex-chainstate. It's easily possible when using node 2 for the test instead which is not pruned: fjahr@38c5d80

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I took your suggestion (with a minor change to logging).

@DrahtBot DrahtBot requested a review from BrandonOdiwuor March 27, 2024 17:31
Copy link
Contributor

@BrandonOdiwuor BrandonOdiwuor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

crACK cfdef5c

Co-authored-by: Fabian Jahr <fjahr@protonmail.com>
@mzumsande mzumsande force-pushed the 202403_assumeutxo_reindex_fix branch from cfdef5c to b7ba60f Compare March 28, 2024 17:24
@mzumsande
Copy link
Contributor Author

cfdef5c to b7ba60f: Included test coverage for -reindex-chainstate as suggested by @fjahr

@fjahr
Copy link
Contributor

fjahr commented Mar 29, 2024

re-ACK b7ba60f

@DrahtBot DrahtBot requested a review from BrandonOdiwuor March 29, 2024 18:47
Copy link
Contributor

@BrandonOdiwuor BrandonOdiwuor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re-ACK b7ba60f

@hernanmarino
Copy link
Contributor

crACK b7ba60f . Good fix

Copy link

@byaye byaye left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested ACK b7ba60f

Test result before the fix
Screenshot 2024-04-12 at 17 34 14

Test result after the fix
Screenshot 2024-04-12 at 17 21 48

@ryanofsky ryanofsky merged commit 312f542 into bitcoin:master Apr 16, 2024
@mzumsande mzumsande deleted the 202403_assumeutxo_reindex_fix branch April 16, 2024 18:13
luke-jr pushed a commit to bitcoinknots/bitcoin that referenced this pull request Apr 24, 2024
This didn't work for two reasons:
1.) GetSnapshotCoinsDBPath() was used to retrieve the path.
    This requires coins_views to exist, but the initialisation only happens later
    (in CompleteChainstateInitialization) so the node hits an assert in
    CCoinsViewDB& CoinsDB() and crashes.

2.) The snapshot was already activated, so it has the mempool attached.
    Therefore, the mempool needs to be transferred back to the ibd
    chainstate before deleting the snapshot chainstate.

Github-Pull: bitcoin#29726
Rebased-From: e57f951
luke-jr pushed a commit to bitcoinknots/bitcoin that referenced this pull request Apr 24, 2024
Co-authored-by: Fabian Jahr <fjahr@protonmail.com>

Github-Pull: bitcoin#29726
Rebased-From: b7ba60f
luke-jr pushed a commit to bitcoinknots/bitcoin that referenced this pull request Jun 13, 2024
This didn't work for two reasons:
1.) GetSnapshotCoinsDBPath() was used to retrieve the path.
    This requires coins_views to exist, but the initialisation only happens later
    (in CompleteChainstateInitialization) so the node hits an assert in
    CCoinsViewDB& CoinsDB() and crashes.

2.) The snapshot was already activated, so it has the mempool attached.
    Therefore, the mempool needs to be transferred back to the ibd
    chainstate before deleting the snapshot chainstate.

Github-Pull: bitcoin#29726
Rebased-From: e57f951
luke-jr pushed a commit to bitcoinknots/bitcoin that referenced this pull request Jun 13, 2024
Co-authored-by: Fabian Jahr <fjahr@protonmail.com>

Github-Pull: bitcoin#29726
Rebased-From: b7ba60f
Fabcien pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Apr 7, 2025
Summary:
> validation: assumeutxo: swap m_mempool on snapshot activation
>
> Otherwise we will not receive transactions during background sync until restart.
bitcoin/bitcoin@9511fb3

And swap it back before deleting the snapshot chainstate:
> init, validation: Fix -reindex option with an existing snapshot
>
> This didn't work for two reasons:
> 1.) GetSnapshotCoinsDBPath() was used to retrieve the path.
>     This requires coins_views to exist, but the initialisation only happens later
>     (in CompleteChainstateInitialization) so the node hits an assert in
>     CCoinsViewDB& CoinsDB() and crashes.
>
> 2.) The snapshot was already activated, so it has the mempool attached.
>     Therefore, the mempool needs to be transferred back to the ibd
>     chainstate before deleting the snapshot chainstate.
bitcoin/bitcoin@e57f951

A functional test for [[bitcoin/bitcoin#29726 | core#29726]] is added later, when we can test assumeutxo (requires loadtxoutset)

This is a partial backport of [[bitcoin/bitcoin#27596 | core#27596]] and [[bitcoin/bitcoin#29726 | core#29726]]

Depends on D17896

Test Plan:
`ninja all check-all` (on my dev branch which has the functional test)

Run a full assumeutxo IBD (on my dev branch)

Reviewers: #bitcoin_abc, Fabien

Reviewed By: #bitcoin_abc, Fabien

Differential Revision: https://reviews.bitcoinabc.org/D17897
@bitcoin bitcoin locked and limited conversation to collaborators Apr 16, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

7 participants