help: enrich help text for `-loadblock` #33343

HowHsu · 2025-09-08T17:41:11Z

-loadblock doesn't support XOR-ed files, mention it in its help text to avoid troubles for users.

DrahtBot · 2025-09-08T17:41:17Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/33343.

Reviews

See the guideline for information on the review process.
A summary of reviews will appear here.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#17783 (common: Disallow calling IsArgSet() on ALLOW_LIST options by ryanofsky)
#17581 (refactor: Remove settings merge reverse precedence code by ryanofsky)
#17580 (refactor: Add ALLOW_LIST flags and enforce usage in CheckArgFlags by ryanofsky)
#17493 (util: Forbid ambiguous multiple assignments in config file by ryanofsky)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

LLM Linter (✨ experimental)

Possible typos and grammar issues:

"Imports blocks from external file" -> "Imports blocks from an external file" [missing article "an" makes the phrase ungrammatical and may hinder comprehension]

^{drahtbot_id_5_m}

l0rinc

I'm not sure it's a documentation issue, I think we should rather fix it by adding the obfuscation key to the AutoFile instead.
I will investigate if adding a fix and updating feature_loadblock.py makes more sense and will push an alternative PR to see what people think is more appropriate here.
Thanks for jumping on this so quickly.

src/init.cpp

l0rinc · 2025-09-08T22:59:17Z

I wasn't aware of this, it seems we have a dedicated tool to https://github.com/bitcoin/bitcoin/tree/master/contrib/linearize.
It's explicitly written to deobfuscate blocks from your node’s blk*.dat (handling XOR if present) and writes them plain to output_file or to an output/blkNNNNN.dat.

HowHsu · 2025-09-09T03:56:47Z

I'm not sure it's a documentation issue, I think we should rather fix it by adding the obfuscation key to the AutoFile instead. I will investigate if adding a fix and updating feature_loadblock.py makes more sense and will push an alternative PR to see what people think is more appropriate here. Thanks for jumping on this so quickly.

Hi l0rinc,
I don't think it's good to either

update the Autofile code to use the current obfuscation key by the running node
this makes it too restrictive.
add a new argument like -loadblock-xor=xor_value to indicate the xor key used by -loadblock
since we can restore the obfuscated file with some external tools, it's already very flexible, adding -loadblock-xor would be kind of meaningless.
That's why I only updated the help message.

l0rinc · 2025-09-09T04:32:11Z

I wrote that before I found the contrib/linearize tool, which seems to be designed for unobfuscated block sharing anyway - maybe it can solve your problem of using -loadblock. If it does, we might want to mention that instead in the argument's documentation.

mzumsande · 2025-09-10T15:06:03Z

I wrote that before I found the contrib/linearize tool, which seems to be designed for unobfuscated block sharing anyway

It was originally designed to to something else (create a linear, in order chain), so using the tool has side effects (maybe the users doesn't want to lose historical forks for some reason). Could mention it as an example though.

I'm not convinced that changing the interface to support adding a xor-key per -loadblock arg (or one for all?) is really worth the effort, since I'm not sure people out there actually use this feature today, and, if so, to what end.

`-loadblock` doesn't support obfuscated blocks, mention it in its help text to avoid troubles for users. Signed-off-by: Hao Xu <hao.xu@linux.dev>

l0rinc

@HowHsu Did you try contrib/linearize yourself to export the .blk files for -loadblock? Did it solve your issue? I have suggested a simpler documentation, do you think that help others to use the linearize tool when they're in your situation?

l0rinc · 2025-09-15T18:30:17Z

src/init.cpp

    argsman.AddArg("-includeconf=<file>", "Specify additional configuration file, relative to the -datadir path (only useable from configuration file, not command line)", ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
    argsman.AddArg("-allowignoredconf", strprintf("For backwards compatibility, treat an unused %s file in the datadir as a warning, not an error.", BITCOIN_CONF_FILENAME), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
-    argsman.AddArg("-loadblock=<file>", "Imports blocks from external file on startup", ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
+    argsman.AddArg("-loadblock=<file>", "Imports blocks from external file on startup (does not support obfuscated blocks, see Issue #33280 for related conversation, and see contrib/linearize tool for clues of deobfuscation and reobfuscation)", ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);


Suggested change

argsman.AddArg("-loadblock=<file>", "Imports blocks from external file on startup (does not support obfuscated blocks, see Issue #33280 for related conversation, and see contrib/linearize tool for clues of deobfuscation and reobfuscation)", ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);

argsman.AddArg("-loadblock=<file>", "Imports blocks from external file on startup (obfuscated block files are not supported; use contrib/linearize to export blocks to a plain, loadable format)", ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);

HowHsu · 2025-09-16T13:19:53Z

@HowHsu Did you try contrib/linearize yourself to export the .blk files for -loadblock? Did it solve your issue? I have suggested a simpler documentation, do you think that help others to use the linearize tool when they're in your situation?

I didn't, but I read the code of them (files under contrib/linearize), linearize-data.py only generates a list of blocks based on a list of block hashes generated by linearize-hashes.py, which accepts a height range [min_height, max_height] and get block hashes of these heights from RPC. That is said, the blocks are not arbitrary, if a user use -loadblock to load some discontinuous blocks, it has to generate its own block hashes list to feed linearize-data.py. So I just add for clues of deobfuscation and reobfuscation. For my scenario, I've found another way to do the test, so I didn't continue it with -loadblock

l0rinc · 2025-09-16T21:12:48Z

I think we should try it ourselves before recommending it in the documentation

luke-jr · 2025-09-18T05:29:49Z

I think it would make sense to support XOR for loadblock if we're assuming the chain has data that can trigger bad things

maflcko · 2025-09-25T13:55:49Z

I think we should try it ourselves before recommending it in the documentation

It is already tested in test/functional/feature_loadblock.py, no?

HowHsu · 2025-09-25T15:42:47Z

I think it would make sense to support XOR for loadblock if we're assuming the chain has data that can trigger bad things

Hi Luke, what do you mean by "assuming the chain has data that can trigger bad things"

HowHsu · 2025-09-25T15:44:59Z

I think we should try it ourselves before recommending it in the documentation

I'll try that later when I have some time for it, though the function of the script is already clear enough for me by carefully reading the code.

l0rinc · 2025-09-25T16:17:47Z

It is already tested in test/functional/feature_loadblock.py, no?

That's where I found it, but to see if it solves the original problem for why this PR was opened, it would help to have personal experience with the script so that we can document it such that others don't need to browse the tests. I'm also fine with anyone else having experience with the script suggesting a useful help text.

HowHsu mentioned this pull request Sep 8, 2025

use -loadblock to load blk*****.dat files, but the blocks in it are not recognized #33280

Open

1 task

l0rinc reviewed Sep 8, 2025

View reviewed changes

src/init.cpp Outdated Show resolved Hide resolved

HowHsu force-pushed the loadblock-help branch 3 times, most recently from d96fac4 to c1789d4 Compare September 13, 2025 14:30

HowHsu requested a review from l0rinc September 13, 2025 18:12

help: enrich help text for -loadblock

d787e55

`-loadblock` doesn't support obfuscated blocks, mention it in its help text to avoid troubles for users. Signed-off-by: Hao Xu <hao.xu@linux.dev>

HowHsu force-pushed the loadblock-help branch from c1789d4 to d787e55 Compare September 15, 2025 13:13

l0rinc suggested changes Sep 15, 2025

View reviewed changes

HowHsu requested a review from l0rinc September 16, 2025 13:41

DrahtBot mentioned this pull request Oct 15, 2025

scripted-diff: Type-safe settings retrieval #31260

Open

	argsman.AddArg("-loadblock=<file>", "Imports blocks from external file on startup (does not support obfuscated blocks, see Issue #33280 for related conversation, and see contrib/linearize tool for clues of deobfuscation and reobfuscation)", ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
	argsman.AddArg("-loadblock=<file>", "Imports blocks from external file on startup (obfuscated block files are not supported; use contrib/linearize to export blocks to a plain, loadable format)", ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);

help: enrich help text for -loadblock #33343

Are you sure you want to change the base?

help: enrich help text for -loadblock #33343

Conversation

HowHsu commented Sep 8, 2025

Uh oh!

DrahtBot commented Sep 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage & Benchmarks

Reviews

Conflicts

LLM Linter (✨ experimental)

Uh oh!

l0rinc left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

l0rinc commented Sep 8, 2025

Uh oh!

HowHsu commented Sep 9, 2025

Uh oh!

l0rinc commented Sep 9, 2025

Uh oh!

mzumsande commented Sep 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

l0rinc left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

l0rinc Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

HowHsu commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

l0rinc commented Sep 16, 2025

Uh oh!

luke-jr commented Sep 18, 2025

Uh oh!

maflcko commented Sep 25, 2025

Uh oh!

HowHsu commented Sep 25, 2025

Uh oh!

HowHsu commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

l0rinc commented Sep 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

help: enrich help text for `-loadblock` #33343

help: enrich help text for `-loadblock` #33343

DrahtBot commented Sep 8, 2025 •

edited

Loading

mzumsande commented Sep 10, 2025 •

edited

Loading

l0rinc left a comment •

edited

Loading

HowHsu commented Sep 16, 2025 •

edited

Loading

HowHsu commented Sep 25, 2025 •

edited

Loading