qa: Make wallet_multiwallet.py Windows crossbuild-compatible#34418
qa: Make wallet_multiwallet.py Windows crossbuild-compatible#34418achow101 merged 10 commits intobitcoin:masterfrom
Conversation
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. Code Coverage & BenchmarksFor details see: https://corecheck.dev/bitcoin/bitcoin/pulls/34418. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please copy-paste ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
wallet_multiwallet.py Windows-compatible
hebasto
left a comment
There was a problem hiding this comment.
Concept ACK.
Thanks for taking this over!
w0xlt
left a comment
There was a problem hiding this comment.
Approach ACK. I’ll review it more closely soon.
There was a problem hiding this comment.
ACK 596752a
EDIT: I made the wrong judgement on the difference in loggin (seeing more tests) as enough proof that the PR is an improvement next to an code-review. Will re-vist this PR.
This PR makes wallet_multiwallet.py Windows-compatible. Tested it on windows 11, msvc build (vs 2022)
this PR
2026-02-06T10:25:48.016394Z TestFramework (INFO): PRNG seed is: 3924437959039469419
2026-02-06T10:25:48.049997Z TestFramework (INFO): Initializing test directory C:\Users\user\AppData\Local\Temp\bitcoin_func_test_nylizqfb
2026-02-06T10:25:48.550233Z TestFramework (WARNING): Skipping chmod+symlink checks on Windows: chmod works differently due to how access rights work and symlink behavior with regard to the standard library is non-standard on cross-built binaries.
2026-02-06T10:25:48.756512Z TestFramework (INFO): Test mixed wallets
2026-02-06T10:25:49.850237Z TestFramework (INFO): Test initialization
2026-02-06T10:25:52.819255Z TestFramework (INFO): Test balances and fees
2026-02-06T10:25:54.266821Z TestFramework (INFO): Check for per-wallet settxfee call
2026-02-06T10:25:54.270096Z TestFramework (INFO): Test dynamic wallet loading
2026-02-06T10:25:54.748885Z TestFramework (INFO): Load first wallet
2026-02-06T10:25:54.760269Z TestFramework (INFO): Load second wallet
2026-02-06T10:25:54.769925Z TestFramework (INFO): Concurrent wallet loading
2026-02-06T10:25:54.803331Z TestFramework (INFO): Load remaining wallets
2026-02-06T10:25:54.856230Z TestFramework (INFO): Test dynamic wallet creation
2026-02-06T10:25:54.929120Z TestFramework (INFO): Test dynamic wallet unloading
2026-02-06T10:25:56.604190Z TestFramework (INFO): Test wallet backup and restore
2026-02-06T10:25:57.908829Z TestFramework (INFO): Test wallet lock file is closed
2026-02-06T10:25:58.309178Z TestFramework (INFO): Stopping nodes
2026-02-06T10:25:58.464283Z TestFramework (INFO): Cleaning up C:\Users\user\AppData\Local\Temp\bitcoin_func_test_nylizqfb on exit
2026-02-06T10:25:58.464577Z TestFramework (INFO): Tests successfulMaster
2026-02-06T10:44:32.520149Z TestFramework (INFO): PRNG seed is: 9079423242423824445
2026-02-06T10:44:32.557824Z TestFramework (INFO): Initializing test directory C:\Users\user\AppData\Local\Temp\bitcoin_func_test_3ac39i27
2026-02-06T10:44:33.134825Z TestFramework (INFO): Verify warning is emitted when failing to scan the wallets directory
2026-02-06T10:44:33.135065Z TestFramework (WARNING): Skipping test involving chmod as Windows does not support it.
2026-02-06T10:44:38.490235Z TestFramework (INFO): Check for per-wallet settxfee call
2026-02-06T10:44:38.492665Z TestFramework (INFO): Test dynamic wallet loading
2026-02-06T10:44:38.978126Z TestFramework (INFO): Load first wallet
2026-02-06T10:44:38.990162Z TestFramework (INFO): Load second wallet
2026-02-06T10:44:38.999576Z TestFramework (INFO): Concurrent wallet loading
2026-02-06T10:44:39.036463Z TestFramework (INFO): Load remaining wallets
2026-02-06T10:44:39.087449Z TestFramework (INFO): Test dynamic wallet creation.
2026-02-06T10:44:39.150340Z TestFramework (INFO): Test dynamic wallet unloading
2026-02-06T10:44:40.821876Z TestFramework (INFO): Test wallet backup
2026-02-06T10:44:42.673883Z TestFramework (INFO): Stopping nodes
2026-02-06T10:44:42.828266Z TestFramework (INFO): Cleaning up C:\Users\user\AppData\Local\Temp\bitcoin_func_test_3ac39i27 on exit
2026-02-06T10:44:42.828464Z TestFramework (INFO): Tests successfulTest also still works on nix-shell/aarch64
| if platform.system() == 'Windows': | ||
| # Additional context: | ||
| # - chmod: Posix has one user per file while Windows has an ACL approach | ||
| # - symlinks: GCC 13 has FIXME notes for symlinks under Windows: |
There was a problem hiding this comment.
GCC 13 has FIXME notes for symlinks under Windows:
I wonder if this will change. The FIXME has existed since 2018, when std::filesystem support for Windows was first added: gcc-mirror/gcc@9534a5e.
There was a problem hiding this comment.
Updated the comment to document that it's still the case in latest release instead:
bitcoin/test/functional/wallet_multiwallet.py
Lines 74 to 79 in 9a7b725
Hopefully that's slightly better for maintainability.
Also dropped the comment about chmod as I don't think it really added anything relevant to this test beyond what the warning below already states.
|
Thanks for your review @janb84! At first I was astonished that the test succeeds for you on master. Then I realized that this PR actually makes the test work for Linux->Windows cross builds. Will make it clearer in the title/PR description. |
596752a to
9a7b725
Compare
|
🚧 At least one of the CI tasks failed. HintsTry to run the tests locally, according to the documentation. However, a CI failure may still
Leave a comment here, if you need help tracking down a confusing failure. |
janb84
left a comment
There was a problem hiding this comment.
ACK 9a7b725
I revisited the PR, now using a cross compilation build. Using gcc, the master branch functional test will indeed fail to run:
Master
Temporary test directory at C:\Users\user\AppData\Local\Temp/test_runner_₿_🏃_20260209_120146
Remaining jobs: [wallet_multiwallet.py, wallet_multiwallet.py --usecli]
1/2 - wallet_multiwallet.py failed, Duration: 4 s
stdout:
2026-02-09T11:01:50.077850Z TestFramework (INFO): PRNG seed is: 4370893628585504645
2026-02-09T11:01:50.136334Z TestFramework (INFO): Initializing test directory C:\Users\user\AppData\Local\Temp\test_runner_₿_🏃_20260209_120146\wallet_multiwallet_1
2026-02-09T11:01:51.071252Z TestFramework (INFO): Verify warning is emitted when failing to scan the wallets directory
2026-02-09T11:01:51.071428Z TestFramework (WARNING): Skipping test involving chmod as Windows does not support it.
2026-02-09T11:01:54.529466Z TestFramework (ERROR): Unexpected exception
Traceback (most recent call last):
File "C:\Users\user\bitcoin\build\test\functional\test_framework\test_framework.py", line 142, in main
self.run_test()
~~~~~~~~~~~~~^^
File "C:\Users\user\bitcoin\build\/test/functional/wallet_multiwallet.py", line 150, in run_test
with self.nodes[0].assert_debug_log(expected_msgs=["Error while scanning wallet dir"]):
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\AppData\Local\Programs\Python\Python314\Lib\contextlib.py", line 148, in __exit__
next(self.gen)
~~~~^^^^^^^^^^
File "C:\Users\user\bitcoin\build\test\functional\test_framework\test_node.py", line 584, in assert_debug_log
self._raise_assertion_error(f'Expected message(s) {remaining_expected!s} '
~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
f'not found in log:\n\n{join_log(log)}\n\n')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\bitcoin\build\test\functional\test_framework\test_node.py", line 229, in _raise_assertion_error
raise AssertionError(self._node_msg(msg))
AssertionError: [node 0] Expected message(s) ['Error while scanning wallet dir'] not found in log:
- 2026-02-09T11:01:52.527937Z [http] [httpserver.cpp:307] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:59769
- 2026-02-09T11:01:52.528211Z [httpworker.0] [rpc/request.cpp:243] [parse] [rpc] ThreadRPCServer method=listwalletdir user=__cookie__
- 2026-02-09T11:01:52.623421Z [httpworker.0] [wallet/db.cpp:69] [ListDatabases] [warning] Error scanning directory entries under C:\Users\user\AppData\Local\Temp\test_runner_₿_🏃_20260209_120146\wallet_multiwallet_1\node0\regtest\wallets: No such file or directory
2026-02-09T11:01:54.584969Z TestFramework (INFO): Not stopping nodes as test failed. The dangling processes will be cleaned up later.
2026-02-09T11:01:54.585174Z TestFramework (WARNING): Not cleaning up dir C:\Users\user\AppData\Local\Temp\test_runner_₿_🏃_20260209_120146\wallet_multiwallet_1
2026-02-09T11:01:54.585290Z TestFramework (ERROR): Test failed. Test logging available at C:\Users\user\AppData\Local\Temp\test_runner_₿_🏃_20260209_120146\wallet_multiwallet_1/test_framework.log
2026-02-09T11:01:54.585616Z TestFramework (ERROR):
2026-02-09T11:01:54.586080Z TestFramework (ERROR): Hint: Call C:\Users\user\bitcoin\build\test\functional\combine_logs.py 'C:\Users\user\AppData\Local\Temp\test_runner_₿_🏃_20260209_120146\wallet_multiwallet_1' to consolidate all logs
2026-02-09T11:01:54.586260Z TestFramework (ERROR):
2026-02-09T11:01:54.586372Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
2026-02-09T11:01:54.586471Z TestFramework (ERROR): https://github.com/bitcoin/bitcoin/issues
2026-02-09T11:01:54.586540Z TestFramework (ERROR):
stderr:
[node 0] Cleaning up leftover process
Remaining jobs: [wallet_multiwallet.py --usecli]
2/2 - wallet_multiwallet.py --usecli failed, Duration: 6 s
stdout:
2026-02-09T11:01:50.088312Z TestFramework (INFO): PRNG seed is: 8248417181429283859
2026-02-09T11:01:50.138382Z TestFramework (INFO): Initializing test directory C:\Users\user\AppData\Local\Temp\test_runner_₿_🏃_20260209_120146\wallet_multiwallet_0
2026-02-09T11:01:51.977816Z TestFramework (INFO): Verify warning is emitted when failing to scan the wallets directory
2026-02-09T11:01:51.978010Z TestFramework (WARNING): Skipping test involving chmod as Windows does not support it.
2026-02-09T11:01:55.794679Z TestFramework (ERROR): Unexpected exception
Traceback (most recent call last):
File "C:\Users\user\bitcoin\build\test\functional\test_framework\test_framework.py", line 142, in main
self.run_test()
~~~~~~~~~~~~~^^
File "C:\Users\user\bitcoin\build\/test/functional/wallet_multiwallet.py", line 150, in run_test
with self.nodes[0].assert_debug_log(expected_msgs=["Error while scanning wallet dir"]):
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\AppData\Local\Programs\Python\Python314\Lib\contextlib.py", line 148, in __exit__
next(self.gen)
~~~~^^^^^^^^^^
File "C:\Users\user\bitcoin\build\test\functional\test_framework\test_node.py", line 584, in assert_debug_log
self._raise_assertion_error(f'Expected message(s) {remaining_expected!s} '
~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
f'not found in log:\n\n{join_log(log)}\n\n')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\user\bitcoin\build\test\functional\test_framework\test_node.py", line 229, in _raise_assertion_error
raise AssertionError(self._node_msg(msg))
AssertionError: [node 0] Expected message(s) ['Error while scanning wallet dir'] not found in log:
- 2026-02-09T11:01:53.850291Z [http] [httpserver.cpp:307] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:59814
- 2026-02-09T11:01:53.850472Z [httpworker.1] [rpc/request.cpp:243] [parse] [rpc] ThreadRPCServer method=listwalletdir user=__cookie__
- 2026-02-09T11:01:53.936116Z [httpworker.1] [wallet/db.cpp:69] [ListDatabases] [warning] Error scanning directory entries under C:\Users\user\AppData\Local\Temp\test_runner_₿_🏃_20260209_120146\wallet_multiwallet_0\node0\regtest\wallets: No such file or directory
2026-02-09T11:01:55.850081Z TestFramework (INFO): Not stopping nodes as test failed. The dangling processes will be cleaned up later.
2026-02-09T11:01:55.850322Z TestFramework (WARNING): Not cleaning up dir C:\Users\user\AppData\Local\Temp\test_runner_₿_🏃_20260209_120146\wallet_multiwallet_0
2026-02-09T11:01:55.850430Z TestFramework (ERROR): Test failed. Test logging available at C:\Users\user\AppData\Local\Temp\test_runner_₿_🏃_20260209_120146\wallet_multiwallet_0/test_framework.log
2026-02-09T11:01:55.850616Z TestFramework (ERROR):
2026-02-09T11:01:55.851275Z TestFramework (ERROR): Hint: Call C:\Users\user\bitcoin\build\test\functional\combine_logs.py 'C:\Users\user\AppData\Local\Temp\test_runner_₿_🏃_20260209_120146\wallet_multiwallet_0' to consolidate all logs
2026-02-09T11:01:55.851362Z TestFramework (ERROR):
2026-02-09T11:01:55.851432Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
2026-02-09T11:01:55.851523Z TestFramework (ERROR): https://github.com/bitcoin/bitcoin/issues
2026-02-09T11:01:55.851626Z TestFramework (ERROR):
stderr:
[node 0] Cleaning up leftover process
TEST | STATUS | DURATION
wallet_multiwallet.py | ✖ Failed | 4 s
wallet_multiwallet.py --usecli | ✖ Failed | 6 s
ALL | ✖ Failed | 10 s (accumulated)Using the same cross compilation build, this PR fixes the functional test:
PR
python .\build\test\functional\test_runner.py wallet_multiwallet.py
Temporary test directory at C:\Users\user\AppData\Local\Temp/test_runner_₿_🏃_20260209_132101
Remaining jobs: [wallet_multiwallet.py, wallet_multiwallet.py --usecli]
1/2 - wallet_multiwallet.py passed, Duration: 14 s
Remaining jobs: [wallet_multiwallet.py --usecli]
2/2 - wallet_multiwallet.py --usecli passed, Duration: 24 s
TEST | STATUS | DURATION
wallet_multiwallet.py | ✓ Passed | 14 s
wallet_multiwallet.py --usecli | ✓ Passed | 24 s
ALL | ✓ Passed | 38 s (accumulated)
Runtime: 24 s9a7b725 to
022b6e7
Compare
|
Latest push resolves conflict with master where exclusion of wallet_multiwallet.py was moved from ci.yml to ci-windows-cross.py. |
Last use was removed in 0d32d66.
-BEGIN VERIFY SCRIPT- sed --in-place 's/self\.nodes\[0\]/node/g; s/node \= node/node \= self\.nodes\[0\]/' ./test/functional/wallet_multiwallet.py -END VERIFY SCRIPT-
This prepares for later breaking apart of run_test(). Note that the "wallet" lambda was renamed to "get_wallet" since otherwise the Python interpreter emitted: "UnboundLocalError: cannot access local variable 'wallet' where it is not associated with a value"
Makes the functions broken out from run_test() in the next commit more cohesive.
On Windows one gets different exception messages depending on whether running a native build or cross build.
This change ensures that each condition potentially triggering the "Error while scanning" log message is tested independently, avoiding false positives. Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
test_scanning_sub_dir(): - Remove try/finally - we don't need to clean up after a failed test (done in this commit to maintain indentation). Regarding symlinks: bitcoin#31410 (comment) Kept some symlink creation which didn't disrupt Windows cross builds to make for a smaller diff and less cumbersome code. There is some hope of eventually getting better symlink support via bitcoin#34603. Co-authored-by: Ava Chow <github@achow101.com>
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
022b6e7 to
111864a
Compare
|
ACK 111864a |
Makes the functional test compatible with Linux->Windows cross-built executables.
Main parts:
wallet_multiwallet.py#31410.wallet_multiwallet.py#31410 (comment).Also:
run_test()into smaller test functions.assert_equal()output for dicts.Fixes #31409.