Skip to content

Add Bloom filter network messages#580

Merged
apoelstra merged 1 commit intorust-bitcoin:masterfrom
benthecarman:filter-add-clear-p2p-msgs
Sep 13, 2021
Merged

Add Bloom filter network messages#580
apoelstra merged 1 commit intorust-bitcoin:masterfrom
benthecarman:filter-add-clear-p2p-msgs

Conversation

@benthecarman
Copy link
Copy Markdown
Contributor

@benthecarman benthecarman commented Mar 21, 2021

Adds initial support for parsing the merkleblock, filterload, filteradd and filterclear network messages.

https://developer.bitcoin.org/reference/p2p_networking.html#filteradd

https://developer.bitcoin.org/reference/p2p_networking.html#filterclear

@jrawsthorne
Copy link
Copy Markdown
Contributor

Looks like other implementations represent the element in filteradd as raw bytes. The example you linked shows a txid being sent as an element which isn't a script. I worked on this a while ago but never created a pr. https://github.com/jrawsthorne/rust-bitcoin/blob/master/src/network/message_bloom.rs contains filterload as well if you want to include that as well

@benthecarman
Copy link
Copy Markdown
Contributor Author

Looks like other implementations represent the element in filteradd as raw bytes. The example you linked shows a txid being sent as an element which isn't a script. I worked on this a while ago but never created a pr. https://github.com/jrawsthorne/rust-bitcoin/blob/master/src/network/message_bloom.rs contains filterload as well if you want to include that as well

You're right, thanks

@benthecarman benthecarman force-pushed the filter-add-clear-p2p-msgs branch from 86a4ce3 to 9b4dc28 Compare May 18, 2021 06:01
@benthecarman
Copy link
Copy Markdown
Contributor Author

Implemented @jrawsthorne's suggestions, added him as a co-author too :)

@benthecarman benthecarman changed the title Add filteradd & filterclear network messages Add Bloom filter network messages May 18, 2021
@benthecarman benthecarman force-pushed the filter-add-clear-p2p-msgs branch from 9b4dc28 to 0cf61ad Compare May 19, 2021 08:08
jrawsthorne
jrawsthorne previously approved these changes May 19, 2021
Copy link
Copy Markdown
Contributor

@jrawsthorne jrawsthorne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Followed https://github.com/bitcoin/bips/blob/master/bip-0037.mediawiki to double check everything is correct.

@sgeisler sgeisler added the API break This PR requires a version bump for the next release label May 20, 2021
@sgeisler sgeisler added this to the 0.27.0 milestone May 20, 2021
sgeisler
sgeisler previously approved these changes May 20, 2021
Copy link
Copy Markdown
Contributor

@sgeisler sgeisler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK 0cf61adfd48c197ed1f6ede2986bd714dcfb258b

I'm operating under the assumption that the MerkleBlock implementation is correct, it looked ok at least. If that's the case this PR looks good.

@benthecarman benthecarman dismissed stale reviews from sgeisler and jrawsthorne via c2135c9 May 21, 2021 17:46
@benthecarman benthecarman force-pushed the filter-add-clear-p2p-msgs branch from 0cf61ad to c2135c9 Compare May 21, 2021 17:46
sgeisler
sgeisler previously approved these changes May 21, 2021
Copy link
Copy Markdown
Contributor

@sgeisler sgeisler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK c2135c976e0dbe798e357fd4264dea4f941a3b84

apoelstra
apoelstra previously approved these changes Jun 29, 2021
Copy link
Copy Markdown
Member

@apoelstra apoelstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack c2135c976e0dbe798e357fd4264dea4f941a3b84

Reviewed the code and ran tests. Did not verify that the new structures match the spec.

Note that this is a breaking change.

@benthecarman benthecarman dismissed stale reviews from apoelstra and sgeisler via e8904df July 28, 2021 17:05
@benthecarman benthecarman force-pushed the filter-add-clear-p2p-msgs branch from c2135c9 to e8904df Compare July 28, 2021 17:05
@benthecarman
Copy link
Copy Markdown
Contributor Author

Rebased

apoelstra
apoelstra previously approved these changes Sep 8, 2021
Copy link
Copy Markdown
Member

@apoelstra apoelstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK e8904df5f36139ed9cea2725164a16b6fa0a96ef

Don't know why CI is not running. It passes locally (although I did not run the examples/ directory, and I have only access to x86_64, and not wasm)

@benthecarman
Copy link
Copy Markdown
Contributor Author

Don't know why CI is not running. It passes locally (although I did not run the examples/ directory, and I have only access to x86_64, and not wasm)

Github is weird with first time contributors and sometimes doesn't run CI

dr-orlovsky
dr-orlovsky previously approved these changes Sep 13, 2021
Copy link
Copy Markdown
Collaborator

@dr-orlovsky dr-orlovsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK e8904df5f36139ed9cea2725164a16b6fa0a96ef

Compared correspondence of data structures and serialization to the spec https://github.com/bitcoin/bips/blob/master/bip-0037.mediawiki#New_messages

@dr-orlovsky
Copy link
Copy Markdown
Collaborator

@benthecarman I tried everything to get the tests running so I can merge this PR. No success :( Seems like the only way is to force-push, which will discard all reviews however :( I will ask one of maintainers to add his confirmation ASAP if you will chose this path (I actually see no other route...)

Co-authored-by: jrawsthorne <jake@jakerawsthorne.co.uk>
@benthecarman benthecarman dismissed stale reviews from dr-orlovsky and apoelstra via 894f0f0 September 13, 2021 20:08
@benthecarman benthecarman force-pushed the filter-add-clear-p2p-msgs branch from e8904df to 894f0f0 Compare September 13, 2021 20:08
@benthecarman
Copy link
Copy Markdown
Contributor Author

@benthecarman I tried everything to get the tests running so I can merge this PR. No success :( Seems like the only way is to force-push, which will discard all reviews however :( I will ask one of maintainers to add his confirmation ASAP if you will chose this path (I actually see no other route...)

Rebased

Copy link
Copy Markdown
Collaborator

@dr-orlovsky dr-orlovsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 894f0f0

Copy link
Copy Markdown
Member

@apoelstra apoelstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 894f0f0

@apoelstra apoelstra merged commit 57d7baf into rust-bitcoin:master Sep 13, 2021
@benthecarman benthecarman deleted the filter-add-clear-p2p-msgs branch September 13, 2021 22:00
apoelstra added a commit to apoelstra/rust-bitcoin that referenced this pull request Aug 22, 2025
This commit confirms that there is not a single untouched script instance
across the project; every one has been converted to a tagged script.

In bitcoin/src/network/params.rs I changed the signet example to use a
WitnessScript. I wasn't sure if this script is interpreted as a witness
script or a scriptpubkey on Signet itself so I guessed.

In p2p/src/message.rs there is a "script" that is actually just a blob
of raw bytes (that happens to be a script) used in a "test" of the bloom
filter messages. See rust-bitcoin#580
where this was added for discussion about script vs bytes.

(BTW why did I let a bloom filtering PR into this project? This whole
system was a bad idea and a DoS vector. Is it even used?)
apoelstra added a commit to apoelstra/rust-bitcoin that referenced this pull request Aug 22, 2025
This commit confirms that there is not a single untouched script instance
across the project; every one has been converted to a tagged script.

In bitcoin/src/network/params.rs I changed the signet example to use a
WitnessScript. I wasn't sure if this script is interpreted as a witness
script or a scriptpubkey on Signet itself so I guessed.

In p2p/src/message.rs there is a "script" that is actually just a blob
of raw bytes (that happens to be a script) used in a "test" of the bloom
filter messages. See rust-bitcoin#580
where this was added for discussion about script vs bytes.

(BTW why did I let a bloom filtering PR into this project? This whole
system was a bad idea and a DoS vector. Is it even used?)
apoelstra added a commit to apoelstra/rust-bitcoin that referenced this pull request Aug 22, 2025
This commit confirms that there is not a single untouched script instance
across the project; every one has been converted to a tagged script.

This triggers a number of doc changes which should have gone into previous
commits. I apologize for that. But it's a PITA to go back and locate exactly
which commit each doc change was supposed to go into.

In bitcoin/src/network/params.rs I changed the signet example to use a
WitnessScript. I wasn't sure if this script is interpreted as a witness
script or a scriptpubkey on Signet itself so I guessed.

In p2p/src/message.rs there is a "script" that is actually just a blob
of raw bytes (that happens to be a script) used in a "test" of the bloom
filter messages. See rust-bitcoin#580
where this was added for discussion about script vs bytes.

(BTW why did I let a bloom filtering PR into this project? This whole
system was a bad idea and a DoS vector. Is it even used?)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

API break This PR requires a version bump for the next release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants