Skip to content

hex_bytes is slow #207

@dpc

Description

@dpc

I am doing blockchain indexing and when analyzing perf output of my indexer (which is quite CPU-bound at this point), I can see 10% of CPU time sitting in hex_bytes. I query blocks from the node, which get decoded from hex response to Vec<u8> and then to Block via consensus_decode. All of this happens in rust-bitcoincore-rpc https://github.com/rust-bitcoin/rust-bitcoincore-rpc/blob/master/client/src/queryable.rs#L23

Looking at:

https://github.com/rust-bitcoin/rust-bitcoin/blob/master/src/util/misc.rs#L24

I see the following room for improvement:

  • output vector's capacity could be preallocated (should be typically 1/2 of the string length)
  • chars is for handling Unicode chars, so I suspect there's quite a bit of overhead there. We can assume that the hex string is ASCII, and just convert it to &[u8] directly via as_bytes() and work on that; unicode characters are > 128 so it all should be fine;
  • the whole iterator-based approach seems suspicious, performance-wise; pair, fold; who knows how it gets optimized; a simpler, imperative code would be both more readable, and maybe faster

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions