Skip to content
This repository was archived by the owner on Jul 27, 2022. It is now read-only.

Problem: latest parity/scale-codec failed to compile (CRO-161) #120#133

Merged
tomtau merged 3 commits intocrypto-com:masterfrom
leejw51crypto:master
Jun 12, 2019
Merged

Problem: latest parity/scale-codec failed to compile (CRO-161) #120#133
tomtau merged 3 commits intocrypto-com:masterfrom
leejw51crypto:master

Conversation

@leejw51crypto
Copy link
Copy Markdown
Collaborator

@leejw51crypto leejw51crypto commented Jun 10, 2019

Solution: converted usize fields (in datatypes that use scale-codec) to u64


This change is Reviewable

@tomtau tomtau requested review from devashishdxt and tomtau and removed request for tomtau June 10, 2019 13:22
Copy link
Copy Markdown
Contributor

@devashishdxt devashishdxt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments:

  • There's a compilation error in tests. You'll have to change usize to u64 in tests as well.
  • Run cargo fmt and cargo clippy before pushing your code. This will properly format your code and also warn you about non-idiomatic Rust code. Current CI has lint and formatting checks in place so, build won't succeed without properly formatted code.

@leejw51crypto
Copy link
Copy Markdown
Collaborator Author

A few comments:

  • There's a compilation error in tests. You'll have to change usize to u64 in tests as well.
  • Run cargo fmt and cargo clippy before pushing your code. This will properly format your code and also warn you about non-idiomatic Rust code. Current CI has lint and formatting checks in place so, build won't succeed without properly formatted code.

thanks
i'll fix today

Copy link
Copy Markdown
Contributor

@tomtau tomtau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the basic fix looks OK, but needs a few small changes.
In addition to @devashishdxt suggestions here (cargo fmt, cargo clippy, cargo test): #133 (review)

It'd be good to follow the patch convention for commit / pull request message: https://rfc.zeromq.org/spec:42/C4/#23-patch-requirements
Here, it could be, for example:

Problem: latest parity/scale-codec fails to compile (CRO-161) #120

Solution: converted usize fields (in datatypes that use scale-codec) to u64

@leejw51crypto leejw51crypto force-pushed the master branch 2 times, most recently from bb5f011 to a76ea27 Compare June 11, 2019 07:10
@codecov
Copy link
Copy Markdown

codecov bot commented Jun 11, 2019

Codecov Report

❗ No coverage uploaded for pull request base (master@07db604). Click here to learn what that means.
The diff coverage is 82.6%.

@@            Coverage Diff            @@
##             master     #133   +/-   ##
=========================================
  Coverage          ?   79.29%           
=========================================
  Files             ?       77           
  Lines             ?     6971           
  Branches          ?        0           
=========================================
  Hits              ?     5528           
  Misses            ?     1443           
  Partials          ?        0
Impacted Files Coverage Δ
client-index/src/index/default_index.rs 0% <0%> (ø)
chain-core/src/state/mod.rs 54.83% <0%> (ø)
chain-core/src/tx/data/input.rs 72.72% <100%> (ø)
chain-abci/src/storage/tx.rs 92.98% <100%> (ø)
chain-core/src/tx/data/access.rs 87.09% <100%> (ø)
chain-core/src/common/merkle_tree.rs 99.75% <100%> (ø)
chain-abci/src/app/mod.rs 91.07% <100%> (ø)
client-core/src/service/root_hash_service.rs 94.73% <66.66%> (ø)

devashishdxt
devashishdxt previously approved these changes Jun 11, 2019
@devashishdxt
Copy link
Copy Markdown
Contributor

LGTM. Wait for @tomtau to approve before merging.

Copy link
Copy Markdown
Contributor

@tomtau tomtau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 20 of 20 files at r1.
Reviewable status: all files reviewed, 8 unresolved discussions (waiting on @leejw51crypto)


chain-abci/src/app/validate_tx.rs, line 72 at r1 (raw file):

                let min_fee = state
                    .fee_policy
                    .calculate_fee(_req.tx().len() as u64)

I think calculate_fee could keep usize? it's a function, so no effect on parity/scale codec encoding


chain-abci/src/storage/tx.rs, line 158 at r1 (raw file):

        match txo {
            Ok(Some(v)) => {
                let bv = BitVec::from_bytes(&v).get(txin.index as usize);

perhaps create a casted value in the beginning:

let input_index =  txin.index as usize;

and then use this later instead of all the casts?

 BitVec::from_bytes(&v).get(input_index);
...
if input_index >= outputs.len()  {
...
let txout = &outputs[input_index];

?


chain-core/src/init/address.rs, line 27 at r1 (raw file):

/// Keccak-256 crypto hash length in bytes
pub const KECCAK256_BYTES: u64 = 32;

is it necessary to change this constant's type? I think it's only used in defining the array sizes?


chain-core/src/tx/fee/mod.rs, line 70 at r1 (raw file):

pub enum MilliError {
    /// An invalid length of parts (should be 2)
    InvalidPartsLength(u64),

MilliError doesn't seem to need Encode/Decode, so I think it could be kept as usize?


chain-core/src/tx/fee/mod.rs, line 154 at r1 (raw file):

    }

    pub fn estimate(&self, sz: u64) -> Result<Fee, CoinError> {

I think calculate_fee / estimate could keep usize? They are functions, so no effect on parity/scale codec encoding


client-core/src/service/root_hash_service.rs, line 114 at r1 (raw file):

}

fn combinations(public_keys: Vec<PublicKey>, n: u64) -> Result<Vec<RawPubkey>> {

does n need to be u64 here? @devashishdxt


client-core/src/wallet/default_wallet_client.rs, line 677 at r1 (raw file):
as mentioned:

I think calculate_fee / estimate could keep usize? They are functions, so no effect on parity/scale codec encoding


client-rpc/src/wallet_rpc.rs, line 259 at r1 (raw file):

I think calculate_fee / estimate could keep usize? They are functions, so no effect on parity/scale codec encoding

Copy link
Copy Markdown
Contributor

@tomtau tomtau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@leejw51crypto looks good, just three small requests before merging it:

  • upstream master changed, so just sync up with that (git pull origin master + resolve conflicts)
  • usize can be kept where it's not related to binary encoding/serialisation (i.e. the fee algorithm function arguments + error data types that don't implement binary serialization)
  • if there's a block of code where one value needs to be re-casted multiple times, it could be recasted once let recasted_val = original_val as ...; and then used in those places

Reviewable status: all files reviewed, 8 unresolved discussions (waiting on @leejw51crypto)

@tomtau tomtau changed the title usize to u64 for parity-codec Problem: latest parity/scale-codec failed to compile (CRO-161) #120 Jun 11, 2019
@tomtau tomtau self-assigned this Jun 11, 2019
@devashishdxt
Copy link
Copy Markdown
Contributor

Since we are making this change only to support encoding and decoding of usize, will it make more sense just to implement Encode and Decode for types containing usize instead of changing usize to u64 everywhere? @tomtau @leejw51crypto any thoughts?

@tomtau
Copy link
Copy Markdown
Contributor

tomtau commented Jun 11, 2019

Since we are making this change only to support encoding and decoding of usize, will it make more sense just to implement Encode and Decode for types containing usize instead of changing usize to u64 everywhere? @tomtau @leejw51crypto any thoughts?

do you mean to keep usize and have a manual implementation of Encode and Decode?

@devashishdxt
Copy link
Copy Markdown
Contributor

do you mean to keep usize and have a manual implementation of Encode and Decode?

Yes. This will also help with error handling and avoiding integer overflows. Since Rust does not perform any overflow checks in release mode, I think manually implementing Encode and Decode with overflow checks for these special cases will give us more control and safety as compared to casting u64 to usize at multiple places.

@tomtau
Copy link
Copy Markdown
Contributor

tomtau commented Jun 11, 2019

do you mean to keep usize and have a manual implementation of Encode and Decode?

Yes. This will also help with error handling and avoiding integer overflows. Since Rust does not perform any overflow checks in release mode, I think manually implementing Encode and Decode with overflow checks for these special cases will give us more control and safety as compared to casting u64 to usize at multiple places.

There were two cases where usize was used:

  • unnecessary (e.g. nonce): here, it's fine to change to u64
  • lengths / indices: here, we'll want manual Encode/Decode with size checking

For the latter one, it'll require more thought case-by-case (or datatype-by-datatype), so I'll delay that for now -- the simplest solution for the problem here ("latest parity/scale-codec failed to compile") is to keep using auto-derived implementations.
Later PRs could then fix individual problems that this may cause ("unreasonably big tx input/output length", "Merkle Tree length may overflow on 32-bit archs"...)

@leejw51crypto
Copy link
Copy Markdown
Collaborator Author

ok, got it

Bumps [quickcheck](https://github.com/BurntSushi/quickcheck) from 0.8.3 to 0.8.5.
- [Release notes](https://github.com/BurntSushi/quickcheck/releases)
- [Commits](BurntSushi/quickcheck@0.8.3...0.8.5)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Problem: latest parity/scale-codec failed to compile (CRO-161) crypto-com#120

Solution: converted usize fields (in datatypes that use scale-codec) to u64
@tomtau
Copy link
Copy Markdown
Contributor

tomtau commented Jun 12, 2019

@leejw51crypto leejw51crypto force-pushed the master branch 3 times, most recently from 791ac43 to c9a7e8b Compare June 12, 2019 07:28
@leejw51crypto
Copy link
Copy Markdown
Collaborator Author

merged with latest master

@tomtau tomtau merged commit a80b902 into crypto-com:master Jun 12, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants