Skip to content

libs: remove most of libs/rand#6364

Merged
cmwaters merged 28 commits intomasterfrom
ismail/remove_libs-rand
Apr 23, 2021
Merged

libs: remove most of libs/rand#6364
cmwaters merged 28 commits intomasterfrom
ismail/remove_libs-rand

Conversation

@liamsi
Copy link
Contributor

@liamsi liamsi commented Apr 17, 2021

The only functions that are left (for convenience) are:

  • NewRand (used to init. math/rand.Rand instance with crypto/rand as before - not go-routine safe)
  • Reseed (new method to seed the math/rand global source with crypto/rand - math/rand is still thread safe)
  • Str (used in a bunch of tests)
  • Bytes (used in a bunch of tests too)

Everything else is either replaced with vanilla math/rand (in tests) or with math/rand.Rand instances from NewRand that are seeded with crypto/rand (which is as "secure" as before, maybe even a bit more "secure" as there is no global object that is only seeded once).

TODO: there are a bunch of G404: Use of weak random number generator (math/rand instead of crypto/rand) (gosec). We can either use crypto/rand (e.g. in real code) or stick with math/rand and silence the linter (in tests). Note, that this wasn't (cryptographically) secure before either. It was just obfuscated because of the use of libs/rand (no one has written a linter here although @ValarDragon kinda warned us in #2956 and probably on various other channels too :-D ).
Edit: made the above more explicit as suggested in #6364 (review)

closes #2956

@liamsi liamsi changed the title Remove most of libs/rand libs: remove most of libs/rand Apr 17, 2021
@codecov
Copy link

codecov bot commented Apr 17, 2021

Codecov Report

Merging #6364 (cae5712) into master (0d6055a) will increase coverage by 0.06%.
The diff coverage is 82.05%.

@@            Coverage Diff             @@
##           master    #6364      +/-   ##
==========================================
+ Coverage   60.80%   60.87%   +0.06%     
==========================================
  Files         283      283              
  Lines       26945    26866      -79     
==========================================
- Hits        16384    16354      -30     
+ Misses       8859     8814      -45     
+ Partials     1702     1698       -4     
Impacted Files Coverage Δ
p2p/test_util.go 60.80% <0.00%> (ø)
libs/rand/random.go 76.92% <76.92%> (+9.95%) ⬆️
abci/example/kvstore/helpers.go 100.00% <100.00%> (ø)
consensus/wal_generator.go 70.75% <100.00%> (ø)
libs/bits/bit_array.go 80.37% <100.00%> (+0.09%) ⬆️
p2p/pex/addrbook.go 69.09% <100.00%> (+0.50%) ⬆️
p2p/pex/pex_reactor.go 76.76% <100.00%> (+0.20%) ⬆️
p2p/switch.go 61.53% <100.00%> (+1.63%) ⬆️
rpc/jsonrpc/client/ws_client.go 63.08% <100.00%> (ø)
types/validator.go 81.92% <100.00%> (ø)
... and 13 more

Copy link
Contributor

@ValarDragon ValarDragon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very excited this is getting changed! Thanks for doing it!

Re: The gosec concern, I think outside of tests, its useful to mark the lines explicitly with nolint gosec, so its explicit to the reader that these are intentionally not cryptographically secure.

For tests, maybe the entire file should get the nolint: gosec applied?

func NewRand() *mrand.Rand {
var seed int64
binary.Read(crand.Reader, binary.BigEndian, &seed)
return mrand.New(mrand.NewSource(seed))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this actually isn't thread-safe, we instead need to use mrand.Seed for it to be thread-safe. (Per comment on https://golang.org/pkg/math/rand/#Seed, and NewSource not being thread safe comment on https://golang.org/pkg/math/rand/#NewSource)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! It's probably best to make this function not return anything but only call math/rand.Seed instead. So wherever a freshly seeded math/rand.Rand was used, I'll just reseed the global one lives internally in math/Rand by calling this function instead. Tomorrow...

Copy link
Contributor Author

@liamsi liamsi Apr 17, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, I've added a comment that clarifies that this isn't thread-safe (5bd124f) and added a convenience function to reseed math/rand's global Source from crypto/rand (7845a53).

} else {
chars = append(chars, strChars[v])
if len(chars) == length {
break MAIN_LOOP
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

woo, happy this is getting deleted!

Comment on lines +418 to +419
// nolint:gosec // G404: Use of weak random number generator
j := mrand.Intn(len(allAddr)-i) + i

This comment was marked as off-topic.

@liamsi liamsi marked this pull request as ready for review April 17, 2021 12:09
Copy link
Contributor

@cmwaters cmwaters left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Co-authored-by: Callum Waters <cmwaters19@gmail.com>
Bytes/Str are not reseeded each time they are called
Copy link
Contributor

@tac0turtle tac0turtle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. Could you add a changelog entry before we merge?

@liamsi
Copy link
Contributor Author

liamsi commented Apr 20, 2021

Of course, will do it later today 👍🏼

@liamsi
Copy link
Contributor Author

liamsi commented Apr 20, 2021

Did they screw up on that codecov action update or did someone tamper with their bash script again? https://github.com/tendermint/tendermint/pull/6364/checks?check_run_id=2392223532#step:9:1

It was the update: https://twitter.com/thomasrockhu/status/1384619989634797576 and merging in #6379 will fix that.

@cmwaters
Copy link
Contributor

I am going to merge this now. Thanks for the contribution @liamsi

@cmwaters cmwaters merged commit ee70430 into master Apr 23, 2021
@cmwaters cmwaters deleted the ismail/remove_libs-rand branch April 23, 2021 14:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove usage of rand.Rand

4 participants