Skip to content

[pubsub] Prioritise internal subscribers (e.g. reactor) over external (e.g. RPC)#1574

Merged
melekes merged 7 commits intodevelopfrom
847-separate-internal-pubsub
May 24, 2018
Merged

[pubsub] Prioritise internal subscribers (e.g. reactor) over external (e.g. RPC)#1574
melekes merged 7 commits intodevelopfrom
847-separate-internal-pubsub

Conversation

@melekes
Copy link
Contributor

@melekes melekes commented May 15, 2018

Refs #847

  • Updated all relevant documentation in docs
  • Updated all code comments where relevant
  • Wrote tests
  • Updated CHANGELOG.md

@melekes melekes requested a review from ebuchman as a code owner May 15, 2018 10:34
fastSync: fastSync,
}
// XXX: modifing state to send us new round steps, votes and proposal heartbeats
consensusState.reactorChs = &reactorChs{
Copy link
Contributor Author

@melekes melekes May 15, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not good, is it? should we bring events pkg to avoid doing this?

@melekes
Copy link
Contributor Author

melekes commented May 16, 2018

I had to alter events package - see 30b5688

@melekes melekes force-pushed the 847-separate-internal-pubsub branch 2 times, most recently from ecc94bc to 9c1a022 Compare May 16, 2018 08:34
@codecov-io
Copy link

codecov-io commented May 16, 2018

Codecov Report

Merging #1574 into develop will increase coverage by 0.64%.
The diff coverage is 69.85%.

@@             Coverage Diff             @@
##           develop    #1574      +/-   ##
===========================================
+ Coverage    63.57%   64.22%   +0.64%     
===========================================
  Files          103      109       +6     
  Lines         9538    10633    +1095     
===========================================
+ Hits          6064     6829     +765     
- Misses        2907     3188     +281     
- Partials       567      616      +49
Impacted Files Coverage Δ
rpc/client/localclient.go 88.46% <ø> (ø) ⬆️
rpc/lib/types/types.go 29.16% <ø> (ø) ⬆️
rpc/client/httpclient.go 68.31% <ø> (-1%) ⬇️
state/txindex/kv/kv.go 74.77% <ø> (ø) ⬆️
libs/events/event_cache.go 100% <100%> (ø)
libs/pubsub/query/empty.go 50% <50%> (ø)
libs/pubsub/query/query.peg.go 63.05% <63.05%> (ø)
libs/pubsub/query/query.go 63.67% <63.67%> (ø)
consensus/state.go 76.05% <66.66%> (+0.22%) ⬆️
libs/pubsub/pubsub.go 88.32% <88.32%> (ø)
... and 13 more

@melekes melekes force-pushed the 847-separate-internal-pubsub branch from 5d2cfa8 to 07ea687 Compare May 16, 2018 09:05
@melekes melekes requested a review from jaekwon May 17, 2018 06:51
@melekes melekes mentioned this pull request May 17, 2018
4 tasks
@ebuchman
Copy link
Contributor

I had to alter events package

What required this ?

// newStep is called by updateToStep in NewConsensusState before the eventBus is set!
if cs.eventBus != nil {
cs.eventBus.PublishEventNewRoundStep(rs)
cs.evsw.FireEvent(types.EventNewRoundStep, &cs.RoundState)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not pass rs here ?

Copy link
Contributor Author

@melekes melekes May 21, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why send a shallow copy if I can send an original object? (events is sync. so there is no race)

@melekes
Copy link
Contributor Author

melekes commented May 21, 2018

What required this ?

Not sure how it was done before, but the problem is in order to catch first NewRoundStep I need to subscribe to such events before SwitchToConsensus emits it (30b5688#diff-f50d554d9c7f92cf7686b247b8c91419L85). See 30b5688 Because of that, I can't use conS.Start() to start eventSwitch and I have to initialise it upon construction.

@melekes melekes force-pushed the 847-separate-internal-pubsub branch from c0b4fda to a9d0adb Compare May 21, 2018 06:56
@ebuchman
Copy link
Contributor

Sounds good. Test failure:

--- FAIL: TestPEXReactorRunning (10.16s)
	pex_reactor_test.go:409: expected all switches to be connected to at least one peer (switches: 0 => {outbound: 0, inbound: 1}, 1 => {outbound: 1, inbound: 1}, 2 => {outbound: 2, inbound: 0}, )
FAIL
coverage: 73.7% of statements
FAIL	github.com/tendermint/tendermint/p2p/pex	19.873s
Exited with code 1

I can reproduce locally on this branch but not on develop. What's that about !?

@ebuchman
Copy link
Contributor

Oh the message is wrong - it actually expects everyone to be fully connected. And now I can't reproduce it ...

@ebuchman
Copy link
Contributor

Hmm I reran the test on Circle and it timedout of the consensus test.

Ran the consensus tests locally and it failed on TestMempoolProgressAfterCreateEmptyBlocksInterval

Looks like this might be introducing some non-determinism ?

@melekes
Copy link
Contributor Author

melekes commented May 22, 2018

Hmm.. I will take a look tmrw

@melekes
Copy link
Contributor Author

melekes commented May 24, 2018

Looks like this might be introducing some non-determinism ?

maybe, but how? all that got changed is the way CS reactor gets its events.

@melekes melekes merged commit a885af0 into develop May 24, 2018
@ebuchman
Copy link
Contributor

Yeh especially given the test failure in p2p. Maybe the change is just helping surface previously lurking non-determinism. We're also seeing it in #1607

@zramsay zramsay deleted the 847-separate-internal-pubsub branch October 30, 2018 15:09
firelizzard18 pushed a commit to AccumulateNetwork/tendermint that referenced this pull request Feb 1, 2024
)

Bumps [golang.org/x/sync](https://github.com/golang/sync) from 0.4.0 to 0.5.0.
- [Commits](golang/sync@v0.4.0...v0.5.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sync
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants