Skip to content

booster-bitswap devnet and tracing#796

Merged
nonsense merged 3 commits intorelease/lotus1.17.2from
nonsense/devnet-and-tracing
Sep 13, 2022
Merged

booster-bitswap devnet and tracing#796
nonsense merged 3 commits intorelease/lotus1.17.2from
nonsense/devnet-and-tracing

Conversation

@nonsense
Copy link
Copy Markdown
Member

@nonsense nonsense commented Sep 13, 2022

Initial tracing setup for booster-bitswap.

@nonsense nonsense marked this pull request as ready for review September 13, 2022 13:14
Comment on lines +39 to +48
&cli.BoolFlag{
Name: "tracing",
Usage: "enables tracing of booster-http calls",
Value: false,
},
&cli.StringFlag{
Name: "tracing-endpoint",
Usage: "the endpoint for the tracing exporter",
Value: "http://tempo:14268/api/traces",
},
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to only have the "tracing-endpoint" flag? It's implied that tracing is turned on if this flag is there so maybe we don't need the "tracing" flag?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find it nice not having to specify the tracing-endpoint when running the devnet, although what you suggest also makes sense.

@nonsense nonsense merged commit 7f2aed0 into release/lotus1.17.2 Sep 13, 2022
nonsense added a commit that referenced this pull request Sep 13, 2022
@nonsense nonsense mentioned this pull request Sep 13, 2022
8 tasks
LexLuthr pushed a commit that referenced this pull request Oct 4, 2022
* booster bitswap MVP executable (#707)

* feat(booster-bitswap): booster bitswap MVP untested

* refactor(booster-bitswap): use API for fetching blocks

* fix(deps): update deps to compile

* feat(booster-bitswap): makefile & fixes

add commands to build booster-bitswap, and very a round tripped successful fetch from
booster-bitswap

* refactor: clean up unused vars etc

* fix: booster-bitsawp - check error when creating libp2p key

* refactor(node): avoid FreeAndUnsealed method

Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>
Co-authored-by: Anton Evangelatov <anton.evangelatov@gmail.com>

* booster-bitswap devnet and tracing (#796)

* return ipld ErrNotFound from remote blockstore interface (#798)

* fix: return ipld ErrNotFound from remote blockstore interface

* test: add more tests for ipld ErrNotFound

* test: comment out part of TestDummydealOnline that is flaky due to a bug in latest lotus (#802)

* fix normaliseError nil ptr dereference (#803)

* feat: shard selector (#807)

* LoadBalancer for bitswap (and later, more of libp2p) (#786)

* feat(loadbalancer): add message types

* feat(messages): add utility functions

* feat(loadbalancer): initial load balancer impl

implementation of the load balancer node itself

* feat(loadbalancer): add service node

implements code for running a service node

* feat(loadbalancer): integrate into boost and booster-bitswap

* Update loadbalancer/loadbalancer.go

Co-authored-by: Rod Vagg <rod@vagg.org>

* Update loadbalancer/servicenode.go

Co-authored-by: Rod Vagg <rod@vagg.org>

* Update loadbalancer/servicenode.go

Co-authored-by: Rod Vagg <rod@vagg.org>

* Update loadbalancer/messages/messages.ipldsch

Co-authored-by: Rod Vagg <rod@vagg.org>

* Update loadbalancer/messages/messages.ipldsch

Co-authored-by: Rod Vagg <rod@vagg.org>

* refactor(loadbalancer): remove routing protocol

remove the routing protocol, instead relying on a set config. also remove forwarding response for
inbound requests

* fix(loadbalancer): update tests

* refactor(loadbalancer): integrate simplified load balancer

removed pub keys to minimize network traffic, added api's to configure and update bitswap peer id,
added auto config of bitswap peer id in booster-bitswap

* docs(gen): regenerate api docs

* chore(lint): fix lint errors

* fix(loadbalancer): minor bridgestream fix

* Update loadbalancer/servicenode.go

Co-authored-by: dirkmc <dirkmdev@gmail.com>

* refactor(protocolproxy): address PR comments

renames, reconfigured architecture, etc

* refactor(make init print out peer id): remove apis and transparent peer id setting. have init print

Co-authored-by: Rod Vagg <rod@vagg.org>
Co-authored-by: dirkmc <dirkmdev@gmail.com>

* Add block filter via BadBits (#825)

* feat(booster-bitswap): add block filter via BadBits

* refactor(booster-bitswap): use bitswap blockfilter for filtering

* feat(blockfilter): only update when list is modified

* feat(blockFilter): add on disk caching

* Update cmd/booster-bitswap/blockfilter/blockfilter.go

Co-authored-by: dirkmc <dirkmdev@gmail.com>

* fix(blockfilter): minor PR fixups

Co-authored-by: dirkmc <dirkmdev@gmail.com>

* Libp2p 0.22 upgrade (#837)

* chore(deps): upgrade to Lotus RC & libp2p v0.22

* chore(deps): update go to 1.18

* ci(circle): update circle to go 1.18

* style(imports): fix imports

* fix(build): update ffi

* fix(lint): fix deprecated strings.Title method

* fix(mod): mod tidy

* Protocol Proxy cleanup (#836)

* refactor(booster-bitswap): minor UI fixes for booster-bitswap UI

* Update cmd/booster-bitswap/init.go

Co-authored-by: dirkmc <dirkmdev@gmail.com>

Co-authored-by: dirkmc <dirkmdev@gmail.com>

* feat: update to dagstore v0.5.5 (#849)

* add booster-bitswap to devnet (#866)

* bump lotus-test version

* add docker/booster-bitswap target in Makefile

Co-authored-by: Hannah Howard <hannah@hannahhoward.net>
Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>
Co-authored-by: Rod Vagg <rod@vagg.org>
dirkmc added a commit that referenced this pull request Oct 4, 2022
* booster bitswap MVP executable (#707)

* feat(booster-bitswap): booster bitswap MVP untested

* refactor(booster-bitswap): use API for fetching blocks

* fix(deps): update deps to compile

* feat(booster-bitswap): makefile & fixes

add commands to build booster-bitswap, and very a round tripped successful fetch from
booster-bitswap

* refactor: clean up unused vars etc

* fix: booster-bitsawp - check error when creating libp2p key

* refactor(node): avoid FreeAndUnsealed method

Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>
Co-authored-by: Anton Evangelatov <anton.evangelatov@gmail.com>

* booster-bitswap devnet and tracing (#796)

* return ipld ErrNotFound from remote blockstore interface (#798)

* fix: return ipld ErrNotFound from remote blockstore interface

* test: add more tests for ipld ErrNotFound

* test: comment out part of TestDummydealOnline that is flaky due to a bug in latest lotus (#802)

* fix normaliseError nil ptr dereference (#803)

* feat: shard selector (#807)

* LoadBalancer for bitswap (and later, more of libp2p) (#786)

* feat(loadbalancer): add message types

* feat(messages): add utility functions

* feat(loadbalancer): initial load balancer impl

implementation of the load balancer node itself

* feat(loadbalancer): add service node

implements code for running a service node

* feat(loadbalancer): integrate into boost and booster-bitswap

* Update loadbalancer/loadbalancer.go

Co-authored-by: Rod Vagg <rod@vagg.org>

* Update loadbalancer/servicenode.go

Co-authored-by: Rod Vagg <rod@vagg.org>

* Update loadbalancer/servicenode.go

Co-authored-by: Rod Vagg <rod@vagg.org>

* Update loadbalancer/messages/messages.ipldsch

Co-authored-by: Rod Vagg <rod@vagg.org>

* Update loadbalancer/messages/messages.ipldsch

Co-authored-by: Rod Vagg <rod@vagg.org>

* refactor(loadbalancer): remove routing protocol

remove the routing protocol, instead relying on a set config. also remove forwarding response for
inbound requests

* fix(loadbalancer): update tests

* refactor(loadbalancer): integrate simplified load balancer

removed pub keys to minimize network traffic, added api's to configure and update bitswap peer id,
added auto config of bitswap peer id in booster-bitswap

* docs(gen): regenerate api docs

* chore(lint): fix lint errors

* fix(loadbalancer): minor bridgestream fix

* Update loadbalancer/servicenode.go

Co-authored-by: dirkmc <dirkmdev@gmail.com>

* refactor(protocolproxy): address PR comments

renames, reconfigured architecture, etc

* refactor(make init print out peer id): remove apis and transparent peer id setting. have init print

Co-authored-by: Rod Vagg <rod@vagg.org>
Co-authored-by: dirkmc <dirkmdev@gmail.com>

* Add block filter via BadBits (#825)

* feat(booster-bitswap): add block filter via BadBits

* refactor(booster-bitswap): use bitswap blockfilter for filtering

* feat(blockfilter): only update when list is modified

* feat(blockFilter): add on disk caching

* Update cmd/booster-bitswap/blockfilter/blockfilter.go

Co-authored-by: dirkmc <dirkmdev@gmail.com>

* fix(blockfilter): minor PR fixups

Co-authored-by: dirkmc <dirkmdev@gmail.com>

* Libp2p 0.22 upgrade (#837)

* chore(deps): upgrade to Lotus RC & libp2p v0.22

* chore(deps): update go to 1.18

* ci(circle): update circle to go 1.18

* style(imports): fix imports

* fix(build): update ffi

* fix(lint): fix deprecated strings.Title method

* fix(mod): mod tidy

* Protocol Proxy cleanup (#836)

* refactor(booster-bitswap): minor UI fixes for booster-bitswap UI

* Update cmd/booster-bitswap/init.go

Co-authored-by: dirkmc <dirkmdev@gmail.com>

Co-authored-by: dirkmc <dirkmdev@gmail.com>

* feat: update to dagstore v0.5.5 (#849)

* feat: bitswap client

* feat: bitswap client - output car file

* refactor: bitswap client - remove tracing

* feat: debug logs

* fix: write blocks to blockstore

* fix: duration output

* fix: duration output for block received

* feat: add pprof to bitswap client

* feat: protocol proxy logging

* feat: bitswap client - check host supports bitswap protocol

* feat: listen for bitswap requests locally as well as through forwarding protocol

Co-authored-by: Hannah Howard <hannah@hannahhoward.net>
Co-authored-by: Anton Evangelatov <anton.evangelatov@gmail.com>
Co-authored-by: Rod Vagg <rod@vagg.org>
@nonsense nonsense deleted the nonsense/devnet-and-tracing branch October 17, 2022 09:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants