write blocks retrieved from the exchange to the blockstore#92
write blocks retrieved from the exchange to the blockstore#92Jorropo merged 9 commits intoipfs:masterfrom
Conversation
|
cc @Jorropo |
blockservice.go
Outdated
| logger.Debugf("BlockService.BlockFetched %s", b.Cid()) | ||
|
|
||
| // also write in the blockstore for caching | ||
| // TODO: for performance reason, it might make sense to batch those writes |
There was a problem hiding this comment.
This probably needs to happen before we merge (it has significant performance implications). A simple version would be to read from rblocks with a default select statement (i.e., drain it), then write a batch. That's not perfect, but it'll help as long as rblocks is buffered (IIRC, it is).
There was a problem hiding this comment.
That's not perfect, but it'll help as long as rblocks is buffered (IIRC, it is).
Turns out, it's not buffered: https://github.com/ipfs/go-bitswap/blob/b18a91d6023b83821c72253bbe5e37190db64d63/internal/getter/getter.go#L93
This follows the change in bitswap where that responsibility was removed.
c519ec9 to
4a60416
Compare
|
I added the Put batching, tests to make sure that blockservice does write blocks, and followed the interface changes. Again, we need to release and bubble up the dependency. I think the last remaining point is possibly to buffer the output channel in bitswap: #92 (comment) |
195d8b0 to
f2a4f4f
Compare
…rvice into bitswap-no-add-bs
|
Blocking in MichaelMure#1 |
06f850f to
7374061
Compare
|
The batch loop was too confusing, I rewrote it (hopefully) slightly better. I would like a merge review on 30ca6aa pls. |
Jorropo
left a comment
There was a problem hiding this comment.
LGTM, let's dog food this
LGTM |
This follows the change in bitswap where that responsibility was removed (ipfs/go-bitswap#571).