perf(storagenode): estimate the number of batchlets by ijsong · Pull Request #414 · kakao/varlog

ijsong · 2023-04-11T08:04:06Z

What this PR does

This PR estimates the number of batchlets more accurately. Previously, the number of batchlets is
estimated by dividing the size of batches by the smallest batchlet. It is now divided by the right
size of batchlet.

ijsong · 2023-04-11T08:04:20Z

Current dependencies on/for this PR:

main
- PR build(deps): bump go.etcd.io/etcd from v0.0.0-20211015134708-72d3e382e73c to v0.5.0-alpha.5.0.20230414071934-94593e63d45c #444
  - PR perf(storagenode): check log level #411
    - PR perf(storagenode): remove backup from append request #412
      - PR refactor(storagenode): simplify unary RPC logging #413
        
        PR perf(storagenode): estimate the number of batchlets #414 👈
        
        PR chore(metarepos): do not use unnecessary labels #445
        
        PR perf(metarepos): add mrpb.StorageNodeUncommitReport pool #446
        
        PR feat(all): add flags for logger #447
        
        PR refactor(all): use atomic types #448
        PR feat(storagenode): change append rpc from unary to stream #449
        PR fix(storagenode): support partial success/failure for append #450
        PR feat(storagenode): add configurations for initial window size #451
        PR feat(storagenode): add pipelined Append RPC handler #457
        PR feat(client): add LogStreamAppender #459
        PR test(benchmark): support log stream appender #462

This comment was auto-generated by Graphite.

This PR estimates the number of batchlets more accurately. Previously, the number of batchlets is estimated by dividing the size of batches by the smallest batchlet. It is now divided by the right size of batchlet.

ijsong · 2023-06-01T05:26:30Z

@ijsong started a stack merge that includes this pull request via Graphite.

ijsong · 2023-06-01T05:27:15Z

@ijsong merged this pull request with Graphite.

ijsong requested a review from hungryjang as a code owner April 11, 2023 08:04

ijsong self-assigned this Apr 11, 2023

ijsong force-pushed the unary_rpc_logging branch from 51494e9 to d928d2c Compare April 13, 2023 13:23

ijsong force-pushed the append_batchlet_size branch from 546b9b9 to 430808b Compare April 13, 2023 13:23

ijsong force-pushed the unary_rpc_logging branch from d928d2c to 467ee59 Compare April 13, 2023 13:43

ijsong force-pushed the append_batchlet_size branch from 430808b to a329065 Compare April 13, 2023 13:43

hungryjang approved these changes Apr 14, 2023

View reviewed changes

ijsong force-pushed the unary_rpc_logging branch from 467ee59 to a7dd5e9 Compare April 15, 2023 13:18

ijsong force-pushed the append_batchlet_size branch from a329065 to 5c97657 Compare April 15, 2023 13:18

ijsong force-pushed the unary_rpc_logging branch from a7dd5e9 to 12e5586 Compare April 25, 2023 12:13

ijsong force-pushed the append_batchlet_size branch from 5c97657 to c2975b2 Compare April 25, 2023 12:13

ijsong mentioned this pull request May 16, 2023

build(deps): bump go.etcd.io/etcd from v0.0.0-20211015134708-72d3e382e73c to v0.5.0-alpha.5.0.20230414071934-94593e63d45c #444

Merged

ijsong force-pushed the unary_rpc_logging branch from 12e5586 to 017c6d7 Compare May 16, 2023 14:54

ijsong force-pushed the append_batchlet_size branch from c2975b2 to 39fcf2e Compare May 16, 2023 14:54

ijsong marked this pull request as draft May 25, 2023 06:03

ijsong mentioned this pull request May 25, 2023

feat(client): add LogStreamAppender #459

Merged

ijsong marked this pull request as ready for review May 25, 2023 06:08

perf(storagenode): estimate the number of batchlets

3c91b62

This PR estimates the number of batchlets more accurately. Previously, the number of batchlets is estimated by dividing the size of batches by the smallest batchlet. It is now divided by the right size of batchlet.

ijsong force-pushed the unary_rpc_logging branch from 017c6d7 to b0787c4 Compare June 1, 2023 03:31

ijsong force-pushed the append_batchlet_size branch from 39fcf2e to 3c91b62 Compare June 1, 2023 03:31

Base automatically changed from unary_rpc_logging to main June 1, 2023 05:27

ijsong merged commit cb25d19 into main Jun 1, 2023

ijsong deleted the append_batchlet_size branch June 1, 2023 05:27

This was referenced Jun 1, 2023

chore(main): release 0.13.0 #404

Merged

test(benchmark): support log stream appender #462

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

perf(storagenode): estimate the number of batchlets#414

perf(storagenode): estimate the number of batchlets#414
ijsong merged 1 commit intomainfrom
append_batchlet_size

ijsong commented Apr 11, 2023

Uh oh!

ijsong commented Apr 11, 2023 •

edited

Loading

Uh oh!

ijsong commented Jun 1, 2023

Uh oh!

ijsong commented Jun 1, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

ijsong commented Apr 11, 2023

What this PR does

Uh oh!

ijsong commented Apr 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ijsong commented Jun 1, 2023

Uh oh!

ijsong commented Jun 1, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ijsong commented Apr 11, 2023 •

edited

Loading