Favour coroutines over callbacks where possible by uatuko · Pull Request #2 · uatuko/grpcxx

uatuko · 2023-09-25T23:20:27Z

Benchmarks

Scenarios

1a: Single client with no concurrent streams
1b: Single client with 10 concurrent streams
2a: 10 clients with no concurrent streams
2b: 10 clients with 10 concurrent streams
3a: 100 clients with no concurrent streams
3b: 100 clients with 10 concurrent streams

Results (req/s)

⚠️ h2load run with a single thread which potentially limits the throughput to have ~300k req/s ceiling.

	1a	1b	2a	2b	3a	3b
gRPC v1.48.4 (callback)	25k	87k	76k	152k	96k	142k
grpc-go v1.56.2	27k	103k	94k	191k	90k	308k
Baseline (`59be46f`)*	25k	96k	71k	130k	65k	137k
Coroutines, single-threaded (`15b41bc`)	32k	119k	80k	175k	101k	144k
Coroutines with 4 workers (`3f3680f`)	27k	113k	71k	175k	64k	150k
Coroutines with 1 worker (`3f3680f`)	27k	121k	77k	227k	93k	219k
WIP: Coroutines, hardware concurrency (`ba679a2`)	30k	146k	80k	312k	79k	314k
Multi-threaded, hardware concurrency (`4b13fce`)	31k	149k	89k	346k	74k	337k
Multi-threaded, 2 workers (`4b13fce`)	31k	143k	98k	341k	99k	355k

Benchmark details

Benchmarks were run on a MacBook Pro 2021 (M1 Max, 32GB).

[2023-10-29] gRPC v1.48.4

[2023-10-29] gRPC v1.48.4 (callback)

1a. Single client with no concurrent streams

❯ h2load --clients=1 --requests=100000 --header='Content-Type: application/grpc' --header='te: trailers' --data=tmp/helloworld.Greeter_SayHello.request.data http://localhost:50051/helloworld.Greeter/SayHello
starting benchmark...
spawning thread #0: 1 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 4.02s, 24861.84 req/s, 1.61MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 6.49MB (6800301) total, 293.04KB (300075) headers (space savings 96.30%), 2.29MB (2400000) data
                     min         max         mean         sd        +/- sd
time for request:       21us       320us        37us         8us    83.08%
time for connect:      333us       333us       333us         0us   100.00%
time to 1st byte:      677us       677us       677us         0us   100.00%
req/s           :   24861.85    24861.85    24861.85        0.00   100.00%

1b. Single client with 10 concurrent streams

❯ h2load --clients=1 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --header='te: trailers' --data=tmp/helloworld.Greeter_SayHello.request.data http://localhost:50051/helloworld.Greeter/SayHello
starting benchmark...
spawning thread #0: 1 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 1.15s, 87156.98 req/s, 4.70MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.39MB (5655293) total, 293.04KB (300075) headers (space savings 96.30%), 2.29MB (2400000) data
                     min         max         mean         sd        +/- sd
time for request:       37us       649us       108us        28us    81.66%
time for connect:      728us       728us       728us         0us   100.00%
time to 1st byte:     1.40ms      1.40ms      1.40ms         0us   100.00%
req/s           :   87157.11    87157.11    87157.11        0.00   100.00%

2a. 10 clients with no concurrent streams

❯ h2load --clients=10 --requests=100000 --header='Content-Type: application/grpc' --header='te: trailers' --data=tmp/helloworld.Greeter_SayHello.request.data http://localhost:50051/helloworld.Greeter/SayHello
starting benchmark...
spawning thread #0: 10 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 1.31s, 76339.08 req/s, 4.95MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 6.49MB (6802160) total, 293.70KB (300750) headers (space savings 96.29%), 2.29MB (2400000) data
                     min         max         mean         sd        +/- sd
time for request:       22us      1.50ms       117us        48us    74.08%
time for connect:      399us      1.08ms       851us       189us    80.00%
time to 1st byte:     1.69ms      2.42ms      2.14ms       204us    80.00%
req/s           :    7636.89     8192.53     7792.81      181.67    80.00%

2b. 10 clients with 10 concurrent streams

❯ h2load --clients=10 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --header='te: trailers' --data=tmp/helloworld.Greeter_SayHello.request.data http://localhost:50051/helloworld.Greeter/SayHello
starting benchmark...
spawning thread #0: 10 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 655.83ms, 152479.47 req/s, 8.31MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.45MB (5715373) total, 293.70KB (300750) headers (space savings 96.29%), 2.29MB (2400000) data
                     min         max         mean         sd        +/- sd
time for request:       45us      3.41ms       617us       319us    65.22%
time for connect:      476us       901us       715us       133us    70.00%
time to 1st byte:     1.92ms      4.07ms      3.28ms       822us    80.00%
req/s           :   15261.87    16098.41    15620.10      287.66    60.00%

3a. 100 clients with no concurrent streams

❯ h2load --clients=100 --requests=100000 --header='Content-Type: application/grpc' --header='te: trailers' --data=tmp/helloworld.Greeter_SayHello.request.data http://localhost:50051/helloworld.Greeter/SayHello
starting benchmark...
spawning thread #0: 100 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 1.04s, 96331.04 req/s, 6.27MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 6.50MB (6820767) total, 300.29KB (307500) headers (space savings 96.20%), 2.29MB (2400000) data
                     min         max         mean         sd        +/- sd
time for request:       28us     10.53ms       875us       440us    81.18%
time for connect:     2.44ms      3.38ms      3.04ms       234us    66.00%
time to 1st byte:     5.93ms     13.30ms     11.84ms      2.56ms    83.00%
req/s           :     965.96     1121.39     1010.93       43.74    71.00%

3b. 100 clients with 10 concurrent streams

❯ h2load --clients=100 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --header='te: trailers' --data=tmp/helloworld.Greeter_SayHello.request.data http://localhost:50051/helloworld.Greeter/SayHello
❯ h2load --clients=100 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --header='te: trailers' --data=tmp/helloworld.Greeter_SayHello.request.data http://localhost:50051/helloworld.Greeter/SayHello                                                                        23:42:34 
starting benchmark...
spawning thread #0: 100 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 702.86ms, 142275.44 req/s, 7.90MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.56MB (5825005) total, 300.29KB (307500) headers (space savings 96.20%), 2.29MB (2400000) data
                     min         max         mean         sd        +/- sd
time for request:      553us     17.64ms      6.82ms      1.79ms    84.20%
time for connect:     2.87ms      4.42ms      3.83ms       424us    65.00%
time to 1st byte:     7.15ms     21.91ms     17.13ms      5.47ms    76.00%
req/s           :    1429.01     1508.47     1448.63       25.59    76.00%

[2023-11-01] grpc-go v1.56.2

[2023-10-29] grpc-go v1.56.2

💡 Log lines removed before running benchmarks.

1a. Single client with no concurrent streams

❯ h2load --clients=1 --requests=100000 --header='Content-Type: application/grpc' --header='te: trailers' --data=tmp/helloworld.Greeter_SayHello.request.data http://localhost:50051/helloworld.Greeter/SayHello
starting benchmark...
spawning thread #0: 1 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 3.65s, 27376.74 req/s, 2.22MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 8.11MB (8500058) total, 195.32KB (200012) headers (space savings 94.74%), 2.29MB (2400000) data
                     min         max         mean         sd        +/- sd
time for request:       22us       812us        33us         9us    96.00%
time for connect:      666us       666us       666us         0us   100.00%
time to 1st byte:     1.03ms      1.03ms      1.03ms         0us   100.00%
req/s           :   27376.75    27376.75    27376.75        0.00   100.00%

1b. Single client with 10 concurrent streams

❯ h2load --clients=1 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --header='te: trailers' --data=tmp/helloworld.Greeter_SayHello.request.data http://localhost:50051/helloworld.Greeter/SayHello
starting benchmark...
spawning thread #0: 1 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 968.33ms, 103270.79 req/s, 5.75MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.57MB (5837888) total, 195.32KB (200012) headers (space savings 94.74%), 2.29MB (2400000) data
                     min         max         mean         sd        +/- sd
time for request:       24us       511us        90us        30us    83.35%
time for connect:      975us       975us       975us         0us   100.00%
time to 1st byte:     1.45ms      1.45ms      1.45ms         0us   100.00%
req/s           :  103270.95   103270.95   103270.95        0.00   100.00%

2a. 10 clients with no concurrent streams

❯ h2load --clients=10 --requests=100000 --header='Content-Type: application/grpc' --header='te: trailers' --data=tmp/helloworld.Greeter_SayHello.request.data http://localhost:50051/helloworld.Greeter/SayHello
starting benchmark...
spawning thread #0: 10 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 1.07s, 93862.96 req/s, 7.61MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 8.11MB (8500580) total, 195.43KB (200120) headers (space savings 94.73%), 2.29MB (2400000) data
                     min         max         mean         sd        +/- sd
time for request:       21us      3.03ms        88us        63us    93.33%
time for connect:      535us      1.35ms       898us       325us    80.00%
time to 1st byte:     1.08ms      2.12ms      1.63ms       405us    70.00%
req/s           :    9389.22     9450.08     9414.22       21.27    70.00%

2b. 10 clients with 10 concurrent streams

❯ h2load --clients=10 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --header='te: trailers' --data=tmp/helloworld.Greeter_SayHello.request.data http://localhost:50051/helloworld.Greeter/SayHello
starting benchmark...
spawning thread #0: 10 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 522.54ms, 191374.01 req/s, 10.69MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.59MB (5857400) total, 195.43KB (200120) headers (space savings 94.73%), 2.29MB (2400000) data
                     min         max         mean         sd        +/- sd
time for request:       32us      3.23ms       461us       352us    81.45%
time for connect:      517us      1.31ms       898us       287us    60.00%
time to 1st byte:     1.77ms      4.24ms      3.06ms       818us    60.00%
req/s           :   19137.45    19797.83    19360.07      232.97    80.00%

3a. 100 clients with no concurrent streams

❯ h2load --clients=100 --requests=100000 --header='Content-Type: application/grpc' --header='te: trailers' --data=tmp/helloworld.Greeter_SayHello.request.data http://localhost:50051/helloworld.Greeter/SayHello
starting benchmark...
spawning thread #0: 100 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 1.11s, 90239.43 req/s, 7.32MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 8.11MB (8505800) total, 196.48KB (201200) headers (space savings 94.71%), 2.29MB (2400000) data
                     min         max         mean         sd        +/- sd
time for request:       30us      7.72ms       772us       392us    84.41%
time for connect:     3.61ms      6.50ms      5.64ms       906us    78.00%
time to 1st byte:     9.58ms     13.31ms     10.77ms       833us    65.00%
req/s           :     903.75      933.07      910.13        5.23    74.00%

3b. 100 clients with 10 concurrent streams

⚠️ The results are inconsistent. It is possible the single threaded h2load is hitting a ceiling.

❯ h2load --clients=100 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --header='te: trailers' --data=tmp/helloworld.Greeter_SayHello.request.data http://localhost:50051/helloworld.Greeter/SayHello
starting benchmark...
spawning thread #0: 100 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 325.12ms, 307579.69 req/s, 17.05MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.54MB (5813660) total, 196.48KB (201200) headers (space savings 94.71%), 2.29MB (2400000) data
                     min         max         mean         sd        +/- sd
time for request:       54us     20.45ms      2.43ms      1.71ms    83.10%
time for connect:     3.53ms      6.57ms      5.47ms       975us    66.00%
time to 1st byte:     9.41ms     22.54ms     14.05ms      3.19ms    57.00%
req/s           :    3084.89     3548.68     3208.77       96.67    74.00%

[2023-10-26] Baseline

[2023-10-26] Baseline (`59be46f`)

⚠️ Max concurrent streams setting has been adjusted to be 10 for the benchmarks.

grpcxx/lib/grpcxx/h2/conn.cpp

Line 26 in 59be46f

{NGHTTP2_SETTINGS_MAX_CONCURRENT_STREAMS, 8},

1a. Single client with no concurrent streams

❯ h2load --clients=1 --requests=100000 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 1 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 4.06s, 24604.48 req/s, 1.43MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6100749) total, 195.32KB (200012) headers (space savings 94.74%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       25us      3.16ms        38us        13us    97.18%
time for connect:      556us       556us       556us         0us   100.00%
time to 1st byte:     1.07ms      1.07ms      1.07ms         0us   100.00%
req/s           :   24604.48    24604.48    24604.48        0.00   100.00%

1b. Single client with 10 concurrent streams

❯ h2load --clients=1 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 1 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 1.04s, 96377.92 req/s, 5.61MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6100749) total, 195.32KB (200012) headers (space savings 94.74%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       28us       963us        97us        32us    86.46%
time for connect:      832us       832us       832us         0us   100.00%
time to 1st byte:     1.42ms      1.42ms      1.42ms         0us   100.00%
req/s           :   96378.04    96378.04    96378.04        0.00   100.00%

2a. 10 clients with no concurrent streams

❯ h2load --clients=10 --requests=100000 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 10 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 1.40s, 71424.49 req/s, 4.16MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6101120) total, 195.43KB (200120) headers (space savings 94.73%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       35us      2.33ms       125us        43us    85.79%
time for connect:      980us      1.45ms      1.24ms       203us    50.00%
time to 1st byte:     2.54ms      3.51ms      3.05ms       288us    70.00%
req/s           :    7145.11     7166.05     7151.77        6.53    70.00%

2b. 10 clients with 10 concurrent streams

❯ h2load --clients=10 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 10 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 766.72ms, 130425.54 req/s, 7.59MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6101120) total, 195.43KB (200120) headers (space savings 94.73%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:      220us      9.78ms       755us       306us    96.26%
time for connect:      701us      1.11ms       863us       135us    70.00%
time to 1st byte:     2.28ms     10.37ms      7.09ms      3.45ms    70.00%
req/s           :   13051.36    13068.18    13058.30        4.79    70.00%

3a. 100 clients with no concurrent streams

❯ h2load --clients=100 --requests=100000 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 100 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 1.54s, 64973.80 req/s, 3.78MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6104700) total, 196.48KB (201200) headers (space savings 94.71%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:      184us     13.35ms      1.34ms       400us    89.62%
time for connect:     4.77ms      7.15ms      6.09ms       709us    59.00%
time to 1st byte:    12.16ms     19.42ms     15.57ms      2.94ms    42.00%
req/s           :     651.13      653.42      652.29        0.41    69.00%

3b. 100 clients with 10 concurrent streams

❯ h2load --clients=100 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 100 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 731.67ms, 136673.64 req/s, 7.96MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6104700) total, 196.48KB (201200) headers (space savings 94.71%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:      118us    185.60ms      6.72ms      8.19ms    99.35%
time for connect:     4.23ms      9.23ms      7.52ms      1.56ms    71.00%
time to 1st byte:    12.56ms    193.35ms     70.91ms     58.07ms    64.00%
req/s           :    1376.07     1572.76     1471.69       74.92    43.00%

[2023-10-26] Single threaded

[2023-10-26] `15b41bc` (single-threaded)

1a. Single client with no concurrent streams

❯ h2load --clients=1 --requests=100000 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 1 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 3.10s, 32257.74 req/s, 1.88MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6100749) total, 195.32KB (200012) headers (space savings 94.74%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       18us       514us        28us         7us    95.39%
time for connect:      609us       609us       609us         0us   100.00%
time to 1st byte:      968us       968us       968us         0us   100.00%
req/s           :   32257.75    32257.75    32257.75        0.00   100.00%

1b. Single client with 10 concurrent streams

❯ h2load --clients=1 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 1 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 840.13ms, 119029.34 req/s, 6.93MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6100749) total, 195.32KB (200012) headers (space savings 94.74%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       28us      2.21ms        78us        38us    93.52%
time for connect:     1.48ms      1.48ms      1.48ms         0us   100.00%
time to 1st byte:     3.41ms      3.41ms      3.41ms         0us   100.00%
req/s           :  119029.54   119029.54   119029.54        0.00   100.00%

2a. 10 clients with no concurrent streams

❯ h2load --clients=10 --requests=100000 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 10 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 1.24s, 80399.55 req/s, 4.68MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6101120) total, 195.43KB (200120) headers (space savings 94.73%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       30us      8.63ms       104us       100us    99.12%
time for connect:      427us       862us       582us       140us    70.00%
time to 1st byte:     1.33ms      2.28ms      1.75ms       373us    60.00%
req/s           :    8043.04     8080.22     8054.74       10.71    70.00%

2b. 10 clients with 10 concurrent sterams

❯ h2load --clients=10 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 10 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 570.22ms, 175370.91 req/s, 10.20MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6101120) total, 195.43KB (200120) headers (space savings 94.73%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       64us     11.21ms       528us       275us    97.43%
time for connect:      730us      1.93ms      1.23ms       522us    70.00%
time to 1st byte:     3.84ms      8.35ms      5.61ms      1.55ms    60.00%
req/s           :   17548.21    17620.31    17580.31       24.05    60.00%

3a. 100 clients with no concurrent streams

❯ h2load --clients=100 --requests=100000 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 100 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 994.01ms, 100603.12 req/s, 5.86MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6104700) total, 196.48KB (201200) headers (space savings 94.71%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       33us     14.60ms       646us       439us    92.74%
time for connect:     3.24ms      5.82ms      4.01ms       497us    74.00%
time to 1st byte:     7.73ms     17.96ms     14.64ms      4.14ms    71.00%
req/s           :    1009.14     1015.54     1012.06        1.52    68.00%

3b. 100 clients with 10 concurrent streams

❯ h2load --clients=100 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 100 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 693.56ms, 144183.63 req/s, 8.39MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6104700) total, 196.48KB (201200) headers (space savings 94.71%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:      128us    111.66ms      6.13ms      5.11ms    98.38%
time for connect:     4.90ms      9.32ms      7.50ms      1.34ms    59.00%
time to 1st byte:    15.08ms    113.54ms     49.36ms     30.29ms    59.00%
req/s           :    1453.96     1637.70     1513.95       58.51    71.00%

[2023-10-29] Multi-threaded (4 workers)

[2023-10-29] Multi-threaded, 4 workers (`3f3680f`)

1a. Single client with no concurrent streams

❯ h2load --clients=1 --requests=100000 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 1 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 3.70s, 27061.01 req/s, 1.57MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6100749) total, 195.32KB (200012) headers (space savings 94.74%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       24us       971us        34us         9us    95.45%
time for connect:      469us       469us       469us         0us   100.00%
time to 1st byte:      804us       804us       804us         0us   100.00%
req/s           :   27061.01    27061.01    27061.01        0.00   100.00%

1b. Single client with 10 concurrent streams

❯ h2load --clients=1 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 1 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 878.28ms, 113858.78 req/s, 6.62MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6100749) total, 195.32KB (200012) headers (space savings 94.74%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       27us       507us        82us        16us    88.08%
time for connect:      656us       656us       656us         0us   100.00%
time to 1st byte:     1.12ms      1.12ms      1.12ms         0us   100.00%
req/s           :  113858.99   113858.99   113858.99        0.00   100.00%

2a. 10 clients with no concurrent streams

❯ h2load --clients=10 --requests=100000 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 10 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 1.41s, 70703.56 req/s, 4.11MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6101120) total, 195.43KB (200120) headers (space savings 94.73%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       31us      4.75ms       119us        67us    96.48%
time for connect:      327us       589us       443us        86us    70.00%
time to 1st byte:      780us      1.00ms       832us        66us    90.00%
req/s           :    7071.14     7208.95     7132.10       45.87    60.00%

2b. 10 clients with 10 concurrent streams

❯ h2load --clients=10 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 10 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 571.22ms, 175062.98 req/s, 10.19MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6101120) total, 195.43KB (200120) headers (space savings 94.73%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       73us      3.29ms       535us       125us    77.94%
time for connect:      508us       679us       598us        50us    70.00%
time to 1st byte:     1.14ms      2.18ms      1.46ms       346us    80.00%
req/s           :   17512.76    17844.84    17720.36      116.26    60.00%

3a. 100 clients with no concurrent streams

❯ h2load --clients=100 --requests=100000 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 100 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 1.56s, 63970.44 req/s, 3.72MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6104700) total, 196.48KB (201200) headers (space savings 94.71%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       86us      6.73ms      1.04ms       368us    82.81%
time for connect:     3.03ms      5.15ms      4.38ms       624us    65.00%
time to 1st byte:     7.95ms     10.39ms      9.10ms       689us    53.00%
req/s           :     640.41      657.51      647.62        4.12    61.00%

3b. 100 clients with 10 concurrent streams

❯ h2load --clients=100 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 100 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 666.95ms, 149937.18 req/s, 8.73MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6104700) total, 196.48KB (201200) headers (space savings 94.71%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:      129us    126.26ms      5.97ms      5.26ms    99.34%
time for connect:     2.86ms      5.79ms      4.80ms       770us    76.00%
time to 1st byte:     7.54ms    129.85ms     43.91ms     37.84ms    80.00%
req/s           :    1506.12     1699.51     1604.54       70.67    45.00%

[2023-10-29] Multi-threaded (1 worker)

[2023-10-29] Multi-threaded, 1 worker (`3f3680f`)

💡 Worker pool size was controlled using UV_THREADPOOL_SIZE=1 environment variable.

1a. Single client with no concurrent streams

❯ h2load --clients=1 --requests=100000 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 1 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 3.68s, 27190.10 req/s, 1.58MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6100749) total, 195.32KB (200012) headers (space savings 94.74%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       27us      3.91ms        34us        14us    97.74%
time for connect:      527us       527us       527us         0us   100.00%
time to 1st byte:      782us       782us       782us         0us   100.00%
req/s           :   27190.11    27190.11    27190.11        0.00   100.00%

1b. Single client with 10 concurrent streams

❯ h2load --clients=1 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 1 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 827.43ms, 120856.88 req/s, 7.03MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6100749) total, 195.32KB (200012) headers (space savings 94.74%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       27us       305us        77us        16us    87.04%
time for connect:      346us       346us       346us         0us   100.00%
time to 1st byte:      631us       631us       631us         0us   100.00%
req/s           :  120856.97   120856.97   120856.97        0.00   100.00%

2a. 10 clients with no concurrent streams

❯ h2load --clients=10 --requests=100000 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 10 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 1.30s, 77089.35 req/s, 4.49MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6101120) total, 195.43KB (200120) headers (space savings 94.73%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       39us      6.33ms       108us        63us    96.95%
time for connect:      485us       671us       550us        62us    70.00%
time to 1st byte:     1.26ms      1.36ms      1.30ms        31us    70.00%
req/s           :    7710.43     7853.98     7778.15       45.03    60.00%

2b. 10 clients with 10 concurrent streams

❯ h2load --clients=10 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 10 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 440.71ms, 226909.16 req/s, 13.20MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6101120) total, 195.43KB (200120) headers (space savings 94.73%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       49us      3.32ms       409us       110us    85.49%
time for connect:      511us       687us       621us        58us    60.00%
time to 1st byte:      991us      1.34ms      1.23ms       132us    80.00%
req/s           :   22697.85    23212.21    23013.94      175.38    60.00%

3a. 100 clients with no concurrent streams

❯ h2load --clients=100 --requests=100000 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 100 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 1.07s, 93277.49 req/s, 5.43MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6104700) total, 196.48KB (201200) headers (space savings 94.71%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       36us      7.45ms       679us       353us    91.36%
time for connect:     2.76ms      4.21ms      3.64ms       397us    61.00%
time to 1st byte:     8.59ms      9.11ms      8.90ms       154us    66.00%
req/s           :     933.75      961.61      948.78        6.95    52.00%

3b. 100 clients with 10 concurrent streams

❯ h2load --clients=100 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 100 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 455.90ms, 219348.27 req/s, 12.77MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6104700) total, 196.48KB (201200) headers (space savings 94.71%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:      548us     63.76ms      4.06ms      2.35ms    97.55%
time for connect:     2.93ms      4.68ms      4.17ms       497us    79.00%
time to 1st byte:     8.96ms     67.36ms     21.32ms     17.75ms    82.00%
req/s           :    2209.67     2361.59     2303.88       50.21    70.00%

[2023-11-01] WIP: Multi-threaded (hardware concurrency)

[2023-11-01] WIP: Multi-threaded, hardware concurrency (`ba679a2`)

This WIP implementation uses a thread-pool to process connections in parallel. A coroutine is suspended in the main thread and resumed in a worker thread to continue processing. There's a bit of wastage of libuv event loops and a slow client will block a thread, hence marked as WIP.

1a. Single client with no concurrent streams

❯ h2load --clients=1 --requests=100000 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 1 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 3.37s, 29648.54 req/s, 1.72MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6100749) total, 195.32KB (200012) headers (space savings 94.74%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       20us       587us        31us         7us    91.87%
time for connect:      482us       482us       482us         0us   100.00%
time to 1st byte:     1.08ms      1.08ms      1.08ms         0us   100.00%
req/s           :   29648.55    29648.55    29648.55        0.00   100.00%

1b. Single client with 10 concurrent streams

❯ h2load --clients=1 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 1 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 686.78ms, 145606.61 req/s, 8.47MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6100749) total, 195.32KB (200012) headers (space savings 94.74%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       20us       379us        63us        21us    73.52%
time for connect:      393us       393us       393us         0us   100.00%
time to 1st byte:      741us       741us       741us         0us   100.00%
req/s           :  145606.82   145606.82   145606.82        0.00   100.00%

2a. 10 clients with no concurrent streams

❯ h2load --clients=10 --requests=100000 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 10 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 1.24s, 80362.60 req/s, 4.68MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6101120) total, 195.43KB (200120) headers (space savings 94.73%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       22us       620us        99us        32us    76.19%
time for connect:      409us       674us       516us       106us    70.00%
time to 1st byte:      940us      1.10ms      1.04ms        45us    70.00%
req/s           :    8038.86     8063.22     8048.16        8.20    70.00%

2b. 10 clients with 10 concurrent streams

❯ h2load --clients=10 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 10 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 320.67ms, 311849.99 req/s, 18.14MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6101120) total, 195.43KB (200120) headers (space savings 94.73%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       21us       521us       222us        54us    70.24%
time for connect:      412us       675us       524us       100us    70.00%
time to 1st byte:      892us      1.07ms       954us        58us    70.00%
req/s           :   31211.77    31461.69    31292.45       86.03    80.00%

3a. 100 clients with no concurrent streams

❯ h2load --clients=100 --requests=100000 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 100 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 1.27s, 78631.50 req/s, 4.58MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6104700) total, 196.48KB (201200) headers (space savings 94.71%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       18us       1.13s       657us     20.95ms    99.91%
time for connect:     2.06ms      4.68ms      3.92ms       852us    76.00%
time to 1st byte:     7.53ms       1.14s    560.63ms    362.19ms    60.00%
req/s           :     787.91     8055.84     2362.10     2120.89    90.00%

3b. 100 clients with 10 concurrent streams

❯ h2load --clients=100 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 100 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 318.12ms, 314348.76 req/s, 18.30MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6104700) total, 196.48KB (201200) headers (space savings 94.71%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       24us    279.09ms      1.60ms     16.34ms    99.10%
time for connect:     2.11ms      6.45ms      5.32ms      1.45ms    80.00%
time to 1st byte:     8.39ms    285.24ms    144.71ms     88.56ms    60.00%
req/s           :    3168.19    25933.81     8532.16     6644.92    86.00%

[2023-11-06] Multi-threaded (hardware concurrency)

[2023-11-06] Multi-threaded, hardware concurrency (`4b13fce`)

1a. Single client with no concurrent streams

❯ h2load --clients=1 --requests=100000 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 1 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 3.19s, 31379.50 req/s, 1.83MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6100749) total, 195.32KB (200012) headers (space savings 94.74%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       20us       352us        29us         6us    90.90%
time for connect:     1.20ms      1.20ms      1.20ms         0us   100.00%
time to 1st byte:     1.68ms      1.68ms      1.68ms         0us   100.00%
req/s           :   31379.50    31379.50    31379.50        0.00   100.00%

1b. Single client with 10 concurrent streams

❯ h2load --clients=1 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 1 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 672.22ms, 148760.60 req/s, 8.66MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6100749) total, 195.32KB (200012) headers (space savings 94.74%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       20us       395us        61us        21us    78.95%
time for connect:      226us       226us       226us         0us   100.00%
time to 1st byte:      453us       453us       453us         0us   100.00%
req/s           :  148760.81   148760.81   148760.81        0.00   100.00%

2a. 10 clients with no concurrent streams

❯ h2load --clients=10 --requests=100000 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 10 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 1.13s, 88525.97 req/s, 5.15MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6101120) total, 195.43KB (200120) headers (space savings 94.73%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       22us       523us        79us        29us    88.30%
time for connect:      343us       530us       404us        59us    70.00%
time to 1st byte:      807us       972us       869us        62us    60.00%
req/s           :    8855.37     8873.39     8861.61        4.92    80.00%

2b. 10 clients with 10 concurrent streams

❯ h2load --clients=10 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 10 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 288.88ms, 346162.10 req/s, 20.14MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6101120) total, 195.43KB (200120) headers (space savings 94.73%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       22us      1.58ms       201us        82us    83.30%
time for connect:      309us       468us       400us        50us    60.00%
time to 1st byte:      704us       924us       815us        57us    80.00%
req/s           :   34636.82    35054.96    34812.69      141.24    60.00%

3a. 100 clients with no concurrent streams

❯ h2load --clients=100 --requests=100000 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 100 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 1.35s, 74225.44 req/s, 4.32MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6104700) total, 196.48KB (201200) headers (space savings 94.71%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       31us      7.46ms       883us       281us    85.02%
time for connect:     1.93ms      4.40ms      3.84ms       763us    82.00%
time to 1st byte:     7.55ms     10.10ms      8.98ms       608us    73.00%
req/s           :     742.40      745.54      743.51        0.59    65.00%

3b. 100 clients with 10 concurrent streams

❯ h2load --clients=100 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 100 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 296.78ms, 336947.66 req/s, 19.62MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6104700) total, 196.48KB (201200) headers (space savings 94.71%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       57us      6.57ms      1.75ms       525us    86.38%
time for connect:     1.60ms      3.56ms      2.95ms       617us    78.00%
time to 1st byte:     6.01ms      8.73ms      7.20ms       606us    64.00%
req/s           :    3374.01     3457.82     3392.41       12.68    80.00%

[2023-11-06] Multi-threaded (2 workers)

[2023-11-06] Multi-threaded, 2 workers (`4b13fce`)

💡 Worker count was set by passing in the number of workers to grpcxx::server{}.

grpcxx/examples/helloworld/main.cpp

Line 57 in 4b13fce

grpcxx::server server;

1a.

❯ h2load --clients=1 --requests=100000 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 1 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done 
progress: 50% done
progress: 60% done                         
progress: 70% done                                                                                                                                             
progress: 80% done                                                             
progress: 90% done                                                                                                                                             
progress: 100% done                                                                                                                                            
                                                                               
finished in 3.21s, 31175.40 req/s, 1.81MB/s                                                                                                                    
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx                             
traffic: 5.82MB (6100749) total, 195.32KB (200012) headers (space savings 94.74%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       19us       465us        29us         6us    93.64%
time for connect:      358us       358us       358us         0us   100.00%
time to 1st byte:      465us       465us       465us         0us   100.00%
req/s           :   31175.41    31175.41    31175.41        0.00   100.00%

1b.

❯ h2load --clients=1 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 1 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 698.10ms, 143246.98 req/s, 8.33MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6100749) total, 195.32KB (200012) headers (space savings 94.74%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       22us       414us        63us        19us    76.10%
time for connect:      375us       375us       375us         0us   100.00%
time to 1st byte:      553us       553us       553us         0us   100.00%
req/s           :  143247.23   143247.23   143247.23        0.00   100.00%

2a.

❯ h2load --clients=10 --requests=100000 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 10 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 1.02s, 97963.63 req/s, 5.70MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6101120) total, 195.43KB (200120) headers (space savings 94.73%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       22us       448us        70us        23us    87.33%
time for connect:      339us       538us       433us        80us    60.00%
time to 1st byte:      764us       918us       830us        54us    60.00%
req/s           :    9799.54     9813.92     9805.44        5.60    70.00%

2b.

❯ h2load --clients=10 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 10 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 292.86ms, 341464.75 req/s, 19.87MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6101120) total, 195.43KB (200120) headers (space savings 94.73%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       28us       773us       262us        93us    66.60%
time for connect:      428us       612us       515us        71us    50.00%
time to 1st byte:      779us       999us       919us        68us    70.00%
req/s           :   34182.06    35472.82    34809.87      511.17    40.00%

3a.

❯ h2load --clients=100 --requests=100000 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello
starting benchmark...
spawning thread #0: 100 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done 
progress: 80% done
progress: 90% done                             
progress: 100% done                                                                                                                                            
                                                                               
finished in 1.01s, 99366.54 req/s, 5.79MB/s                                                                                                                    
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx                             
traffic: 5.82MB (6104700) total, 196.48KB (201200) headers (space savings 94.71%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       51us      6.22ms       586us       292us    91.79%
time for connect:     2.13ms      3.95ms      3.40ms       551us    81.00%
time to 1st byte:     8.24ms      9.51ms      8.95ms       314us    70.00%
req/s           :     993.78      999.29      995.83        1.01    71.00%

3b.

❯ h2load --clients=100 --requests=100000 --max-concurrent-streams=10 --header='Content-Type: application/grpc' --data=examples/helloworld/testdata/hello.grpc-lpm.data http://localhost:7000/helloworld.v1.Greeter/Hello 
starting benchmark...
spawning thread #0: 100 total client(s). 100000 total requests
Application protocol: h2c
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done

finished in 281.56ms, 355161.56 req/s, 20.68MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 5.82MB (6104700) total, 196.48KB (201200) headers (space savings 94.71%), 2.96MB (3100000) data
                     min         max         mean         sd        +/- sd
time for request:       46us      8.16ms      2.42ms       824us    74.75%
time for connect:     1.67ms      3.57ms      3.02ms       575us    80.00%
time to 1st byte:     6.29ms     11.71ms      7.25ms      1.34ms    90.00%
req/s           :    3559.36     4326.89     3758.05      169.65    68.00%

uatuko added 22 commits September 26, 2023 00:17

(grpcxx) crude connection handling using coroutines

9c334d0

(grpcxx) crude h2 session handling

9eeed21

update makefile

5f72e05

update lint rules

42ac116

(grpcxx) tidy-up cmake files

1af4da1

(grpcxx/h2) tidy-up includes

083bc85

(grpcxx) crude request structure

5b65658

(grpcxx/h2) emit end of stream events

a3578ba

(grpcxx) tidy-up

3002707

(grpcxx) rpc messages from h2 data frames

89d75db

(grpcxx) handle client connection and h2 session errors

ddd9c9d

(grpcxx) tidy-up

77c9b44

(grpcxx/h2) tidy-up

7bf5d18

(grpcxx) drop callback for sending h2 session data

ca2b9f3

(grpcxx) working single-threaded server using coroutines

15b41bc

(grpcxx) tidy-up

00dc552

(grpcxx) process requests in parallel using libuv worker threads

3f3680f

(grpcxx) tidy-up

979d1e9

(grpcxx) WIP: quick and dirty thread-pool for parallel processing

ba679a2

(grpcxx) process requests in parallel using thread/worker pool

324cf60

(grpcxx) tidy-up

4b13fce

(grpcxx) tidy-up

fcbc628

uatuko marked this pull request as ready for review November 6, 2023 18:50

uatuko added 3 commits November 7, 2023 21:26

(grpcxx) tidy-up

4d878d1

(grpcxx) tidy-up

8eaad0f

(grpcxx) tidy-up

c140287

uatuko merged commit de91823 into main Nov 10, 2023

uatuko deleted the feature/coroutines branch November 10, 2023 13:38

This was referenced Jan 12, 2024

Fix to avoid idle connections from blocking worker threads #20

Merged

Rust benchmarks #21

Closed

uatuko mentioned this pull request Feb 2, 2024

v0.2.0 benchmark results #25

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Favour coroutines over callbacks where possible#2

Favour coroutines over callbacks where possible#2
uatuko merged 25 commits intomainfrom
feature/coroutines

uatuko commented Sep 25, 2023 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

uatuko commented Sep 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Scenarios

Results (req/s)

Benchmark details

[2023-10-29] gRPC v1.48.4 (callback)

1a. Single client with no concurrent streams

1b. Single client with 10 concurrent streams

2a. 10 clients with no concurrent streams

2b. 10 clients with 10 concurrent streams

3a. 100 clients with no concurrent streams

3b. 100 clients with 10 concurrent streams

[2023-10-29] grpc-go v1.56.2

1a. Single client with no concurrent streams

1b. Single client with 10 concurrent streams

2a. 10 clients with no concurrent streams

2b. 10 clients with 10 concurrent streams

3a. 100 clients with no concurrent streams

3b. 100 clients with 10 concurrent streams

[2023-10-26] Baseline (59be46f)

1a. Single client with no concurrent streams

1b. Single client with 10 concurrent streams

2a. 10 clients with no concurrent streams

2b. 10 clients with 10 concurrent streams

3a. 100 clients with no concurrent streams

3b. 100 clients with 10 concurrent streams

[2023-10-26] 15b41bc (single-threaded)

1a. Single client with no concurrent streams

1b. Single client with 10 concurrent streams

2a. 10 clients with no concurrent streams

2b. 10 clients with 10 concurrent sterams

3a. 100 clients with no concurrent streams

3b. 100 clients with 10 concurrent streams

[2023-10-29] Multi-threaded, 4 workers (3f3680f)

1a. Single client with no concurrent streams

1b. Single client with 10 concurrent streams

2a. 10 clients with no concurrent streams

2b. 10 clients with 10 concurrent streams

3a. 100 clients with no concurrent streams

3b. 100 clients with 10 concurrent streams

[2023-10-29] Multi-threaded, 1 worker (3f3680f)

1a. Single client with no concurrent streams

1b. Single client with 10 concurrent streams

2a. 10 clients with no concurrent streams

2b. 10 clients with 10 concurrent streams

3a. 100 clients with no concurrent streams

3b. 100 clients with 10 concurrent streams

[2023-11-01] WIP: Multi-threaded, hardware concurrency (ba679a2)

1a. Single client with no concurrent streams

1b. Single client with 10 concurrent streams

2a. 10 clients with no concurrent streams

2b. 10 clients with 10 concurrent streams

3a. 100 clients with no concurrent streams

3b. 100 clients with 10 concurrent streams

[2023-11-06] Multi-threaded, hardware concurrency (4b13fce)

1a. Single client with no concurrent streams

1b. Single client with 10 concurrent streams

2a. 10 clients with no concurrent streams

2b. 10 clients with 10 concurrent streams

3a. 100 clients with no concurrent streams

3b. 100 clients with 10 concurrent streams

[2023-11-06] Multi-threaded, 2 workers (4b13fce)

1a.

1b.

2a.

2b.

3a.

3b.

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

uatuko commented Sep 25, 2023 •

edited

Loading

[2023-10-26] Baseline (`59be46f`)

[2023-10-26] `15b41bc` (single-threaded)

[2023-10-29] Multi-threaded, 4 workers (`3f3680f`)

[2023-10-29] Multi-threaded, 1 worker (`3f3680f`)

[2023-11-01] WIP: Multi-threaded, hardware concurrency (`ba679a2`)

[2023-11-06] Multi-threaded, hardware concurrency (`4b13fce`)

[2023-11-06] Multi-threaded, 2 workers (`4b13fce`)