zio-http bench by hearnadam · Pull Request #406 · getkyo/kyo

hearnadam · 2024-05-22T04:48:45Z

This is the only way I can think of to initialize a zio-http client once... the zlayer -> runtime mechanism is not type safe, but I don't expect it to be widely used.

Hopefully I fixed the resource leak I had in #354

/resolves #329
/claim #329

fwbrasil · 2024-05-22T05:11:58Z

Can you post the HTTP benchmark results? It might help show if there's a leak.

hearnadam · 2024-05-22T05:35:37Z

As discussed offline, there may be a leak. I think zio-http needs Scope to properly shutdown. Is there an easy way to add a shutdown hook to jmh?

[info] <JMH had finished, but forked VM did not exit, are there stray running threads? Waiting 9 seconds more...>
[info] Non-finished threads:
[info] Thread[#55,KQueueEventLoopGroup-2-6,10,main]
[info]   at app//io.netty.channel.kqueue.Native.keventWait(Native Method)
[info]   at app//io.netty.channel.kqueue.Native.keventWait(Native.java:124)
[info]   at app//io.netty.channel.kqueue.KQueueEventLoop.kqueueWait(KQueueEventLoop.java:184)
[info]   at app//io.netty.channel.kqueue.KQueueEventLoop.kqueueWait(KQueueEventLoop.java:176)
[info]   at app//io.netty.channel.kqueue.KQueueEventLoop.run(KQueueEventLoop.java:245)
[info]   at app//io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
[info]   at app//io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
[info]   at app//io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
[info]   at java.base@21.0.2/java.lang.Thread.runWith(Thread.java:1596)
[info]   at java.base@21.0.2/java.lang.Thread.run(Thread.java:1583)

I've seen the KQueueEventLoopGroup error happen previously when using a fixed connection pool (which the default does)... Not sure if there is a fix we can put in place here.

[info] Benchmark                                     Mode  Cnt       Score      Error   Units
[info] HttpClientBench.forkCats                     thrpt    5    8845.799 ± 3583.115   ops/s
[info] HttpClientBench.forkCats:gc.alloc.rate       thrpt    5     698.909 ±  284.070  MB/sec
[info] HttpClientBench.forkCats:gc.alloc.rate.norm  thrpt    5   82942.747 ±   51.667    B/op
[info] HttpClientBench.forkCats:gc.count            thrpt    5      39.000             counts
[info] HttpClientBench.forkCats:gc.time             thrpt    5      51.000                 ms
[info] HttpClientBench.forkKyo                      thrpt    5   12920.627 ± 4977.834   ops/s
[info] HttpClientBench.forkKyo:gc.alloc.rate        thrpt    5     647.318 ±  249.423  MB/sec
[info] HttpClientBench.forkKyo:gc.alloc.rate.norm   thrpt    5   52540.361 ±   10.817    B/op
[info] HttpClientBench.forkKyo:gc.count             thrpt    5      20.000             counts
[info] HttpClientBench.forkKyo:gc.time              thrpt    5      27.000                 ms
[info] HttpClientBench.forkZio                      thrpt    5    2318.310 ±  979.256   ops/s
[info] HttpClientBench.forkZio:gc.alloc.rate        thrpt    5     594.874 ±  252.083  MB/sec
[info] HttpClientBench.forkZio:gc.alloc.rate.norm   thrpt    5  269112.004 ± 1992.527    B/op
[info] HttpClientBench.forkZio:gc.count             thrpt    5      14.000             counts
[info] HttpClientBench.forkZio:gc.time              thrpt    5     228.000                 ms

The client is being reused 🙂

fwbrasil · 2024-05-22T15:23:24Z

kyo-bench/shared/main/scala/kyo/bench/Bench.scala

+            else
+                zio.Runtime.default.unsafe
        else
            KyoSchedulerZioRuntime.default.unsafe


Would it be possible to support the case when Kyo's scheduler is enabled for ZIO?

With the layer mechanism? Definitely

I just pushed a change to do so, but I realized that the override val zioRuntimeLayer = zio.http.Client.default will hide the fact that the kyo scheduler is not being used. We would need to write:

override def zioRuntimeLayer = super.zioRuntimeLayer ++ zio.http.Client.default

I think it could be a def since zioRuntime is a lazy val and will call zioRuntimeLayer only once. Don't you need to handle the else branch here when Kyo's scheduler is enabled?

You are correct, we need to ensure the kyo scheduler is combined with the layers in the child bench. This is fixed in my latest commit.

hearnadam · 2024-05-22T15:51:19Z

The updated code with finalizers fixes the shutdown problem. Netty still throws some random errors intermittently - though I don't know if we can fix that.

jmh:clean;kyo-bench/jmh:run -wi 15 -i 5 -r 1 -w 1 -f 1 -t 1 -foe true Http

[info] Benchmark                                Mode  Cnt      Score      Error  Units
[info] HttpClientBench.forkCats                thrpt    5  10517.030 ± 1979.995  ops/s
[info] HttpClientBench.forkKyo                 thrpt    5  15953.661 ± 1315.121  ops/s
[info] HttpClientBench.forkZio                 thrpt    5   2377.328 ± 2547.156  ops/s
[info] HttpClientContentionBench.forkCats      thrpt    5   3752.692 ± 1254.563  ops/s
[info] HttpClientContentionBench.forkKyo       thrpt    5   5347.494 ± 1066.285  ops/s
[info] HttpClientContentionBench.forkZio       thrpt    5     14.925 ±   18.440  ops/s
[info] HttpClientRaceContentionBench.forkCats  thrpt    5      1.937 ±    4.417  ops/s
[info] HttpClientRaceContentionBench.forkKyo   thrpt    5     63.564 ±  280.880  ops/s
[info] HttpClientRaceContentionBench.forkZio   thrpt    5     51.870 ±   72.368  ops/s

fwbrasil · 2024-05-22T17:41:47Z

Netty still throws some random errors intermittently - though I don't know if we can fix that.

Are the errors after the benchmark execution? It should be ok if that's the case.

hearnadam

jmh:clean;kyo-bench/jmh:run -wi 15 -i 5 -r 1 -w 1 -f 1 -t 1 -foe true Http

[info] Benchmark                                Mode  Cnt      Score      Error  Units
[info] HttpClientBench.forkCats                thrpt    5   8371.550 ± 1340.961  ops/s
[info] HttpClientBench.forkKyo                 thrpt    5  10806.917 ± 5024.439  ops/s
[info] HttpClientBench.forkZio                 thrpt    5   1429.425 ± 1310.774  ops/s
[info] HttpClientContentionBench.forkCats      thrpt    5   3147.521 ±  787.903  ops/s
[info] HttpClientContentionBench.forkKyo       thrpt    5   3686.940 ±  981.584  ops/s
[info] HttpClientContentionBench.forkZio       thrpt    5    695.971 ±  616.321  ops/s
[info] HttpClientRaceContentionBench.forkCats  thrpt    5      1.539 ±    1.978  ops/s
[info] HttpClientRaceContentionBench.forkKyo   thrpt    5     70.515 ±  227.973  ops/s
[info] HttpClientRaceContentionBench.forkZio   thrpt    5     58.362 ±   59.659  ops/s

Still seeing errors like:

[error] May 22, 2024 9:15:31 PM io.netty.channel.kqueue.KQueueEventLoop processReady
[error] WARNING: events[0]=[153, -1] had no channel!

But that may be a bug/issue with zio-http.

hearnadam · 2024-05-23T04:14:34Z

kyo-bench/src/main/scala/kyo/bench/HttpClientBench.scala


 class HttpClientBench extends Bench.ForkOnly("pong"):

+    override val zioRuntimeLayer = super.zioRuntimeLayer.merge(zio.http.Client.default)


theoretically I think we could make the parent class do this, but we would need to make Bench accept an R parameter. Adding to a ZEnvironment will need a tag for the type representation.

Let me know what you think @fwbrasil

hearnadam · 2024-05-23T04:14:56Z

kyo-bench/src/main/scala/kyo/bench/ZIORuntime.scala

+extension (parent: ZLayer[Any, Any, Any])
+    def merge[R](child: ZLayer[Any, Any, R])(using Tag[R]): ZLayer[Any, Any, R] =


Are you okay with this being in this file?

algora-pbc bot mentioned this pull request May 22, 2024

Implement HTTP Client Benchmarks for ZIO #329

Closed

algora-pbc bot added the 🙋 Bounty claim label May 22, 2024

fwbrasil reviewed May 22, 2024

View reviewed changes

hearnadam added 5 commits May 22, 2024 19:45

Introduce generalized ZLayer -> zio.Runtime mechanism

b03c4f7

Introduce zio-http bench

7c6cd7c

add finalizer mechanism to shutdown zio-http client

b7ac18e

avoid FiberFailure from prematurely ending race

b4a4a66

update HttpClientContentionBench to share client layer

76f6aea

hearnadam force-pushed the zio-http-bench branch from d8e853c to 05b17da Compare May 23, 2024 04:06

set kyo-scheduler via layer

8822569

hearnadam force-pushed the zio-http-bench branch from 05b17da to 8822569 Compare May 23, 2024 04:10

hearnadam commented May 23, 2024

View reviewed changes

fwbrasil merged commit 19053ba into getkyo:main May 23, 2024


		class HttpClientBench extends Bench.ForkOnly("pong"):

		override val zioRuntimeLayer = super.zioRuntimeLayer.merge(zio.http.Client.default)

		extension (parent: ZLayer[Any, Any, Any])
		def merge[R](child: ZLayer[Any, Any, R])(using Tag[R]): ZLayer[Any, Any, R] =

Conversation

hearnadam commented May 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fwbrasil commented May 22, 2024

Uh oh!

hearnadam commented May 22, 2024

Uh oh!

fwbrasil May 22, 2024

Choose a reason for hiding this comment

Uh oh!

hearnadam May 22, 2024

Choose a reason for hiding this comment

Uh oh!

hearnadam May 22, 2024

Choose a reason for hiding this comment

Uh oh!

fwbrasil May 22, 2024

Choose a reason for hiding this comment

Uh oh!

hearnadam May 23, 2024

Choose a reason for hiding this comment

Uh oh!

hearnadam commented May 22, 2024

Uh oh!

fwbrasil commented May 22, 2024

Uh oh!

hearnadam left a comment

Choose a reason for hiding this comment

Uh oh!

hearnadam May 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hearnadam May 23, 2024

Choose a reason for hiding this comment

Uh oh!

fwbrasil May 23, 2024

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

hearnadam commented May 22, 2024 •

edited

Loading

hearnadam May 23, 2024 •

edited

Loading