Skip to content

Conversation

@joschi
Copy link
Member

@joschi joschi commented Jun 29, 2022

It turns out that Jackson Blackbird isn't better/faster on Java 11/17/18 right now and Jackson Afterburner still works.

Let's revert the change from Jackson Afterburner to Blackbird in Dropwizard 2.1.x and keep using it in Dropwizard 3.x and later.

This reverts commit c65d010.
Refs #3803

It turns out that Jackson Blackbird isn't better/faster on Java 11/17/18 right now and Jackson Afterburner still works.

Let's revert the change from Jackson Afterburner to Blackbird in Dropwizard 2.1.x and keep using it in Dropwizard 3.x and later.

This reverts commit c65d010.
Refs #3803
@joschi joschi added this to the 2.1.1 milestone Jun 29, 2022
@joschi joschi requested a review from a team as a code owner June 29, 2022 09:50
@gitpod-io
Copy link

gitpod-io bot commented Jun 29, 2022

@joschi
Copy link
Member Author

joschi commented Jun 29, 2022

I've updated the benchmarks for Jackson Afterburner and Blackbird and ran them with a range of OpenJDK distributions and Java versions.

Surprisingly, Jackson Blackbird never seems to be faster than Jackson Afterburner, at least not on GitHub Actions.

Here are a few benchmark results (only Eclipse Temurin). Keep in mind that only the results within the same VM are comparable, so don't compare the results of Temurin 8 with Temurin 17, etc.

Eclipse Temurin 8u332

Benchmark                           Mode  Cnt    Score    Error   Units
Afterburner.beanArrayDeser         thrpt   25    1.540 ±  0.038  ops/ms
Afterburner.beanArraySer           thrpt   25    2.690 ±  0.053  ops/ms
Afterburner.classicBeanItemDeser   thrpt   25    1.757 ±  0.039  ops/ms
Afterburner.classicBeanItemSer     thrpt   25    3.159 ±  0.153  ops/ms
Afterburner.classicMediaItemDeser  thrpt   25  536.382 ±  8.701  ops/ms
Afterburner.classicMediaItemSer    thrpt   25  736.334 ± 19.504  ops/ms
Afterburner.constructorArrayDeser  thrpt   25    1.303 ±  0.009  ops/ms
Blackbird.beanArrayDeser           thrpt   25    1.509 ±  0.015  ops/ms
Blackbird.beanArraySer             thrpt   25    2.472 ±  0.028  ops/ms
Blackbird.classicBeanItemDeser     thrpt   25    1.600 ±  0.030  ops/ms
Blackbird.classicBeanItemSer       thrpt   25    2.870 ±  0.078  ops/ms
Blackbird.classicMediaItemDeser    thrpt   25  492.964 ± 11.611  ops/ms
Blackbird.classicMediaItemSer      thrpt   25  674.266 ±  8.258  ops/ms
Blackbird.constructorArrayDeser    thrpt   25    1.322 ±  0.014  ops/ms
Vanilla.beanArrayDeser             thrpt   25    1.394 ±  0.014  ops/ms
Vanilla.beanArraySer               thrpt   25    2.309 ±  0.031  ops/ms
Vanilla.classicBeanItemDeser       thrpt   25    1.326 ±  0.021  ops/ms
Vanilla.classicBeanItemSer         thrpt   25    2.399 ±  0.018  ops/ms
Vanilla.classicMediaItemDeser      thrpt   25  426.367 ± 13.226  ops/ms
Vanilla.classicMediaItemSer        thrpt   25  660.151 ± 17.219  ops/ms
Vanilla.constructorArrayDeser      thrpt   25    1.286 ±  0.011  ops/ms
StartupTimeBenchmark.afterburner      ss   50    3.828 ±  0.508   ms/op
StartupTimeBenchmark.blackbird        ss   50    5.475 ±  0.679   ms/op
StartupTimeBenchmark.vanilla          ss   50    2.547 ±  0.244   ms/op

Eclipse Temurin 11.0.15

Benchmark                           Mode  Cnt    Score    Error   Units
Afterburner.beanArrayDeser         thrpt   25    1.276 ±  0.018  ops/ms
Afterburner.beanArraySer           thrpt   25    2.032 ±  0.045  ops/ms
Afterburner.classicBeanItemDeser   thrpt   25    1.327 ±  0.072  ops/ms
Afterburner.classicBeanItemSer     thrpt   25    2.271 ±  0.028  ops/ms
Afterburner.classicMediaItemDeser  thrpt   25  396.335 ± 16.244  ops/ms
Afterburner.classicMediaItemSer    thrpt   25  522.893 ± 17.182  ops/ms
Afterburner.constructorArrayDeser  thrpt   25    1.096 ±  0.017  ops/ms
Blackbird.beanArrayDeser           thrpt   25    1.268 ±  0.044  ops/ms
Blackbird.beanArraySer             thrpt   25    2.028 ±  0.039  ops/ms
Blackbird.classicBeanItemDeser     thrpt   25    1.252 ±  0.017  ops/ms
Blackbird.classicBeanItemSer       thrpt   25    2.159 ±  0.026  ops/ms
Blackbird.classicMediaItemDeser    thrpt   25  383.857 ±  3.025  ops/ms
Blackbird.classicMediaItemSer      thrpt   25  504.182 ±  8.851  ops/ms
Blackbird.constructorArrayDeser    thrpt   25    1.125 ±  0.012  ops/ms
Vanilla.beanArrayDeser             thrpt   25    1.094 ±  0.039  ops/ms
Vanilla.beanArraySer               thrpt   25    1.955 ±  0.069  ops/ms
Vanilla.classicBeanItemDeser       thrpt   25    0.998 ±  0.024  ops/ms
Vanilla.classicBeanItemSer         thrpt   25    1.914 ±  0.038  ops/ms
Vanilla.classicMediaItemDeser      thrpt   25  322.614 ± 12.997  ops/ms
Vanilla.classicMediaItemSer        thrpt   25  449.004 ±  9.866  ops/ms
Vanilla.constructorArrayDeser      thrpt   25    1.021 ±  0.039  ops/ms
StartupTimeBenchmark.afterburner      ss   50    4.463 ±  0.415   ms/op
StartupTimeBenchmark.blackbird        ss   50    9.017 ±  1.800   ms/op
StartupTimeBenchmark.vanilla          ss   50    3.709 ±  0.928   ms/op

Eclipse Temurin 17.0.3

Benchmark                           Mode  Cnt    Score    Error   Units
Afterburner.beanArrayDeser         thrpt   25    1.491 ±  0.010  ops/ms
Afterburner.beanArraySer           thrpt   25    2.595 ±  0.025  ops/ms
Afterburner.classicBeanItemDeser   thrpt   25    1.728 ±  0.012  ops/ms
Afterburner.classicBeanItemSer     thrpt   25    3.035 ±  0.013  ops/ms
Afterburner.classicMediaItemDeser  thrpt   25  470.254 ±  3.325  ops/ms
Afterburner.classicMediaItemSer    thrpt   25  771.092 ± 11.677  ops/ms
Afterburner.constructorArrayDeser  thrpt   25    1.287 ±  0.009  ops/ms
Blackbird.beanArrayDeser           thrpt   25    1.493 ±  0.015  ops/ms
Blackbird.beanArraySer             thrpt   25    2.541 ±  0.039  ops/ms
Blackbird.classicBeanItemDeser     thrpt   25    1.529 ±  0.023  ops/ms
Blackbird.classicBeanItemSer       thrpt   25    2.774 ±  0.025  ops/ms
Blackbird.classicMediaItemDeser    thrpt   25  459.270 ±  4.827  ops/ms
Blackbird.classicMediaItemSer      thrpt   25  743.579 ± 10.982  ops/ms
Blackbird.constructorArrayDeser    thrpt   25    1.304 ±  0.011  ops/ms
Vanilla.beanArrayDeser             thrpt   25    1.272 ±  0.034  ops/ms
Vanilla.beanArraySer               thrpt   25    2.477 ±  0.040  ops/ms
Vanilla.classicBeanItemDeser       thrpt   25    1.174 ±  0.041  ops/ms
Vanilla.classicBeanItemSer         thrpt   25    2.630 ±  0.079  ops/ms
Vanilla.classicMediaItemDeser      thrpt   25  367.201 ± 10.045  ops/ms
Vanilla.classicMediaItemSer        thrpt   25  741.668 ± 11.162  ops/ms
Vanilla.constructorArrayDeser      thrpt   25    1.274 ±  0.019  ops/ms
StartupTimeBenchmark.afterburner      ss   50    5.090 ±  0.119   ms/op
StartupTimeBenchmark.blackbird        ss   50    6.458 ±  1.136   ms/op
StartupTimeBenchmark.vanilla          ss   50    2.669 ±  0.393   ms/op

Eclipse Temurin 18.0.1

Benchmark                           Mode  Cnt    Score    Error   Units
Afterburner.beanArrayDeser         thrpt   25    1.401 ±  0.027  ops/ms
Afterburner.beanArraySer           thrpt   25    2.552 ±  0.012  ops/ms
Afterburner.classicBeanItemDeser   thrpt   25    1.590 ±  0.103  ops/ms
Afterburner.classicBeanItemSer     thrpt   25    2.892 ±  0.033  ops/ms
Afterburner.classicMediaItemDeser  thrpt   25  458.608 ±  3.477  ops/ms
Afterburner.classicMediaItemSer    thrpt   25  756.037 ± 10.573  ops/ms
Afterburner.constructorArrayDeser  thrpt   25    1.280 ±  0.011  ops/ms
Blackbird.beanArrayDeser           thrpt   25    1.437 ±  0.006  ops/ms
Blackbird.beanArraySer             thrpt   25    2.506 ±  0.016  ops/ms
Blackbird.classicBeanItemDeser     thrpt   25    1.474 ±  0.024  ops/ms
Blackbird.classicBeanItemSer       thrpt   25    2.704 ±  0.026  ops/ms
Blackbird.classicMediaItemDeser    thrpt   25  443.932 ±  7.397  ops/ms
Blackbird.classicMediaItemSer      thrpt   25  735.044 ±  6.442  ops/ms
Blackbird.constructorArrayDeser    thrpt   25    1.271 ±  0.015  ops/ms
Vanilla.beanArrayDeser             thrpt   25    1.292 ±  0.006  ops/ms
Vanilla.beanArraySer               thrpt   25    2.305 ±  0.042  ops/ms
Vanilla.classicBeanItemDeser       thrpt   25    1.241 ±  0.021  ops/ms
Vanilla.classicBeanItemSer         thrpt   25    2.344 ±  0.067  ops/ms
Vanilla.classicMediaItemDeser      thrpt   25  381.539 ±  2.080  ops/ms
Vanilla.classicMediaItemSer        thrpt   25  658.118 ±  8.560  ops/ms
Vanilla.constructorArrayDeser      thrpt   25    1.242 ±  0.026  ops/ms
StartupTimeBenchmark.afterburner      ss   50    5.533 ±  0.567   ms/op
StartupTimeBenchmark.blackbird        ss   50    7.535 ±  1.870   ms/op
StartupTimeBenchmark.vanilla          ss   50    2.670 ±  0.342   ms/op

@joschi
Copy link
Member Author

joschi commented Jun 29, 2022

All logs: logs.zip

@zUniQueX
Copy link
Member

If the switch to Blackbird was also meant to encourage users upgrading their JDK, we could make Afterburner opt-in. We could declare an optional dependency on Afterburner and check, if the class is present. If it is not, we can default to Blackbird.

@joschi
Copy link
Member Author

joschi commented Jun 29, 2022

We could declare an optional dependency on Afterburner and check, if the class is present. If it is not, we can default to Blackbird.

Sounds like a sane approach. See updated PR. 😄

@zUniQueX
Copy link
Member

Thanks 👍

@joschi joschi enabled auto-merge (squash) June 29, 2022 13:14
@joschi joschi self-assigned this Jun 29, 2022
@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

100.0% 100.0% Coverage
0.0% 0.0% Duplication

@joschi joschi disabled auto-merge June 30, 2022 15:07
@joschi joschi merged commit f240f99 into release/2.1.x Jun 30, 2022
@joschi joschi deleted the revert-jackson-blackbird branch June 30, 2022 15:08
@mattnelson
Copy link
Contributor

I've updated the benchmarks for Jackson Afterburner and Blackbird and ran them with a range of OpenJDK distributions and Java versions.

Surprisingly, Jackson Blackbird never seems to be faster than Jackson Afterburner, at least not on GitHub Actions.

@joschi Have these results been cross posted in any of the @FasterXML repos? Would like to get @cowtowncoder input here on the recommended usage between the two implementations.

@cowtowncoder
Copy link
Contributor

I think Steven S (author of Blackbird and contributor to Afterburner) could comment, but realistically it both depends on JDK/JVM (BB being relatively faster on newer ones, I think). But there is also the issue that Afterburner may not really work on newer JDKs at all due to stronger limitations.

So I think the choice likely depends on JDK you run things on: with Java 8, Afterburner probably is a better choice, but with 11 and later, Blackbird might be the better choice.

One thing I would suggest is to have means to configure which module -- if either -- should be registered.
There are cases where one or neither works (especially on later JDKs), or, neither provides any benefit: if only using, say, JsonNode based processing there isn't anything either module can do.
I know DropWizard is meant to be opinionated and not to expose many configuration settings, but this seems like it might be something that should be configurable in some way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants