Skip to content

add benchmark test for baggage#4468

Merged
xrmx merged 2 commits intoopen-telemetry:mainfrom
emdneto:benchmark-baggage
Mar 11, 2025
Merged

add benchmark test for baggage#4468
xrmx merged 2 commits intoopen-telemetry:mainfrom
emdneto:benchmark-baggage

Conversation

@emdneto
Copy link
Copy Markdown
Member

@emdneto emdneto commented Mar 10, 2025 •
edited
Loading

To help show outcomes from #4466

$ tox -f benchmark-sdk -- -k 'baggage'

opentelemetry-sdk/benchmarks/test_baggage.py::test_set_baggage[10] PASSED                                                                                                                      [  6%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_set_baggage[100] PASSED                                                                                                                     [ 12%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_set_baggage[1000] PASSED                                                                                                                    [ 18%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_set_baggage[10000] PASSED                                                                                                                   [ 25%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_get_baggage[10] PASSED                                                                                                                      [ 31%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_get_baggage[100] PASSED                                                                                                                     [ 37%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_get_baggage[1000] PASSED                                                                                                                    [ 43%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_get_baggage[10000] PASSED                                                                                                                   [ 50%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_remove_baggage[10] PASSED                                                                                                                   [ 56%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_remove_baggage[100] PASSED                                                                                                                  [ 62%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_remove_baggage[1000] PASSED                                                                                                                 [ 68%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_remove_baggage[10000] PASSED                                                                                                                [ 75%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_clear_baggage[10] PASSED                                                                                                                    [ 81%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_clear_baggage[100] PASSED                                                                                                                   [ 87%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_clear_baggage[1000] PASSED                                                                                                                  [ 93%]
opentelemetry-sdk/benchmarks/test_baggage.py::test_clear_baggage[10000] PASSED                                                                                                                 [100%]

I'm adding the test under opentelemetry-sdk benchmarks because we have only one job to run benchmarks and it's tailored to SDK. For now, I think it's more important to have the benchmark published.

@emdneto
add benchmark test for baggage
8422e8f 
Signed-off-by: emdneto <9735060+emdneto@users.noreply.github.com>
@emdneto emdneto requested a review from a team as a code owner March 10, 2025 14:49
@emdneto emdneto added the Skip Changelog PRs that do not require a CHANGELOG.md entry label Mar 10, 2025
@emdneto emdneto mentioned this pull request Mar 10, 2025
Merged
4 tasks
@emdneto
ignore pylint
17b9b63 
Signed-off-by: emdneto <9735060+emdneto@users.noreply.github.com>
tammy-baylis-swi
tammy-baylis-swi approved these changes Mar 10, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@tammy-baylis-swi tammy-baylis-swi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great! Is there a way to run that would also show the times each took?

@emdneto
Copy link
Copy Markdown
Member Author

emdneto commented Mar 10, 2025 •
edited
Loading

This is great! Is there a way to run that would also show the times each took?

Benchmarks are published here you can control+f for the specific benchmark you want. Most recent benchmarks are at the very bottom of the page.

Locally you can have results like this: tox -f benchmark-sdk -- -k 'baggage'

------------------------------------------------------------------------------------------------------------------------ benchmark: 16 tests ------------------------------------------------------------------------------------------------------------------------
Name (time in ns)                             Min                           Max                          Mean                      StdDev                        Median                         IQR            Outliers             OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_clear_baggage[10]                   166.2411 (1.0)            328,541.2677 (127.61)             294.9771 (1.37)           1,357.0462 (62.34)              250.0601 (1.19)              41.9095 (12.13)     80;8539  3,390,093.7895 (0.73)     133335           1
test_clear_baggage[100]                  203.4118 (1.22)             8,029.3976 (3.12)               217.3468 (1.01)              43.1534 (1.98)               213.2181 (1.01)               4.8757 (1.41)   7856;17200  4,600,942.2890 (0.99)     196693          17
test_clear_baggage[10000]                204.8716 (1.23)             2,574.6607 (1.0)                215.8921 (1.0)               21.7683 (1.0)                211.8371 (1.00)               3.4537 (1.0)    7847;16647  4,631,942.3873 (1.0)      198327          24
test_clear_baggage[1000]                 207.0796 (1.25)           127,797.4993 (49.64)              230.9329 (1.07)             664.0449 (30.51)              210.8281 (1.0)                6.6590 (1.93)     140;6143  4,330,262.0387 (0.93)      46785         100
test_get_baggage[10]                   2,520.3917 (15.16)       10,833,708.1037 (>1000.0)          3,061.5884 (14.18)         34,318.6225 (>1000.0)          2,645.8874 (12.55)             62.6314 (18.13)   163;12093    326,627.8431 (0.07)     177787           2
test_remove_baggage[10]                6,957.9110 (41.85)          125,250.3134 (48.65)            7,398.8153 (34.27)          1,221.1860 (56.10)            7,207.9711 (34.19)            125.2629 (36.27)   4902;9242    135,156.7735 (0.03)      86648           1
test_set_baggage[10]                   7,874.7980 (47.37)          278,374.6459 (108.12)           9,565.3656 (44.31)          6,335.4588 (291.04)           8,833.1290 (41.90)            208.1506 (60.27)    324;1934    104,543.8343 (0.02)      13575           1
test_get_baggage[100]                 24,457.9278 (147.12)       4,219,542.2575 (>1000.0)         26,085.4991 (120.83)        29,431.0285 (>1000.0)         25,167.1299 (119.37)           291.0383 (84.27)    193;3925     38,335.4751 (0.01)      36091           1
test_set_baggage[100]                179,375.0562 (>1000.0)      1,242,415.9795 (482.56)         190,795.0193 (883.75)        22,146.5954 (>1000.0)        185,916.6659 (881.84)         7,458.0312 (>1000.0)   105;279      5,241.2270 (0.00)       4362           1
test_remove_baggage[100]             199,040.8637 (>1000.0)        602,959.1896 (234.19)         203,913.0065 (944.51)        10,649.5800 (489.22)         201,249.9608 (954.57)         3,542.7511 (>1000.0)   259;425      4,904.0521 (0.00)       4402           1
test_get_baggage[1000]               251,374.6731 (>1000.0)      2,801,582.6829 (>1000.0)        266,384.3175 (>1000.0)       91,680.8942 (>1000.0)        254,790.7643 (>1000.0)        4,249.8577 (>1000.0)    41;637      3,753.9747 (0.00)       3861           1
test_get_baggage[10000]            2,591,624.8560 (>1000.0)      2,729,250.1181 (>1000.0)      2,627,058.8946 (>1000.0)       19,690.7752 (904.56)       2,623,104.0247 (>1000.0)       24,249.7772 (>1000.0)      96;8        380.6538 (0.00)        378           1
test_set_baggage[1000]            13,605,582.9935 (>1000.0)     14,368,624.8921 (>1000.0)     13,724,878.6769 (>1000.0)      126,854.0544 (>1000.0)     13,685,083.4824 (>1000.0)       91,875.4376 (>1000.0)       6;4         72.8604 (0.00)         68           1
test_remove_baggage[1000]         14,673,875.1791 (>1000.0)     15,584,084.2053 (>1000.0)     14,778,549.0365 (>1000.0)      123,663.6792 (>1000.0)     14,758,416.4515 (>1000.0)       89,457.9571 (>1000.0)       4;2         67.6656 (0.00)         68           1
test_set_baggage[10000]        1,650,506,999.8093 (>1000.0)  1,667,188,624.8514 (>1000.0)  1,658,910,741.4708 (>1000.0)    6,128,545.5021 (>1000.0)  1,659,008,624.9635 (>1000.0)    7,700,999.7331 (>1000.0)       2;0          0.6028 (0.00)          5           1
test_remove_baggage[10000]     1,676,516,375.0388 (>1000.0)  1,893,300,541.6766 (>1000.0)  1,789,521,425.1243 (>1000.0)  105,281,425.0826 (>1000.0)  1,805,116,917.0067 (>1000.0)  206,181,510.7008 (>1000.0)       1;0          0.5588 (0.00)          5           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

@xrmx xrmx merged commit 6754bd2 into open-telemetry:main Mar 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@xrmx xrmx xrmx approved these changes

@tammy-baylis-swi tammy-baylis-swi tammy-baylis-swi approved these changes

Assignees

No one assigned

Labels

Skip Changelog PRs that do not require a CHANGELOG.md entry

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@emdneto @xrmx @tammy-baylis-swi