Migrate text field to match_only_text by ebadyano · Pull Request #181 · elastic/rally-tracks

ebadyano · 2021-06-10T14:36:35Z

Relates to #170
Relates to elastic/ecs#1396

Relates to elastic#170 Relates to elastic/ecs#1396

ebadyano · 2021-06-10T14:42:22Z

I ran on low-mem env with and without the change:
esrally race --telemetry="gc" --target-host="192.168.14.13:9200" --track-path=~/tracks/rally-tracks/http_logs --challenge="append-no-conflicts" --car="defaults,basic-license" --runtime-jdk="bundled" --client-options="timeout:240" --on-error="abort"

But I don't see much difference in query performance:

esrally compare --baseline=2fe7194b-f1cd-423e-af63-5d696a679287 --contender=9779b226-49cb-43ec-9bb4-d89ee5df119a

    ____        ____
   / __ \____ _/ / /_  __
  / /_/ / __ `/ / / / / /
 / _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
                /____/


Comparing baseline
  Race ID: 2fe7194b-f1cd-423e-af63-5d696a679287
  Race timestamp: 2021-06-09 19:00:30
  Challenge: append-no-conflicts
  Car: defaults+basic-license

with contender
  Race ID: 9779b226-49cb-43ec-9bb4-d89ee5df119a
  Race timestamp: 2021-06-08 18:26:05
  Challenge: append-no-conflicts
  Car: defaults+basic-license

------------------------------------------------------
    _______             __   _____
   / ____(_)___  ____ _/ /  / ___/_________  ________
  / /_  / / __ \/ __ `/ /   \__ \/ ___/ __ \/ ___/ _ \
 / __/ / / / / / /_/ / /   ___/ / /__/ /_/ / /  /  __/
/_/   /_/_/ /_/\__,_/_/   /____/\___/\____/_/   \___/
------------------------------------------------------

|                                                        Metric |                                                   Task |    Baseline |   Contender |     Diff |    Unit |
|--------------------------------------------------------------:|-------------------------------------------------------:|------------:|------------:|---------:|--------:|
|                    Cumulative indexing time of primary shards |                                                        |     132.457 |     125.747 | -6.71023 |     min |
|             Min cumulative indexing time across primary shard |                                                        |           0 |           0 |        0 |     min |
|          Median cumulative indexing time across primary shard |                                                        |     1.13028 |     1.09126 | -0.03902 |     min |
|             Max cumulative indexing time across primary shard |                                                        |      19.458 |     18.4146 | -1.04342 |     min |
|           Cumulative indexing throttle time of primary shards |                                                        |           0 |           0 |        0 |     min |
|    Min cumulative indexing throttle time across primary shard |                                                        |           0 |           0 |        0 |     min |
| Median cumulative indexing throttle time across primary shard |                                                        |           0 |           0 |        0 |     min |
|    Max cumulative indexing throttle time across primary shard |                                                        |           0 |           0 |        0 |     min |
|                       Cumulative merge time of primary shards |                                                        |     120.233 |      112.07 |  -8.1635 |     min |
|                      Cumulative merge count of primary shards |                                                        |         467 |         464 |       -3 |         |
|                Min cumulative merge time across primary shard |                                                        |           0 |           0 |        0 |     min |
|             Median cumulative merge time across primary shard |                                                        |      0.4094 |    0.365642 | -0.04376 |     min |
|                Max cumulative merge time across primary shard |                                                        |      22.391 |     20.9076 | -1.48335 |     min |
|              Cumulative merge throttle time of primary shards |                                                        |     54.4335 |     53.5529 | -0.88058 |     min |
|       Min cumulative merge throttle time across primary shard |                                                        |           0 |           0 |        0 |     min |
|    Median cumulative merge throttle time across primary shard |                                                        |           0 |           0 |        0 |     min |
|       Max cumulative merge throttle time across primary shard |                                                        |     11.6503 |     11.1526 | -0.49772 |     min |
|                     Cumulative refresh time of primary shards |                                                        |     14.4392 |     13.1283 | -1.31092 |     min |
|                    Cumulative refresh count of primary shards |                                                        |        2135 |        2106 |      -29 |         |
|              Min cumulative refresh time across primary shard |                                                        |           0 |           0 |        0 |     min |
|           Median cumulative refresh time across primary shard |                                                        |    0.136017 |    0.124858 | -0.01116 |     min |
|              Max cumulative refresh time across primary shard |                                                        |       2.142 |     1.97485 | -0.16715 |     min |
|                       Cumulative flush time of primary shards |                                                        |     3.82482 |     3.70147 | -0.12335 |     min |
|                      Cumulative flush count of primary shards |                                                        |         127 |         125 |       -2 |         |
|                Min cumulative flush time across primary shard |                                                        |           0 |           0 |        0 |     min |
|             Median cumulative flush time across primary shard |                                                        |   0.0106833 |      0.0099 | -0.00078 |     min |
|                Max cumulative flush time across primary shard |                                                        |    0.693217 |    0.720533 |  0.02732 |     min |
|                                       Total Young Gen GC time |                                                        |       78.93 |      77.899 |   -1.031 |       s |
|                                      Total Young Gen GC count |                                                        |       11576 |       11520 |      -56 |         |
|                                         Total Old Gen GC time |                                                        |           0 |           0 |        0 |       s |
|                                        Total Old Gen GC count |                                                        |           0 |           0 |        0 |         |
|                                                    Store size |                                                        |     17.6841 |     17.1307 | -0.55336 |      GB |
|                                                 Translog size |                                                        | 2.04891e-06 | 2.04891e-06 |        0 |      GB |
|                                        Heap used for segments |                                                        |    0.064846 |   0.0703316 |  0.00549 |      MB |
|                                      Heap used for doc values |                                                        |  0.00502396 |  0.00503159 |    1e-05 |      MB |
|                                           Heap used for terms |                                                        |   0.0245667 |   0.0320435 |  0.00748 |      MB |
|                                           Heap used for norms |                                                        |  0.00213623 |           0 | -0.00214 |      MB |
|                                          Heap used for points |                                                        |           0 |           0 |        0 |      MB |
|                                   Heap used for stored fields |                                                        |   0.0331192 |   0.0332565 |  0.00014 |      MB |
|                                                 Segment count |                                                        |          35 |          35 |        0 |         |
|                                                Min Throughput |                                           index-append |      143873 |      148836 |  4962.97 |  docs/s |
|                                               Mean Throughput |                                           index-append |      151587 |      156226 |  4639.64 |  docs/s |
|                                             Median Throughput |                                           index-append |      149680 |      153774 |   4094.3 |  docs/s |
|                                                Max Throughput |                                           index-append |      164444 |      168766 |  4321.88 |  docs/s |
|                                       50th percentile latency |                                           index-append |     236.378 |     228.725 | -7.65331 |      ms |
|                                       90th percentile latency |                                           index-append |     358.452 |     346.763 | -11.6885 |      ms |
|                                       99th percentile latency |                                           index-append |     876.317 |      834.59 |  -41.727 |      ms |
|                                     99.9th percentile latency |                                           index-append |     1288.37 |      1176.3 | -112.078 |      ms |
|                                    99.99th percentile latency |                                           index-append |     1593.17 |     1544.97 | -48.1969 |      ms |
|                                      100th percentile latency |                                           index-append |     2058.41 |     1673.21 | -385.192 |      ms |
|                                  50th percentile service time |                                           index-append |     236.377 |     228.725 | -7.65255 |      ms |
|                                  90th percentile service time |                                           index-append |     358.349 |     346.832 | -11.5168 |      ms |
|                                  99th percentile service time |                                           index-append |     876.639 |      834.59 | -42.0493 |      ms |
|                                99.9th percentile service time |                                           index-append |     1288.37 |      1176.3 | -112.078 |      ms |
|                               99.99th percentile service time |                                           index-append |     1593.17 |     1544.97 | -48.1969 |      ms |
|                                 100th percentile service time |                                           index-append |     2058.41 |     1673.21 | -385.192 |      ms |
|                               50th percentile processing time |                                           index-append |      240.04 |     232.307 | -7.73297 |      ms |
|                               90th percentile processing time |                                           index-append |      361.86 |     350.024 | -11.8355 |      ms |
|                               99th percentile processing time |                                           index-append |     881.067 |     839.392 | -41.6757 |      ms |
|                             99.9th percentile processing time |                                           index-append |     1293.07 |     1181.28 |  -111.79 |      ms |
|                            99.99th percentile processing time |                                           index-append |     1597.06 |     1548.53 | -48.5209 |      ms |
|                              100th percentile processing time |                                           index-append |     2062.76 |     1677.62 | -385.148 |      ms |
|                                                    error rate |                                           index-append |           0 |           0 |        0 |       % |
|                                                Min Throughput |                                                default |     8.00625 |     8.00588 | -0.00037 |   ops/s |
|                                               Mean Throughput |                                                default |     8.00674 |     8.00631 | -0.00044 |   ops/s |
|                                             Median Throughput |                                                default |     8.00668 |      8.0063 | -0.00038 |   ops/s |
|                                                Max Throughput |                                                default |     8.00731 |     8.00685 | -0.00046 |   ops/s |
|                                       50th percentile latency |                                                default |     9.27157 |     9.35879 |  0.08722 |      ms |
|                                       90th percentile latency |                                                default |      9.6858 |     9.93309 |  0.24729 |      ms |
|                                       99th percentile latency |                                                default |      10.808 |     12.9243 |  2.11632 |      ms |
|                                      100th percentile latency |                                                default |     12.2593 |     20.0946 |  7.83535 |      ms |
|                                  50th percentile service time |                                                default |     7.73279 |     7.93396 |  0.20117 |      ms |
|                                  90th percentile service time |                                                default |     7.92916 |     8.16558 |  0.23642 |      ms |
|                                  99th percentile service time |                                                default |     8.93102 |     11.4235 |  2.49247 |      ms |
|                                 100th percentile service time |                                                default |     10.6469 |     18.6953 |  8.04845 |      ms |
|                               50th percentile processing time |                                                default |     8.42933 |     8.60487 |  0.17554 |      ms |
|                               90th percentile processing time |                                                default |     8.67629 |     8.87187 |  0.19559 |      ms |
|                               99th percentile processing time |                                                default |     9.68862 |     12.1647 |   2.4761 |      ms |
|                              100th percentile processing time |                                                default |     11.3433 |     19.3656 |  8.02235 |      ms |
|                                                    error rate |                                                default |           0 |           0 |        0 |       % |
|                                                Min Throughput |                                                   term |     49.8586 |     49.9097 |  0.05117 |   ops/s |
|                                               Mean Throughput |                                                   term |      49.864 |     49.9131 |  0.04919 |   ops/s |
|                                             Median Throughput |                                                   term |      49.864 |     49.9131 |  0.04919 |   ops/s |
|                                                Max Throughput |                                                   term |     49.8693 |     49.9166 |  0.04721 |   ops/s |
|                                       50th percentile latency |                                                   term |     14.6919 |     14.5156 | -0.17631 |      ms |
|                                       90th percentile latency |                                                   term |     15.4431 |     15.1509 | -0.29223 |      ms |
|                                       99th percentile latency |                                                   term |      27.481 |     25.8935 | -1.58756 |      ms |
|                                      100th percentile latency |                                                   term |     27.8886 |     30.1782 |  2.28959 |      ms |
|                                  50th percentile service time |                                                   term |     13.4221 |     13.0992 | -0.32299 |      ms |
|                                  90th percentile service time |                                                   term |     13.8477 |      13.462 | -0.38566 |      ms |
|                                  99th percentile service time |                                                   term |     26.2296 |     24.8866 | -1.34299 |      ms |
|                                 100th percentile service time |                                                   term |     26.6056 |     28.7607 |  2.15513 |      ms |
|                               50th percentile processing time |                                                   term |     14.5083 |     14.2343 | -0.27398 |      ms |
|                               90th percentile processing time |                                                   term |     14.9623 |     14.6117 | -0.35051 |      ms |
|                               99th percentile processing time |                                                   term |     27.3141 |     26.0017 | -1.31238 |      ms |
|                              100th percentile processing time |                                                   term |     27.8148 |     29.8818 |  2.06701 |      ms |
|                                                    error rate |                                                   term |           0 |           0 |        0 |       % |
|                                                Min Throughput |                                             terms_enum |     49.7785 |     49.7859 |  0.00744 |   ops/s |
|                                               Mean Throughput |                                             terms_enum |     49.7881 |     49.7881 |    8e-05 |   ops/s |
|                                             Median Throughput |                                             terms_enum |     49.7881 |     49.7881 |    8e-05 |   ops/s |
|                                                Max Throughput |                                             terms_enum |     49.7977 |     49.7904 | -0.00728 |   ops/s |
|                                       50th percentile latency |                                             terms_enum |     14.2765 |     13.8551 | -0.42138 |      ms |
|                                       90th percentile latency |                                             terms_enum |     25.8189 |     29.7765 |  3.95758 |      ms |
|                                       99th percentile latency |                                             terms_enum |     42.0677 |     38.2267 | -3.84099 |      ms |
|                                      100th percentile latency |                                             terms_enum |     43.7508 |     40.8945 |  -2.8563 |      ms |
|                                  50th percentile service time |                                             terms_enum |     11.9784 |     11.0677 | -0.91063 |      ms |
|                                  90th percentile service time |                                             terms_enum |     20.9862 |     22.2667 |  1.28041 |      ms |
|                                  99th percentile service time |                                             terms_enum |     25.8849 |     26.0021 |   0.1172 |      ms |
|                                 100th percentile service time |                                             terms_enum |     29.4196 |     27.1828 | -2.23686 |      ms |
|                               50th percentile processing time |                                             terms_enum |     12.7217 |      11.727 | -0.99474 |      ms |
|                               90th percentile processing time |                                             terms_enum |     21.6889 |     22.9905 |  1.30162 |      ms |
|                               99th percentile processing time |                                             terms_enum |     26.5612 |     26.6529 |  0.09177 |      ms |
|                              100th percentile processing time |                                             terms_enum |     30.0943 |     27.7826 | -2.31166 |      ms |
|                                                    error rate |                                             terms_enum |           0 |           0 |        0 |       % |
|                                                Min Throughput |                                                  range |     1.00421 |     1.00438 |  0.00016 |   ops/s |
|                                               Mean Throughput |                                                  range |     1.00583 |     1.00606 |  0.00023 |   ops/s |
|                                             Median Throughput |                                                  range |      1.0056 |     1.00582 |  0.00022 |   ops/s |
|                                                Max Throughput |                                                  range |     1.00837 |      1.0087 |  0.00033 |   ops/s |
|                                       50th percentile latency |                                                  range |     20.6735 |     20.5687 | -0.10478 |      ms |
|                                       90th percentile latency |                                                  range |     21.0044 |     21.2458 |  0.24144 |      ms |
|                                       99th percentile latency |                                                  range |     21.9158 |     22.7025 |   0.7867 |      ms |
|                                      100th percentile latency |                                                  range |     23.5615 |     25.0108 |  1.44932 |      ms |
|                                  50th percentile service time |                                                  range |     18.1931 |      18.177 | -0.01609 |      ms |
|                                  90th percentile service time |                                                  range |     18.6122 |      18.794 |  0.18177 |      ms |
|                                  99th percentile service time |                                                  range |     19.1681 |     20.0282 |   0.8601 |      ms |
|                                 100th percentile service time |                                                  range |     21.6292 |     23.0702 |  1.44098 |      ms |
|                               50th percentile processing time |                                                  range |      19.284 |     19.2893 |  0.00531 |      ms |
|                               90th percentile processing time |                                                  range |      19.709 |     19.9133 |  0.20428 |      ms |
|                               99th percentile processing time |                                                  range |     20.3823 |     21.1723 |  0.78995 |      ms |
|                              100th percentile processing time |                                                  range |     22.7781 |     24.2044 |  1.42631 |      ms |
|                                                    error rate |                                                  range |           0 |           0 |        0 |       % |
|                                                Min Throughput |                                          200s-in-range |     32.9536 |     32.9628 |  0.00926 |   ops/s |
|                                               Mean Throughput |                                          200s-in-range |     32.9566 |     32.9643 |  0.00762 |   ops/s |
|                                             Median Throughput |                                          200s-in-range |     32.9568 |     32.9635 |  0.00675 |   ops/s |
|                                                Max Throughput |                                          200s-in-range |     32.9596 |     32.9664 |  0.00686 |   ops/s |
|                                       50th percentile latency |                                          200s-in-range |     24.7432 |     22.4972 | -2.24597 |      ms |
|                                       90th percentile latency |                                          200s-in-range |     25.7739 |     23.8862 | -1.88769 |      ms |
|                                       99th percentile latency |                                          200s-in-range |     32.4503 |     25.5153 | -6.93499 |      ms |
|                                      100th percentile latency |                                          200s-in-range |     38.2398 |     25.5336 | -12.7062 |      ms |
|                                  50th percentile service time |                                          200s-in-range |     23.4381 |     21.0007 | -2.43736 |      ms |
|                                  90th percentile service time |                                          200s-in-range |     24.1096 |     22.3637 | -1.74585 |      ms |
|                                  99th percentile service time |                                          200s-in-range |     24.3655 |     22.6871 | -1.67843 |      ms |
|                                 100th percentile service time |                                          200s-in-range |     36.5706 |     22.7044 | -13.8662 |      ms |
|                               50th percentile processing time |                                          200s-in-range |     24.4884 |     22.1501 | -2.33831 |      ms |
|                               90th percentile processing time |                                          200s-in-range |     25.2545 |     23.5122 | -1.74239 |      ms |
|                               99th percentile processing time |                                          200s-in-range |     25.5088 |       23.83 | -1.67888 |      ms |
|                              100th percentile processing time |                                          200s-in-range |     37.7371 |     23.8507 | -13.8864 |      ms |
|                                                    error rate |                                          200s-in-range |           0 |           0 |        0 |       % |
|                                                Min Throughput |                                          400s-in-range |     50.0129 |      50.039 |  0.02608 |   ops/s |
|                                               Mean Throughput |                                          400s-in-range |     50.0134 |     50.0416 |  0.02819 |   ops/s |
|                                             Median Throughput |                                          400s-in-range |     50.0134 |     50.0416 |  0.02819 |   ops/s |
|                                                Max Throughput |                                          400s-in-range |     50.0139 |     50.0441 |   0.0303 |   ops/s |
|                                       50th percentile latency |                                          400s-in-range |     9.82567 |     9.94705 |  0.12138 |      ms |
|                                       90th percentile latency |                                          400s-in-range |     10.2639 |     10.7976 |   0.5337 |      ms |
|                                       99th percentile latency |                                          400s-in-range |     10.7923 |     12.4038 |  1.61149 |      ms |
|                                      100th percentile latency |                                          400s-in-range |     10.8264 |     23.9764 |  13.1501 |      ms |
|                                  50th percentile service time |                                          400s-in-range |     8.15689 |     8.01697 | -0.13993 |      ms |
|                                  90th percentile service time |                                          400s-in-range |     8.49397 |     8.71363 |  0.21966 |      ms |
|                                  99th percentile service time |                                          400s-in-range |     8.81146 |     9.18802 |  0.37656 |      ms |
|                                 100th percentile service time |                                          400s-in-range |     8.93126 |     22.6895 |  13.7582 |      ms |
|                               50th percentile processing time |                                          400s-in-range |     9.30749 |     9.17029 |  -0.1372 |      ms |
|                               90th percentile processing time |                                          400s-in-range |     9.61962 |     9.83447 |  0.21485 |      ms |
|                               99th percentile processing time |                                          400s-in-range |     9.95696 |     10.3051 |   0.3481 |      ms |
|                              100th percentile processing time |                                          400s-in-range |     9.99806 |     23.3484 |  13.3503 |      ms |
|                                                    error rate |                                          400s-in-range |           0 |           0 |        0 |       % |
|                                                Min Throughput |                                             hourly_agg |    0.200324 |    0.200304 |   -2e-05 |   ops/s |
|                                               Mean Throughput |                                             hourly_agg |    0.200448 |    0.200421 |   -3e-05 |   ops/s |
|                                             Median Throughput |                                             hourly_agg |    0.200431 |    0.200405 |   -3e-05 |   ops/s |
|                                                Max Throughput |                                             hourly_agg |    0.200642 |    0.200604 |   -4e-05 |   ops/s |
|                                       50th percentile latency |                                             hourly_agg |     3070.66 |     3073.14 |  2.48315 |      ms |
|                                       90th percentile latency |                                             hourly_agg |     3109.78 |     3097.74 | -12.0432 |      ms |
|                                       99th percentile latency |                                             hourly_agg |     3128.42 |     3125.35 | -3.07415 |      ms |
|                                      100th percentile latency |                                             hourly_agg |     3135.74 |     3130.18 | -5.55713 |      ms |
|                                  50th percentile service time |                                             hourly_agg |     3067.45 |     3069.57 |  2.11792 |      ms |
|                                  90th percentile service time |                                             hourly_agg |     3106.39 |     3094.45 | -11.9427 |      ms |
|                                  99th percentile service time |                                             hourly_agg |     3124.86 |     3122.44 |  -2.4266 |      ms |
|                                 100th percentile service time |                                             hourly_agg |     3132.53 |     3125.76 | -6.76172 |      ms |
|                               50th percentile processing time |                                             hourly_agg |     3068.16 |     3070.27 |   2.1073 |      ms |
|                               90th percentile processing time |                                             hourly_agg |     3107.07 |     3095.14 | -11.9336 |      ms |
|                               99th percentile processing time |                                             hourly_agg |     3125.58 |     3123.16 | -2.42263 |      ms |
|                              100th percentile processing time |                                             hourly_agg |      3133.2 |     3126.45 | -6.75122 |      ms |
|                                                    error rate |                                             hourly_agg |           0 |           0 |        0 |       % |
|                                                Min Throughput |                                                 scroll |      25.035 |     25.0378 |  0.00275 | pages/s |
|                                               Mean Throughput |                                                 scroll |     25.0576 |     25.0623 |  0.00472 | pages/s |
|                                             Median Throughput |                                                 scroll |     25.0525 |     25.0567 |  0.00426 | pages/s |
|                                                Max Throughput |                                                 scroll |     25.1043 |     25.1129 |  0.00858 | pages/s |
|                                       50th percentile latency |                                                 scroll |     830.085 |      832.33 |  2.24506 |      ms |
|                                       90th percentile latency |                                                 scroll |     847.521 |     857.922 |  10.4011 |      ms |
|                                       99th percentile latency |                                                 scroll |     870.021 |     890.248 |  20.2272 |      ms |
|                                      100th percentile latency |                                                 scroll |     886.678 |     894.074 |  7.39581 |      ms |
|                                  50th percentile service time |                                                 scroll |      827.67 |     830.078 |  2.40784 |      ms |
|                                  90th percentile service time |                                                 scroll |     845.138 |      855.53 |  10.3924 |      ms |
|                                  99th percentile service time |                                                 scroll |     867.561 |     887.231 |  19.6698 |      ms |
|                                 100th percentile service time |                                                 scroll |     883.533 |     891.815 |  8.28235 |      ms |
|                               50th percentile processing time |                                                 scroll |     829.197 |     831.615 |  2.41873 |      ms |
|                               90th percentile processing time |                                                 scroll |     846.835 |     857.146 |  10.3114 |      ms |
|                               99th percentile processing time |                                                 scroll |      869.14 |     888.776 |  19.6356 |      ms |
|                              100th percentile processing time |                                                 scroll |     885.227 |     893.351 |  8.12408 |      ms |
|                                                    error rate |                                                 scroll |           0 |           0 |        0 |       % |
|                                                Min Throughput |                                    desc_sort_timestamp |    0.501611 |    0.501602 |   -1e-05 |   ops/s |
|                                               Mean Throughput |                                    desc_sort_timestamp |    0.501957 |    0.501946 |   -1e-05 |   ops/s |
|                                             Median Throughput |                                    desc_sort_timestamp |    0.501931 |    0.501919 |   -1e-05 |   ops/s |
|                                                Max Throughput |                                    desc_sort_timestamp |    0.502411 |    0.502396 |   -2e-05 |   ops/s |
|                                       50th percentile latency |                                    desc_sort_timestamp |     14.8902 |     19.4778 |  4.58758 |      ms |
|                                       90th percentile latency |                                    desc_sort_timestamp |     17.2655 |      21.766 |  4.50053 |      ms |
|                                       99th percentile latency |                                    desc_sort_timestamp |     17.9573 |     22.9708 |  5.01349 |      ms |
|                                      100th percentile latency |                                    desc_sort_timestamp |     18.0244 |     23.2717 |  5.24725 |      ms |
|                                  50th percentile service time |                                    desc_sort_timestamp |     11.5734 |      16.175 |  4.60162 |      ms |
|                                  90th percentile service time |                                    desc_sort_timestamp |     13.7602 |     18.4512 |  4.69104 |      ms |
|                                  99th percentile service time |                                    desc_sort_timestamp |     14.3443 |     19.4569 |  5.11264 |      ms |
|                                 100th percentile service time |                                    desc_sort_timestamp |     14.4753 |     19.6674 |  5.19209 |      ms |
|                               50th percentile processing time |                                    desc_sort_timestamp |     12.2888 |     16.8286 |  4.53983 |      ms |
|                               90th percentile processing time |                                    desc_sort_timestamp |     14.4803 |     19.0877 |  4.60738 |      ms |
|                               99th percentile processing time |                                    desc_sort_timestamp |     15.0838 |     20.1941 |  5.11026 |      ms |
|                              100th percentile processing time |                                    desc_sort_timestamp |     15.2288 |     20.3616 |  5.13281 |      ms |
|                                                    error rate |                                    desc_sort_timestamp |           0 |           0 |        0 |       % |
|                                                Min Throughput |                                     asc_sort_timestamp |    0.501656 |     0.50166 |        0 |   ops/s |
|                                               Mean Throughput |                                     asc_sort_timestamp |    0.502012 |    0.502017 |    1e-05 |   ops/s |
|                                             Median Throughput |                                     asc_sort_timestamp |    0.501984 |     0.50199 |    1e-05 |   ops/s |
|                                                Max Throughput |                                     asc_sort_timestamp |    0.502475 |    0.502482 |    1e-05 |   ops/s |
|                                       50th percentile latency |                                     asc_sort_timestamp |     12.7185 |     12.7835 |  0.06501 |      ms |
|                                       90th percentile latency |                                     asc_sort_timestamp |     14.8656 |     14.5841 |  -0.2815 |      ms |
|                                       99th percentile latency |                                     asc_sort_timestamp |     15.1717 |     15.3596 |  0.18793 |      ms |
|                                      100th percentile latency |                                     asc_sort_timestamp |     15.3168 |     15.6444 |  0.32764 |      ms |
|                                  50th percentile service time |                                     asc_sort_timestamp |     9.41674 |     9.61681 |  0.20007 |      ms |
|                                  90th percentile service time |                                     asc_sort_timestamp |     11.3316 |     11.3482 |  0.01656 |      ms |
|                                  99th percentile service time |                                     asc_sort_timestamp |     11.7439 |     11.6915 | -0.05243 |      ms |
|                                 100th percentile service time |                                     asc_sort_timestamp |     11.7741 |     12.7085 |  0.93438 |      ms |
|                               50th percentile processing time |                                     asc_sort_timestamp |     10.1056 |     10.2952 |  0.18959 |      ms |
|                               90th percentile processing time |                                     asc_sort_timestamp |     12.0428 |     12.0464 |  0.00357 |      ms |
|                               99th percentile processing time |                                     asc_sort_timestamp |     12.4803 |     12.3914 |  -0.0888 |      ms |
|                              100th percentile processing time |                                     asc_sort_timestamp |     12.4821 |     13.3664 |  0.88429 |      ms |
|                                                    error rate |                                     asc_sort_timestamp |           0 |           0 |        0 |       % |
|                                                Min Throughput |                         desc_sort_with_after_timestamp |    0.501347 |    0.501266 |   -8e-05 |   ops/s |
|                                               Mean Throughput |                         desc_sort_with_after_timestamp |    0.503516 |    0.503301 | -0.00021 |   ops/s |
|                                             Median Throughput |                         desc_sort_with_after_timestamp |    0.502454 |    0.502304 | -0.00015 |   ops/s |
|                                                Max Throughput |                         desc_sort_with_after_timestamp |    0.513804 |    0.512947 | -0.00086 |   ops/s |
|                                       50th percentile latency |                         desc_sort_with_after_timestamp |     1332.57 |      1254.2 | -78.3726 |      ms |
|                                       90th percentile latency |                         desc_sort_with_after_timestamp |     1342.86 |     1268.32 | -74.5378 |      ms |
|                                       99th percentile latency |                         desc_sort_with_after_timestamp |      1362.3 |     1309.33 | -52.9723 |      ms |
|                                      100th percentile latency |                         desc_sort_with_after_timestamp |     1371.47 |     1311.44 | -60.0298 |      ms |
|                                  50th percentile service time |                         desc_sort_with_after_timestamp |     1330.67 |     1251.99 | -78.6727 |      ms |
|                                  90th percentile service time |                         desc_sort_with_after_timestamp |     1340.75 |     1266.11 | -74.6353 |      ms |
|                                  99th percentile service time |                         desc_sort_with_after_timestamp |     1360.53 |     1307.08 | -53.4412 |      ms |
|                                 100th percentile service time |                         desc_sort_with_after_timestamp |     1369.88 |     1309.78 | -60.1043 |      ms |
|                               50th percentile processing time |                         desc_sort_with_after_timestamp |     1331.29 |     1252.69 | -78.6015 |      ms |
|                               90th percentile processing time |                         desc_sort_with_after_timestamp |     1341.48 |     1266.77 | -74.7157 |      ms |
|                               99th percentile processing time |                         desc_sort_with_after_timestamp |     1361.24 |     1307.79 | -53.4497 |      ms |
|                              100th percentile processing time |                         desc_sort_with_after_timestamp |     1370.62 |     1310.49 | -60.1289 |      ms |
|                                                    error rate |                         desc_sort_with_after_timestamp |           0 |           0 |        0 |       % |
|                                                Min Throughput |                          asc_sort_with_after_timestamp |    0.501488 |    0.501681 |  0.00019 |   ops/s |
|                                               Mean Throughput |                          asc_sort_with_after_timestamp |    0.503889 |    0.504399 |  0.00051 |   ops/s |
|                                             Median Throughput |                          asc_sort_with_after_timestamp |    0.502712 |    0.503066 |  0.00035 |   ops/s |
|                                                Max Throughput |                          asc_sort_with_after_timestamp |    0.515288 |    0.517341 |  0.00205 |   ops/s |
|                                       50th percentile latency |                          asc_sort_with_after_timestamp |     1369.59 |     1244.19 | -125.394 |      ms |
|                                       90th percentile latency |                          asc_sort_with_after_timestamp |     1389.21 |     1265.15 | -124.059 |      ms |
|                                       99th percentile latency |                          asc_sort_with_after_timestamp |     1401.39 |     1280.47 | -120.913 |      ms |
|                                      100th percentile latency |                          asc_sort_with_after_timestamp |     1405.06 |     1293.58 |  -111.48 |      ms |
|                                  50th percentile service time |                          asc_sort_with_after_timestamp |     1367.64 |     1242.01 | -125.627 |      ms |
|                                  90th percentile service time |                          asc_sort_with_after_timestamp |     1387.47 |     1262.98 | -124.494 |      ms |
|                                  99th percentile service time |                          asc_sort_with_after_timestamp |     1399.56 |     1278.36 | -121.199 |      ms |
|                                 100th percentile service time |                          asc_sort_with_after_timestamp |     1402.67 |     1291.78 | -110.884 |      ms |
|                               50th percentile processing time |                          asc_sort_with_after_timestamp |      1368.3 |      1242.7 |   -125.6 |      ms |
|                               90th percentile processing time |                          asc_sort_with_after_timestamp |     1388.18 |     1263.61 | -124.572 |      ms |
|                               99th percentile processing time |                          asc_sort_with_after_timestamp |     1400.24 |     1279.03 |  -121.21 |      ms |
|                              100th percentile processing time |                          asc_sort_with_after_timestamp |     1403.36 |     1292.49 | -110.866 |      ms |
|                                                    error rate |                          asc_sort_with_after_timestamp |           0 |           0 |        0 |       % |
|                                                Min Throughput |            desc-sort-timestamp-after-force-merge-1-seg |     2.00447 |     2.00407 |  -0.0004 |   ops/s |
|                                               Mean Throughput |            desc-sort-timestamp-after-force-merge-1-seg |     2.00542 |     2.00494 | -0.00048 |   ops/s |
|                                             Median Throughput |            desc-sort-timestamp-after-force-merge-1-seg |     2.00534 |     2.00487 | -0.00047 |   ops/s |
|                                                Max Throughput |            desc-sort-timestamp-after-force-merge-1-seg |     2.00665 |     2.00606 | -0.00059 |   ops/s |
|                                       50th percentile latency |            desc-sort-timestamp-after-force-merge-1-seg |     176.838 |     185.804 |  8.96665 |      ms |
|                                       90th percentile latency |            desc-sort-timestamp-after-force-merge-1-seg |     251.169 |     196.534 | -54.6348 |      ms |
|                                       99th percentile latency |            desc-sort-timestamp-after-force-merge-1-seg |     257.284 |     209.363 | -47.9215 |      ms |
|                                      100th percentile latency |            desc-sort-timestamp-after-force-merge-1-seg |     260.008 |     217.429 | -42.5793 |      ms |
|                                  50th percentile service time |            desc-sort-timestamp-after-force-merge-1-seg |     175.387 |     184.037 |  8.64942 |      ms |
|                                  90th percentile service time |            desc-sort-timestamp-after-force-merge-1-seg |     249.428 |     194.662 | -54.7656 |      ms |
|                                  99th percentile service time |            desc-sort-timestamp-after-force-merge-1-seg |     255.626 |     207.974 | -47.6524 |      ms |
|                                 100th percentile service time |            desc-sort-timestamp-after-force-merge-1-seg |     257.885 |     215.777 | -42.1084 |      ms |
|                               50th percentile processing time |            desc-sort-timestamp-after-force-merge-1-seg |     176.112 |     184.723 |  8.61047 |      ms |
|                               90th percentile processing time |            desc-sort-timestamp-after-force-merge-1-seg |     250.099 |       195.4 | -54.6987 |      ms |
|                               99th percentile processing time |            desc-sort-timestamp-after-force-merge-1-seg |      256.29 |     208.653 | -47.6374 |      ms |
|                              100th percentile processing time |            desc-sort-timestamp-after-force-merge-1-seg |     258.585 |      216.47 | -42.1149 |      ms |
|                                                    error rate |            desc-sort-timestamp-after-force-merge-1-seg |           0 |           0 |        0 |       % |
|                                                Min Throughput |             asc-sort-timestamp-after-force-merge-1-seg |      2.0065 |     2.00654 |    4e-05 |   ops/s |
|                                               Mean Throughput |             asc-sort-timestamp-after-force-merge-1-seg |     2.00789 |     2.00793 |    4e-05 |   ops/s |
|                                             Median Throughput |             asc-sort-timestamp-after-force-merge-1-seg |     2.00779 |     2.00783 |    4e-05 |   ops/s |
|                                                Max Throughput |             asc-sort-timestamp-after-force-merge-1-seg |     2.00968 |     2.00975 |    6e-05 |   ops/s |
|                                       50th percentile latency |             asc-sort-timestamp-after-force-merge-1-seg |     13.5282 |     11.8261 | -1.70213 |      ms |
|                                       90th percentile latency |             asc-sort-timestamp-after-force-merge-1-seg |     14.5446 |      12.985 | -1.55955 |      ms |
|                                       99th percentile latency |             asc-sort-timestamp-after-force-merge-1-seg |     15.5166 |     13.7783 | -1.73831 |      ms |
|                                      100th percentile latency |             asc-sort-timestamp-after-force-merge-1-seg |     15.5866 |     13.8497 | -1.73693 |      ms |
|                                  50th percentile service time |             asc-sort-timestamp-after-force-merge-1-seg |     11.5399 |     9.93365 | -1.60621 |      ms |
|                                  90th percentile service time |             asc-sort-timestamp-after-force-merge-1-seg |     12.7097 |     11.2009 | -1.50873 |      ms |
|                                  99th percentile service time |             asc-sort-timestamp-after-force-merge-1-seg |     13.2296 |     11.5703 | -1.65929 |      ms |
|                                 100th percentile service time |             asc-sort-timestamp-after-force-merge-1-seg |     13.2866 |     11.7253 | -1.56127 |      ms |
|                               50th percentile processing time |             asc-sort-timestamp-after-force-merge-1-seg |     12.2399 |     10.6171 | -1.62278 |      ms |
|                               90th percentile processing time |             asc-sort-timestamp-after-force-merge-1-seg |     13.4015 |     11.8933 | -1.50815 |      ms |
|                               99th percentile processing time |             asc-sort-timestamp-after-force-merge-1-seg |     13.9487 |     12.2719 | -1.67688 |      ms |
|                              100th percentile processing time |             asc-sort-timestamp-after-force-merge-1-seg |     14.0529 |     12.4817 | -1.57125 |      ms |
|                                                    error rate |             asc-sort-timestamp-after-force-merge-1-seg |           0 |           0 |        0 |       % |
|                                                Min Throughput | desc-sort-with-after-timestamp-after-force-merge-1-seg |    0.501495 |    0.501554 |    6e-05 |   ops/s |
|                                               Mean Throughput | desc-sort-with-after-timestamp-after-force-merge-1-seg |    0.503906 |     0.50406 |  0.00015 |   ops/s |
|                                             Median Throughput | desc-sort-with-after-timestamp-after-force-merge-1-seg |    0.502725 |     0.50283 |  0.00011 |   ops/s |
|                                                Max Throughput | desc-sort-with-after-timestamp-after-force-merge-1-seg |     0.51536 |    0.515981 |  0.00062 |   ops/s |
|                                       50th percentile latency | desc-sort-with-after-timestamp-after-force-merge-1-seg |     1329.63 |     1313.05 | -16.5772 |      ms |
|                                       90th percentile latency | desc-sort-with-after-timestamp-after-force-merge-1-seg |     1344.96 |     1339.98 | -4.98215 |      ms |
|                                       99th percentile latency | desc-sort-with-after-timestamp-after-force-merge-1-seg |     1380.77 |     1372.36 |  -8.4051 |      ms |
|                                      100th percentile latency | desc-sort-with-after-timestamp-after-force-merge-1-seg |     1397.51 |      1375.5 | -22.0074 |      ms |
|                                  50th percentile service time | desc-sort-with-after-timestamp-after-force-merge-1-seg |     1327.38 |     1310.87 | -16.5092 |      ms |
|                                  90th percentile service time | desc-sort-with-after-timestamp-after-force-merge-1-seg |     1343.13 |     1338.41 | -4.72856 |      ms |
|                                  99th percentile service time | desc-sort-with-after-timestamp-after-force-merge-1-seg |     1378.26 |     1370.42 | -7.84625 |      ms |
|                                 100th percentile service time | desc-sort-with-after-timestamp-after-force-merge-1-seg |      1395.9 |     1373.21 | -22.6928 |      ms |
|                               50th percentile processing time | desc-sort-with-after-timestamp-after-force-merge-1-seg |     1328.01 |      1311.6 | -16.4027 |      ms |
|                               90th percentile processing time | desc-sort-with-after-timestamp-after-force-merge-1-seg |     1343.86 |      1339.1 | -4.76255 |      ms |
|                               99th percentile processing time | desc-sort-with-after-timestamp-after-force-merge-1-seg |     1379.03 |     1371.13 | -7.89917 |      ms |
|                              100th percentile processing time | desc-sort-with-after-timestamp-after-force-merge-1-seg |     1396.53 |     1373.92 | -22.6066 |      ms |
|                                                    error rate | desc-sort-with-after-timestamp-after-force-merge-1-seg |           0 |           0 |        0 |       % |
|                                                Min Throughput |  asc-sort-with-after-timestamp-after-force-merge-1-seg |    0.501421 |    0.501528 |  0.00011 |   ops/s |
|                                               Mean Throughput |  asc-sort-with-after-timestamp-after-force-merge-1-seg |    0.503709 |    0.503994 |  0.00029 |   ops/s |
|                                             Median Throughput |  asc-sort-with-after-timestamp-after-force-merge-1-seg |    0.502589 |    0.502787 |   0.0002 |   ops/s |
|                                                Max Throughput |  asc-sort-with-after-timestamp-after-force-merge-1-seg |    0.514576 |    0.515725 |  0.00115 |   ops/s |
|                                       50th percentile latency |  asc-sort-with-after-timestamp-after-force-merge-1-seg |     1369.43 |     1326.61 | -42.8146 |      ms |
|                                       90th percentile latency |  asc-sort-with-after-timestamp-after-force-merge-1-seg |     1386.17 |     1351.05 | -35.1131 |      ms |
|                                       99th percentile latency |  asc-sort-with-after-timestamp-after-force-merge-1-seg |     1410.25 |     1401.64 | -8.61136 |      ms |
|                                      100th percentile latency |  asc-sort-with-after-timestamp-after-force-merge-1-seg |     1430.31 |     1410.46 | -19.8533 |      ms |
|                                  50th percentile service time |  asc-sort-with-after-timestamp-after-force-merge-1-seg |     1367.28 |     1324.74 | -42.5407 |      ms |
|                                  90th percentile service time |  asc-sort-with-after-timestamp-after-force-merge-1-seg |     1384.35 |     1349.36 | -34.9923 |      ms |
|                                  99th percentile service time |  asc-sort-with-after-timestamp-after-force-merge-1-seg |     1408.59 |      1399.7 | -8.89043 |      ms |
|                                 100th percentile service time |  asc-sort-with-after-timestamp-after-force-merge-1-seg |      1428.1 |     1408.83 | -19.2766 |      ms |
|                               50th percentile processing time |  asc-sort-with-after-timestamp-after-force-merge-1-seg |     1367.97 |     1325.38 | -42.5823 |      ms |
|                               90th percentile processing time |  asc-sort-with-after-timestamp-after-force-merge-1-seg |     1384.96 |     1350.01 | -34.9511 |      ms |
|                               99th percentile processing time |  asc-sort-with-after-timestamp-after-force-merge-1-seg |     1409.29 |     1400.41 | -8.87495 |      ms |
|                              100th percentile processing time |  asc-sort-with-after-timestamp-after-force-merge-1-seg |     1428.72 |     1409.54 | -19.1786 |      ms |
|                                                    error rate |  asc-sort-with-after-timestamp-after-force-merge-1-seg |           0 |           0 |        0 |       % |

@jpountz What is the expectation for the performance benefit?

Also, please note the original issue also mentions eventdata track from rally-tracks, but that track is going to be deprecated soon in favour of rally-eventdatat-track, so not including it in this change.

ebadyano · 2021-06-10T14:43:08Z

~~I will also backport this to 7.14 branch~~ backported to 7.14 on June 15th.

ebadyano · 2021-06-10T14:46:04Z

I will merge this on Monday.

jpountz · 2021-06-10T16:05:46Z

@jpountz What is the expectation for the performance benefit?

On this dataset where request is unlikely the main contributor to disk usage or CPU usage, I was expecting a modest improvement to disk usage, and between no improvement and a modest improvement to the indexing rate. No change is expected query-wise since this field is used in none of the queries of the http_logs dataset.

jpountz · 2021-06-15T11:53:34Z

This seems to have saved ~3% disk space on nightly benchmarks.

ebadyano · 2021-06-15T12:16:56Z

This seems to have saved ~3% disk space on nightly benchmarks.

@jpountz I merged the change yesterday as per our policy not to merge changes to rally/rally-tracks on Friday. So I think the drop on June 10th is due to something else. Might be just fluctuations, as on June 5th t was also around the same size as on June 10th.

Relates to elastic#170 Relates to elastic/ecs#1396

jpountz · 2021-06-15T12:21:30Z

@ebadyano I'm actually looking at the drop from June 14th on the http_logs track:

ebadyano · 2021-06-15T15:23:54Z

@jpountz oh i missed that, okay I will verify and annotate. Thank you for spotting that!

ebadyano · 2021-06-20T23:40:33Z

confirmed that it did reduce disk space usage by ~3% and annotated the chart.

Relates to #170 Relates to elastic/ecs#1396

Migrate text field to match_only_text

56dcf2a

Relates to elastic#170 Relates to elastic/ecs#1396

ebadyano added enhancement backport pending Awaiting backport to stable release branch labels Jun 10, 2021

ebadyano requested review from danielmitterdorfer and jpountz June 10, 2021 14:36

ebadyano self-assigned this Jun 10, 2021

jpountz approved these changes Jun 10, 2021

View reviewed changes

ebadyano merged commit 7a3ce22 into elastic:master Jun 14, 2021

ebadyano mentioned this pull request Jun 14, 2021

Migrate some text fields to match_only_text. #170

Closed

ebadyano added a commit to ebadyano/rally-tracks that referenced this pull request Jun 15, 2021

Migrate text field to match_only_text (elastic#181)

19a6d76

Relates to elastic#170 Relates to elastic/ecs#1396

ebadyano removed the backport pending Awaiting backport to stable release branch label Jun 15, 2021

ebadyano deleted the match-only-text branch June 20, 2021 23:40

ebadyano added a commit that referenced this pull request Oct 13, 2021

Migrate text field to match_only_text (#181)

9c5209c

Relates to #170 Relates to elastic/ecs#1396

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate text field to match_only_text#181

Migrate text field to match_only_text#181
ebadyano merged 1 commit intoelastic:masterfrom
ebadyano:match-only-text

ebadyano commented Jun 10, 2021

Uh oh!

ebadyano commented Jun 10, 2021 •

edited

Loading

Uh oh!

ebadyano commented Jun 10, 2021 •

edited

Loading

Uh oh!

ebadyano commented Jun 10, 2021

Uh oh!

jpountz commented Jun 10, 2021

Uh oh!

jpountz commented Jun 15, 2021

Uh oh!

ebadyano commented Jun 15, 2021

Uh oh!

jpountz commented Jun 15, 2021 •

edited

Loading

Uh oh!

ebadyano commented Jun 15, 2021

Uh oh!

ebadyano commented Jun 20, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

ebadyano commented Jun 10, 2021

Uh oh!

ebadyano commented Jun 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ebadyano commented Jun 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ebadyano commented Jun 10, 2021

Uh oh!

jpountz commented Jun 10, 2021

Uh oh!

jpountz commented Jun 15, 2021

Uh oh!

ebadyano commented Jun 15, 2021

Uh oh!

jpountz commented Jun 15, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ebadyano commented Jun 15, 2021

Uh oh!

ebadyano commented Jun 20, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ebadyano commented Jun 10, 2021 •

edited

Loading

ebadyano commented Jun 10, 2021 •

edited

Loading

jpountz commented Jun 15, 2021 •

edited

Loading