Skip to content

Add track for sub-bucket-aggs#114

Merged
hub-cap merged 6 commits intoelastic:masterfrom
nik9000:nested_terms
Jun 2, 2020
Merged

Add track for sub-bucket-aggs#114
hub-cap merged 6 commits intoelastic:masterfrom
nik9000:nested_terms

Conversation

@nik9000
Copy link
Copy Markdown
Member

@nik9000 nik9000 commented Apr 29, 2020

Adds a rally track specifically for testing the performance of the terms
agg as I'll be doing some work on it. In particular this focuses on
numeric terms because the first phase of my work only touches them.

Relates to elastic/elasticsearch#55873

Adds a rally track specifically for testing the performance of the terms
agg as I'll be doing some work on it. In particular this focuses on
numeric terms because the first phase of my work only touches them.

Relates to elastic/elasticsearch#55873
@dliappis
Copy link
Copy Markdown
Contributor

dliappis commented May 7, 2020

@nik9000
Copy link
Copy Markdown
Member Author

nik9000 commented May 18, 2020

I have an addition to this for elastic/elasticsearch#56921 which makes it more like a track for "sort of complex aggs". I can certainly hold off on pushing it so as to not get in the way of review for this though.

@nik9000
Copy link
Copy Markdown
Member Author

nik9000 commented May 20, 2020

I've added a couple more things to check now that I'm further in this process.

@hub-cap
Copy link
Copy Markdown

hub-cap commented May 27, 2020

Ill check them out. I tested the original 3 queries, and found that you can lower the interval on 2 of them and it still works reliably.

+          "operation": "numeric_terms_numeric_terms",
+          "clients": 1,
+          "warmup-iterations": 10,
+          "iterations": 50,
+          "target-interval": 2
+        },
+        {
+          "operation": "date_histo_numeric_terms",
+          "clients": 1,
+          "warmup-iterations": 10,
+          "iterations": 50,
+          "target-interval": 3

I will test w/ the new queries before going and recommending that you change those, though.

@nik9000
Copy link
Copy Markdown
Member Author

nik9000 commented May 28, 2020

@hub-cap I've made your changes locally and am playing with some more.

@hub-cap
Copy link
Copy Markdown

hub-cap commented May 28, 2020

cool, im going to also tweak the new tasks youve added, to see if we can get a bit more throughput out of them

@nik9000
Copy link
Copy Markdown
Member Author

nik9000 commented May 28, 2020

cool, im going to also tweak the new tasks youve added, to see if we can get a bit more throughput out of them

I just finished a run with the new throughput you suggested and they choke up Elasticsearch on my desktop a bit:

|                                                 Min Throughput |          keyword_terms_numeric_terms |        0.23 |  ops/s |
|                                              Median Throughput |          keyword_terms_numeric_terms |        0.24 |  ops/s |
|                                                 Max Throughput |          keyword_terms_numeric_terms |        0.24 |  ops/s |
|                                        50th percentile latency |          keyword_terms_numeric_terms |     9482.96 |     ms |
|                                        90th percentile latency |          keyword_terms_numeric_terms |     10416.5 |     ms |
|                                       100th percentile latency |          keyword_terms_numeric_terms |     10693.8 |     ms |
|                                   50th percentile service time |          keyword_terms_numeric_terms |     4056.17 |     ms |
|                                   90th percentile service time |          keyword_terms_numeric_terms |     4140.89 |     ms |
|                                  100th percentile service time |          keyword_terms_numeric_terms |     4369.19 |     ms |
|                                                     error rate |          keyword_terms_numeric_terms |           0 |      % |
|                                                 Min Throughput |          numeric_terms_numeric_terms |        0.45 |  ops/s |
|                                              Median Throughput |          numeric_terms_numeric_terms |        0.45 |  ops/s |
|                                                 Max Throughput |          numeric_terms_numeric_terms |        0.46 |  ops/s |
|                                        50th percentile latency |          numeric_terms_numeric_terms |     9209.06 |     ms |
|                                        90th percentile latency |          numeric_terms_numeric_terms |     12699.1 |     ms |
|                                       100th percentile latency |          numeric_terms_numeric_terms |     13593.9 |     ms |
|                                   50th percentile service time |          numeric_terms_numeric_terms |     2178.83 |     ms |
|                                   90th percentile service time |          numeric_terms_numeric_terms |     2195.56 |     ms |
|                                  100th percentile service time |          numeric_terms_numeric_terms |     2294.51 |     ms |
|                                                     error rate |          numeric_terms_numeric_terms |           0 |      % |

It doesn't fall over but it ain't so happy.

@hub-cap
Copy link
Copy Markdown

hub-cap commented May 28, 2020

oh, indeed they do choke up on the local, im using a multiple machine setup. Ill circle back to find out which of these scenarios is what we should commit to the repo.

Copy link
Copy Markdown

@hub-cap hub-cap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, but names need to be kebap case and not snake case

@nik9000
Copy link
Copy Markdown
Member Author

nik9000 commented May 29, 2020

Looks great, but names need to be kebap case and not snake case

I've stabbed my words.

}
}
}

No newline at end of file
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trailing newline pls

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

disregard i dumb.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Me too! What a coincidence!

}
}
}

No newline at end of file
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

disregard i dumb.

@hub-cap hub-cap changed the title Add track for terms agg Add track for sub-bucket-aggs Jun 2, 2020
@hub-cap hub-cap merged commit 4ae35be into elastic:master Jun 2, 2020
@hub-cap hub-cap added the backport pending Awaiting backport to stable release branch label Jun 2, 2020
hub-cap pushed a commit that referenced this pull request Jun 3, 2020
Adds a rally track specifically for testing the performance of the bucket
aggs when they are "sub" aggs.

Relates to elastic/elasticsearch#55873
@hub-cap hub-cap added backport pending Awaiting backport to stable release branch and removed backport pending Awaiting backport to stable release branch labels Jun 3, 2020
hub-cap pushed a commit that referenced this pull request Jun 3, 2020
Adds a rally track specifically for testing the performance of the bucket
aggs when they are "sub" aggs.

Relates to elastic/elasticsearch#55873
hub-cap pushed a commit that referenced this pull request Jun 3, 2020
Adds a rally track specifically for testing the performance of the bucket
aggs when they are "sub" aggs.

Relates to elastic/elasticsearch#55873
@hub-cap hub-cap removed the backport pending Awaiting backport to stable release branch label Jun 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants