Skip to content

Save memory when date_histogram is not on top#56921

Merged
nik9000 merged 2 commits intoelastic:masterfrom
nik9000:date_histo_mem
May 19, 2020
Merged

Save memory when date_histogram is not on top#56921
nik9000 merged 2 commits intoelastic:masterfrom
nik9000:date_histo_mem

Conversation

@nik9000
Copy link
Copy Markdown
Member

@nik9000 nik9000 commented May 18, 2020

When date_histogram is a sub-aggregator it used to allocate a bunch of
objects for every one of it's parent's buckets. This uses the data
structures that we built in #55873 rework the date_histogram
aggregator instead of all of the allocation.

Part of #56487.

@nik9000
Save memory when date_histogram is not on top
09e35fc 
When `date_histogram` is a sub-aggregator it used to allocate a bunch of
objects for every one of it's parent's buckets. This uses the data
structures that we built in elastic#55873 rework the `date_histogram`
aggregator instead of all of the allocation.

Part of elastic#56487
@nik9000 nik9000 requested a review from talevy May 18, 2020 19:58
@elasticmachine
Copy link
Copy Markdown
Collaborator

elasticmachine commented May 18, 2020

Pinging @elastic/es-analytics-geo (:Analytics/Aggregations)

@elasticmachine elasticmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label May 18, 2020
@nik9000
Copy link
Copy Markdown
Member Author

nik9000 commented May 18, 2020 •
edited
Loading

Unlike when I did this for significant_term (#56789) or terms (#55873) this doesn't really make the agg go any faster. The performance change is pretty much a wash so far as I can tell. It is a little worse when there are maybe 5 or 10 owning buckets and a little better when there are hundreds. But it is something we can build on!

This was referenced May 18, 2020
Closed
Merged
@nik9000 nik9000 merged commit bea2341 into elastic:master May 19, 2020
@nik9000
Copy link
Copy Markdown
Member Author

nik9000 commented May 19, 2020

Thanks @polyfractal !

nik9000 added a commit to nik9000/elasticsearch that referenced this pull request May 19, 2020
@nik9000
Save memory when date_histogram is not on top (elastic#56921)
272f9ee 
When `date_histogram` is a sub-aggregator it used to allocate a bunch of
objects for every one of it's parent's buckets. This uses the data
structures that we built in elastic#55873 rework the `date_histogram`
aggregator instead of all of the allocation.

Part of elastic#56487
@nik9000
Copy link
Copy Markdown
Member Author

nik9000 commented May 19, 2020

Thanks @talevy !

nik9000 added a commit that referenced this pull request May 19, 2020
@nik9000
Save memory when date_histogram is not on top (#56921) (#56960)
8b9c4eb 
When `date_histogram` is a sub-aggregator it used to allocate a bunch of
objects for every one of it's parent's buckets. This uses the data
structures that we built in #55873 rework the `date_histogram`
aggregator instead of all of the allocation.

Part of #56487
nik9000 added a commit to nik9000/elasticsearch that referenced this pull request May 19, 2020
@nik9000
Update skip after backport of elastic#56921
b93107d
nik9000 added a commit that referenced this pull request May 20, 2020
@nik9000
Update skip after backport of #56921 (#56974)
7790e81
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@talevy talevy talevy approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

:Analytics/Aggregations Aggregations >enhancement Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v7.9.0 v8.0.0-alpha1

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@nik9000 @elasticmachine @talevy @jakelandis