Skip to content

Fix two scripted_metric bugs#58547

Merged
nik9000 merged 3 commits intoelastic:masterfrom
nik9000:scripted_metric_mem_fixup
Jun 25, 2020
Merged

Fix two scripted_metric bugs#58547
nik9000 merged 3 commits intoelastic:masterfrom
nik9000:scripted_metric_mem_fixup

Conversation

@nik9000
Copy link
Copy Markdown
Member

@nik9000 nik9000 commented Jun 25, 2020

Fixes two bugs introduced by #57627:

  1. We were not properly letting go of memory from the request breaker
    when the aggregation finished.
  2. We no longer supported totally arbitrary stuff produced by the init
    script because we assumed that it'd be ok to run the script once
    and clone its results. Sadly, cloning can't clone anything that the
    init script can make, like String arrays. This runs the init script
    once for every new bucket so we don't need to clone.

Fixes two bugs introduced by elastic#57627:
1. We were not properly letting go of memory from the request breaker
   when the aggregation finished.
2. We no longer supported totally arbitrary stuff produced by the init
   script because we *assumed* that it'd be ok to run the script once
   and clone its results. Sadly, cloning can't clone *anything* that the
   init script can make, like `String` arrays. This runs the init script
   once for every new bucket so we don't need to clone.
@elasticmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-analytics-geo (:Analytics/Aggregations)

@elasticmachine elasticmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Jun 25, 2020
@nik9000
Copy link
Copy Markdown
Member Author

nik9000 commented Jun 25, 2020

I've marked this a non-issue because it fixes an unreleased bug.It was still a bug though!

Copy link
Copy Markdown
Contributor

@jimczi jimczi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left one question

private CircuitBreakerService circuitBreakerService;

@Before
public void mockBreaker() {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could check this for all aggs in the AggregatorTestCase directly ?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you be ok if I moved it in there in a followup? I think there are quite a few fiddly things involved in getting that right.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, it might uncovered some other bugs so be prepared ;).


@Override
public void close() {
public void doClose() {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

;)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm dreaming of a checkstyle rule.....

Copy link
Copy Markdown
Contributor

@jimczi jimczi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

nik9000 added 2 commits June 25, 2020 11:20
Adds code to handle when the init script mutates the aggregation
parameters. I hadn't caught this in my initial testing and it is
*weird*, but we allow it.
@nik9000 nik9000 merged commit 2e85723 into elastic:master Jun 25, 2020
nik9000 added a commit to nik9000/elasticsearch that referenced this pull request Jun 25, 2020
Fixes two bugs introduced by elastic#57627:
1. We were not properly letting go of memory from the request breaker
   when the aggregation finished.
2. We no longer supported totally arbitrary stuff produced by the init
   script because we *assumed* that it'd be ok to run the script once
   and clone its results. Sadly, cloning can't clone *anything* that the
   init script can make, like `String` arrays. This runs the init script
   once for every new bucket so we don't need to clone.
nik9000 added a commit that referenced this pull request Jun 25, 2020
Fixes two bugs introduced by #57627:
1. We were not properly letting go of memory from the request breaker
   when the aggregation finished.
2. We no longer supported totally arbitrary stuff produced by the init
   script because we *assumed* that it'd be ok to run the script once
   and clone its results. Sadly, cloning can't clone *anything* that the
   init script can make, like `String` arrays. This runs the init script
   once for every new bucket so we don't need to clone.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Analytics/Aggregations Aggregations >non-issue Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v7.9.0 v8.0.0-alpha1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants