[DOCS] Add watcher examples by hub-cap · Pull Request #36565 · elastic/elasticsearch

hub-cap · 2018-12-12T17:52:33Z

No description provided.

elasticmachine · 2018-12-12T17:52:35Z

Pinging @elastic/es-core-infra

hub-cap · 2018-12-12T17:53:00Z

docs/painless/painless-contexts/painless-watcher-condition-context.asciidoc

+You can then use the example watch to retrieve and manipulate total spend
+calculations for each play present in the example data set.
+
+The watcher condition can be used in two places, and below are examples of each.


I dont really like this sentence, and would like advice on whether its needed.

hub-cap · 2018-12-12T17:54:21Z

I hope this is in the right direction. I would love some feedback from the docs team and @jdconrad on whether this is good or not. Next up is transform, and I will be doing something similar to this for transform. Feedback welcome!!!

jdconrad · 2018-12-12T18:00:36Z

Adding @debadair for docs review.

jdconrad · 2018-12-12T18:01:20Z

@hub-cap Thanks for the work so far! Much appreciated.

hub-cap · 2018-12-12T18:07:26Z

while munching a sandwich at lunch I decided I did not like these docs. I think they should be structured differently. Give me some time to formulate, but lets not worry about review for now.

hub-cap · 2018-12-12T22:40:41Z

So my thought is to consolidate both of the watcher contexts into 1 document. Both of the contexts share the same variables accessible within the context. So I figured I can use a single example to show both the transform and the condition, as well as highlight all the values within the context. Its going to take me a little while to glean all the values that the context may have, so it could be another day until I get an update here. Lets wait on any review until then.

honzakral · 2018-12-13T23:29:10Z

Thank you so much for these examples, they will be super useful to users trying to orient themselves.

For completeness I have a similar (in functionality) pattern that I find using over an over in watchers that collect potentially multiple events (which is vast majority in many use cases):

def events = [];
for (def host: ctx.payload.aggregations.hosts.buckets) {
  if (host.max_cpu.value > 90) {
    events.add([
      // add generic metadata about the event
      '_id': ctx.watch_id + '-' + host.key,
      'watch': ctx.watch_id,
      '@timestamp': ctx.trigger.triggered_time,
      // and all the values that contributed to it being an event
      'host': host.key,
      'cpu': host.max_cpu.value
    ]);
  }
}
ctx.payload.events = events;
return events.size() > 0;

I store the resulting list on the payload so that I don't have to repeat the same exact logic in a transform down the line when I want to act. I can pass this directly to an index action or use it in a template for logging or email actions, alternatively pass it to a webhook via {{#toJson}}ctx.payload.events{{/toJson}} keeping all my logic in a single location.

Because the stream API is not as well known among people not fluent in Java I purposefully avoid it and stick with the super basic forloop which a larger number of peoplecan read. (hopefully that will change if we document it here though :) )

Refactor out the variables, and reuse them in the 2 contexts, as well as adding a complete example to each of the actual docs.

…condition

hub-cap · 2018-12-17T19:48:50Z

@honzakral im considering adding this for entry, i just need to find out where is a good place for it. Maybe I can add it in the "complete examples" section.

@debadair and @jdconrad, I have restructured it such that the variables section now has its own file and that is imported into both of the context examples. They also both have a "complete example" imported in at the end so the user can actually understand a full watch example with both conditions and transforms. I think this is probably a good place to put the "for loop" example from honza above, as well. As is, I think its ready for ripping my horrible use of the english language apart.

jdconrad

Looks good to me from the tech side. Left a few comments on wording and such. Thanks so much for doing this! I really appreciate you going through the variables and adding hefty docs for them as well. This should be really helpful for our users.

jdconrad · 2018-12-17T20:13:50Z

docs/painless/painless-contexts/painless-watcher-condition-context.asciidoc

-        Any metadata associated with the watch.
+<1> The Java Stream API is used in the condition. This API allows manipulation of
+the elements of the list in a pipeline.
+<2> A Stream Filter is used to remove items that do not meet the criteria within the


Should stream filter be capitalized?

nope :-)

<2> The stream filter removes items that do not meet the filter criteria.

jdconrad · 2018-12-17T20:14:21Z

docs/painless/painless-contexts/painless-watcher-condition-context.asciidoc

+the elements of the list in a pipeline.
+<2> A Stream Filter is used to remove items that do not meet the criteria within the
+filter.
+<3> The collector reduces the stream back into a java.util.List


add back ticks to java.util.List

And then add a period. I'd also say "reduces the stream to a" rather than "back into a".

jdconrad · 2018-12-17T20:27:49Z

docs/painless/painless-contexts/painless-watcher-condition-context.asciidoc

-`ctx['trigger']['scheduled_time']` (`ZonedDateTime`, read-only)::
-        The scheduled trigger time for the watch.
+The first example is a condition for the watch itself. This determines if any
+of the actions in the watch are executed. The example aggregates the total sold


This may need to be re-worded as it sounds like you can select individual actions to execute for a watch based on a single condition.

In other places, we use an "Example" label before the example intro (like Return and API).

Rather than "first" and "second", I'd just use "the following" to introduce the examples.

Example

The following watch condition script controls overall execution of the watch based on the value of the seats sold for the plays in the data set. The script aggregates the total sold seats for each play and returns true if there is at least one play that has sold under $15,000 or over $50,000.

jdconrad · 2018-12-17T20:28:54Z

docs/painless/painless-contexts/painless-watcher-condition-context.asciidoc

-*Return*
+The next example is a condition on an action. This determines if a distinct action
+in the watch are executed. The example aggregates the total sold seats for each play
+in the data set. The action condition validates that there is at least one play that


Can you be more descriptive here about the aggregation being executed? Aggregates the total seats how?

This is a bit redundant. I'd used the pattern from the watch condition intro:

The following action condition script controls execution of the my_log action based on the value of the seats sold for the plays in the data set. The script aggregates the total sold seats for each play and returns true if there is at least one play that has sold over $50,000.

jdconrad · 2018-12-17T20:30:03Z

docs/painless/painless-contexts/painless-watcher-context-example.asciidoc

@@ -0,0 +1,112 @@
+==== Complete Example
+
+While these examples are contrived, a more complete watch shown below contains


Don't think you need to call out the example as contrived as that makes it sound like it's not really useful. Short examples often don't represent real world scenarios.

Yeah, we generally avoid "meta info" like that as much as possible.

jdconrad · 2018-12-17T20:31:10Z

docs/painless/painless-contexts/painless-watcher-context-variables.asciidoc

+
+==== Time values in a watch
+
+There are 3 time values that are available as part of the context. The


three instead of 3?

hub-cap · 2018-12-18T15:02:56Z

@spinscale if you would like to take a look as well, id be delighted :)

spinscale

I left a few comments for possible improvements, nothing major though

spinscale · 2018-12-19T12:35:56Z

docs/painless/painless-contexts/painless-watcher-condition-context.asciidoc

+      "script" :
+      """
+        return ctx.payload.aggregations.theatres.buckets.stream()       <1>
+          .filter(t -> t.money.value < 15000 || t.money.value > 50000)  <2>


maybe replace the t with threatre or bucket?

spinscale · 2018-12-19T12:37:07Z

docs/painless/painless-contexts/painless-watcher-condition-context.asciidoc

+        return ctx.payload.aggregations.theatres.buckets.stream()       <1>
+          .filter(t -> t.money.value < 15000 || t.money.value > 50000)  <2>
+          .collect(Collectors.toList())                                 <3>
+          .size() > 0                                                   <4>


no need to create a list first and then use size, just use .count()?

spinscale · 2018-12-19T12:39:26Z

docs/painless/painless-contexts/painless-watcher-condition-context.asciidoc

+          """
+            return ctx.payload.aggregations.theatres.buckets.stream()
+            .filter(t -> t.money.value > 50000)                       <2>
+            .collect(Collectors.toList())


hm, you don't even need .count(). You can use return ctx.payload.aggregations.theatres.buckets.stream().anyMatch(b -> b.money.value > 50000)`

spinscale · 2018-12-19T12:40:18Z

docs/painless/painless-contexts/painless-watcher-context-example.asciidoc

+          .filter(t -> t.money.value < ctx.metadata.low_threshold ||
+                       t.money.value > ctx.metadata.high_threshold)
+          .collect(Collectors.toList())
+          .size() > 0


again, use Stream.anyMatch()

spinscale · 2018-12-19T12:45:13Z

docs/painless/painless-contexts/painless-watcher-context-example.asciidoc

+            'msg': ctx.payload.money_makers.stream()
+              .map(t-> formatter.format(t.total_value) + ' for the play ' + t.play)
+              .collect(Collectors.toList())
+              .join(", ")


just do Collectors.joining(' FOO ') instead of collect and join?

spinscale · 2018-12-19T12:47:31Z

docs/painless/painless-contexts/painless-watcher-context-variables.asciidoc

+==== Time values in a watch
+
+There are 3 time values that are available as part of the context. The
+execution_time is the actual time the watch is executed. The other two


backticks around execution time?

spinscale · 2018-12-19T12:49:20Z

docs/painless/painless-contexts/painless-watcher-context-variables.asciidoc

+    "transform" : {
+      "script":
+      """
+        return [ 'elements': ctx.payload.aggregations.theatres.buckets.size() ]


this transform is also not needed, one could call {{ctx.payload.aggregations.theatres.buckets.size}} to get the same result.

spinscale · 2018-12-19T12:50:05Z

docs/painless/painless-contexts/painless-watcher-transform-context.asciidoc

+      }
+    },
+    "actions" : {
+      "mod_log" : {


maybe use the long form for the action names here?

spinscale · 2018-12-19T12:51:08Z

docs/painless/painless-contexts/painless-watcher-transform-context.asciidoc

+          "text" : "The output of the payload was transformed to: {{ctx.payload.msg}}"
+        }
+      },
+      "unmod_log" : {                                                                 <2>


long name as well?

spinscale · 2018-12-19T12:51:36Z

docs/painless/painless-contexts/painless-watcher-transform-context.asciidoc

+            'msg': ctx.payload.aggregations.theatres.buckets.stream()
+              .map(t-> formatter.format(t.money.value) + ' for the play ' + t.key)
+              .collect(Collectors.toList())
+              .join(", ")


Collectors.joining(', ')?

debadair · 2018-12-20T00:33:07Z

docs/painless/painless-contexts/painless-watcher-condition-context.asciidoc

+will return a `boolean` value to determine if the condition was met. A
+condition can be used to short circuit an entire watch or within an action
+to determine execution of said action.



In general, we prefer present tense and active voice. I'd also consistently use either "watch condition" rather than flip-flopping between it and "Watcher condition". The last sentence and the first are largely redundant. I'd probably go with:

Use a Painless script as a {xpack-ref}/condition-script.html[watch condition]
that determines whether to execute a watch or a particular action within a watch.
Condition scripts return a Boolean value to indicate the status of the condition.

I'd bump this up to follow the Watch condition context heading, before the context variables.

debadair · 2018-12-20T00:35:09Z

docs/painless/painless-contexts/painless-watcher-condition-context.asciidoc

-        User-defined parameters passed in as part of the query.
+`boolean`::
+        Expects `true` if the condition is met, and `false` otherwise.



I'd say "Returns true if the condition is met, false if it is not."

debadair · 2018-12-20T00:37:34Z

docs/painless/painless-contexts/painless-watcher-context-example.asciidoc

@@ -0,0 +1,112 @@
+==== Complete Example
+
+While these examples are contrived, a more complete watch shown below contains


Yeah, we generally avoid "meta info" like that as much as possible.

debadair · 2018-12-20T01:00:48Z

docs/painless/painless-contexts/painless-watcher-condition-context.asciidoc

-`ctx['trigger']['scheduled_time']` (`ZonedDateTime`, read-only)::
-        The scheduled trigger time for the watch.
+The first example is a condition for the watch itself. This determines if any
+of the actions in the watch are executed. The example aggregates the total sold


In other places, we use an "Example" label before the example intro (like Return and API).

Rather than "first" and "second", I'd just use "the following" to introduce the examples.

Example

The following watch condition script controls overall execution of the watch based on the value of the seats sold for the plays in the data set. The script aggregates the total sold seats for each play and returns true if there is at least one play that has sold under $15,000 or over $50,000.

debadair · 2018-12-20T01:02:09Z

docs/painless/painless-contexts/painless-watcher-condition-context.asciidoc

-        Any metadata associated with the watch.
+<1> The Java Stream API is used in the condition. This API allows manipulation of
+the elements of the list in a pipeline.
+<2> A Stream Filter is used to remove items that do not meet the criteria within the


nope :-)

<2> The stream filter removes items that do not meet the filter criteria.

debadair · 2018-12-20T01:26:05Z

docs/painless/painless-contexts/painless-watcher-condition-context.asciidoc

-*Return*
+The next example is a condition on an action. This determines if a distinct action
+in the watch are executed. The example aggregates the total sold seats for each play
+in the data set. The action condition validates that there is at least one play that


This is a bit redundant. I'd used the pattern from the watch condition intro:

The following action condition script controls execution of the my_log action based on the value of the seats sold for the plays in the data set. The script aggregates the total sold seats for each play and returns true if there is at least one play that has sold over $50,000.

debadair · 2018-12-20T01:26:45Z

docs/painless/painless-contexts/painless-watcher-context-example.asciidoc

@@ -0,0 +1,112 @@
+==== Complete Example


I'd remove the heading & just have this as part of the Example section.

debadair · 2018-12-20T01:37:01Z

docs/painless/painless-contexts/painless-watcher-context-example.asciidoc

+both <<painless-watcher-condition-context, conditions>> and
+<<painless-watcher-transform-context, transforms>>, and is the basis of the
+above examples. It is shown for completeness.
+


Maybe: The following example shows scripted watch and action conditions within the context of a complete watch. This watch also uses a scripted <<painless-watcher-transform-context, transform>>.

I wouldn't link back to the top of this topic.

debadair · 2018-12-20T01:47:59Z

docs/painless/painless-contexts/painless-watcher-context-variables.asciidoc

+    "condition" : {
+      "script" :
+      """
+        return ctx.metadata.high_threshold == 50000     <1>


This and the time values examples seem overly-heavyweight to simply show how to access these values. I'd incorporate the info/example snippet into the context variable descriptions and maybe mention in the intro that condition scripts can access watch metadata and timestamps.

debadair · 2018-12-20T01:49:19Z

docs/painless/painless-contexts/painless-watcher-transform-context.asciidoc

@@ -1,39 +1,162 @@
 [[painless-watcher-transform-context]]
 === Watcher transform context



Pretty much the same set of comments applies to the transform context. :-)

hub-cap · 2018-12-20T02:01:23Z

Thanks to everyone for the reviews! Im going to get on these ASAP tomorrow!

…condition

hub-cap · 2019-01-02T18:43:10Z

I have updated the docs based on the feedback. thanks to @jdconrad @spinscale and @debadair.

the biggest "change" that i did that was not just following update comments was to move all of the common examples (metadata/time values/payload) into their own example. I just stuck it at the end, after the other example. Id like to still show some date formatting in the example, but its not strictly necessary if it does not fit the rest of the flow. Id be happy to remove it if it does not make sense there.

…condition

debadair

LGTM

…condition

* master: Reformat some classes in the index universe [DOCS] Add watcher context examples (elastic#36565)

[DOCS] Add watcher condition examples

14494f5

hub-cap added >docs General docs changes :Core/Infra/Scripting Scripting abstractions, Painless, and Mustache v7.0.0 v6.6.0 labels Dec 12, 2018

hub-cap requested a review from jdconrad December 12, 2018 17:52

hub-cap commented Dec 12, 2018

View reviewed changes

jdconrad requested a review from debadair December 12, 2018 18:00

hub-cap added 2 commits December 17, 2018 13:36

Further cleanup

ac3e67a

Refactor out the variables, and reuse them in the 2 contexts, as well as adding a complete example to each of the actual docs.

Merge remote-tracking branch 'upstream/master' into painless_watcher_…

44cd655

…condition

hub-cap changed the title ~~[DOCS] Add watcher condition examples~~ [DOCS] Add watcher examples Dec 17, 2018

jdconrad approved these changes Dec 17, 2018

View reviewed changes

spinscale reviewed Dec 19, 2018

View reviewed changes

jasontedor added v6.7.0 and removed v6.6.0 labels Dec 19, 2018

debadair suggested changes Dec 20, 2018

View reviewed changes

hub-cap added 3 commits December 27, 2018 14:29

Merge remote-tracking branch 'upstream/master' into painless_watcher_…

55ed768

…condition

Updating based on reviewer comments

02ffdf0

Merge remote-tracking branch 'upstream/master' into painless_watcher_…

0369638

…condition

hub-cap added 2 commits January 10, 2019 09:24

Merge remote-tracking branch 'upstream/master' into painless_watcher_…

2daddce

…condition

Fix zone with new getZone() method

9d58f6b

debadair approved these changes Jan 14, 2019

View reviewed changes

Merge remote-tracking branch 'upstream/master' into painless_watcher_…

f1195be

…condition

hub-cap merged commit c0368a2 into elastic:master Jan 15, 2019

hub-cap added the backport pending label Jan 15, 2019

jasontedor added a commit to jasontedor/elasticsearch that referenced this pull request Jan 15, 2019

Merge branch 'master' into sync-retention-leases

8947835

* master: Reformat some classes in the index universe [DOCS] Add watcher context examples (elastic#36565)

hub-cap added a commit that referenced this pull request Jan 15, 2019

[DOCS] Add watcher context examples (#36565)

409f7ff

hub-cap removed the backport pending label Jan 15, 2019

colings86 added v7.0.0-beta1 and removed v7.0.0 labels Feb 7, 2019

		@@ -0,0 +1,112 @@
		==== Complete Example

		While these examples are contrived, a more complete watch shown below contains


		==== Time values in a watch

		There are 3 time values that are available as part of the context. The

		@@ -1,39 +1,162 @@
		[[painless-watcher-transform-context]]
		=== Watcher transform context

Conversation

hub-cap commented Dec 12, 2018

Uh oh!

elasticmachine commented Dec 12, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hub-cap commented Dec 12, 2018

Uh oh!

jdconrad commented Dec 12, 2018

Uh oh!

jdconrad commented Dec 12, 2018

Uh oh!

hub-cap commented Dec 12, 2018

Uh oh!

hub-cap commented Dec 12, 2018

Uh oh!

honzakral commented Dec 13, 2018

Uh oh!

hub-cap commented Dec 17, 2018

Uh oh!

jdconrad left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hub-cap commented Dec 18, 2018

Uh oh!

spinscale left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment