Exceptions from scripts are not handled well today:

• There is a lot of repetitive unnecessary boxing and wrapping of exceptions at multiple layers.
• Despite this verbosity, no stacktrace information is given by default (!). Users of this are like "developers" and need to know where in their script something went wrong.
• If the user passes the rest parameter (which is very long) to enable stacktrace output, they get a massive verbose toString, containing all kinds of irrelevant ES elements, and formatted as a terribly ugly massive string.
• ScriptException is meaningless, it doesn't provide any more structure over a RuntimeException!
• json is the worst possible way to transmit source code. We can count on having no line numbers (which always makes stacktraces meaningful), instead just everything shoved on one massive line.

The current situation just encourages even more bad exception handling. People complain if you get NullPointerException when you dereference a null pointer, but that is exactly what you deserve if you do this! And it tells you all you need to figure it out: its right there in the stacktrace with line numbers!

We should just "box" the exception a single time, with all the stuff you need to figure out the problem by default. That means telling you what script hit the problem, what language it was in, a "simple" reason of what happened, and a relevant "stacktrace" to the script. The original root cause is always preserved, if someone asks for the stacktrace or looks at the logs, they will see all that really happened, this is more about the horror of whats happening today at the REST layer.

In order to contend with "everything shoved on a single line", we have to do some highlighting. Normally, line numbers and existing lines work great for code, but since we don't have those, we have to work with what we've got.

For painless that means, encoding offsets as line numbers into the bytecode, wherever an exception can strike. It also means breaking down the (probably massive) script into sentences that have enough meaningful context: we use leaf S* nodes as "sentences".

For expressions it means when a "variable" goes wrong (e.g. not in mappings), we just highlight what went wrong with that variable definition, etc.

Other scripting engines can be adapted to use this, e.g. you can get some of this kind of info from at least some groovy/javascript/python exceptions. For now, they still keep using the deprecated "GeneralScriptException" and do the same as before. If someone else wants to improve them, great.

Here are some example runtime errors:

{
  "type" : "script_exception",
  "reason" : "runtime error",
  "caused_by" : {
    "type" : "unsupported_operation_exception",
    "reason" : null
  },
  "script_stack" : [
    "java.util.AbstractList.add(AbstractList.java:148)"
    "java.util.AbstractList.add(AbstractList.java:108)"
    "return x.add(5);",
    "        ^---- HERE"
  ],
  "script" : "def x = Collections.emptyList(); return x.add(5);",
  "lang" : "painless"
}

{
  "type" : "script_exception",
  "reason" : "runtime error",
  "caused_by" : {
    "type" : "null_pointer_exception",
    "reason" : null
  },
  "script_stack" : [
    "x = Math.log(Math.abs(ctx.sometypo.getSomething())); ",
    "                                  ^---- HERE"
  ],
  "script" : "double y = ctx.thing; double x = Math.log(Math.abs(ctx.sometypo.getSomething())); return x * 5 + y;",
  "lang" : "painless"
}

It helps also for compile-time errors (and the "link time" of expressions where it binds against the mappings):

{
  "type" : "script_exception",
  "reason" : "compile error",
  "caused_by" : {
    "type" : "parse_exception",
    "reason" : "invalid sequence of tokens near '*' on line (1) position (17)",
    "caused_by" : {
      "type" : "no_viable_alt_exception",
      "reason" : null
    }
  },
  "script_stack" : [
    "doc['d'].value * *@#)(@$*@#$ + 4",
    "                 ^---- HERE"
  ],
  "script" : "doc['d'].value * *@#)(@$*@#$ + 4",
  "lang" : "expression"
}

{
  "type" : "script_exception",
  "reason" : "link error",
  "caused_by" : {
    "type" : "parse_exception",
    "reason" : "Field [e] does not exist in mappings"
  },
  "script_stack" : [
    "doc['e'].value",
    "     ^---- HERE"
  ],
  "script" : "doc['e'].value * 5",
  "lang" : "expression"
}