Skip to content

Catch and re-throw Throwable rather than using a success boolean#14633

Merged
ChrisHegarty merged 8 commits intoapache:mainfrom
thecoop:success-exception-handling
May 13, 2025
Merged

Catch and re-throw Throwable rather than using a success boolean#14633
ChrisHegarty merged 8 commits intoapache:mainfrom
thecoop:success-exception-handling

Conversation

@thecoop
Copy link
Contributor

@thecoop thecoop commented May 9, 2025

The use of a boolean success parameter is common in the Lucene codebase. This can be replaced with a catch (Throwable t) {...; throw t} pattern that means a boolean doesn't need to be used at all, and it generally results in less code. This is helped by some new methods on IOUtils.

As a side-effect, any problems that occur whilst closing/deleting during exception handling will now be added to the top-level rethrown exception, rather than being swallowed and disappearing.

I've converted a few here to show the pattern - if this is a direction we want to go in, I can convert more.

@uschindler
Copy link
Contributor

Hi,
In general at the time when the success code was written a catch/rethrow of Throwable would not compile without an inappropriate throws clause on the method. Because the java compiler got better in type tracking, the rethrow would not impose that a Throwable needs to be declared on the method declaration, as the compiler knows which checked exceptions could appear in the catch clause and can restrict types of possible checked exceptions.

So in general I am +1 to do this. Code is cleaner and the suppressed exceptions are also much better.

I have not checked modified logic in the PR, but the overall direction looks fine.

Copy link
Contributor

@uschindler uschindler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve with minor comments.

@thecoop
Copy link
Contributor Author

thecoop commented May 9, 2025

@uschindler PR updated. I've applied this refactor to the most recent set of formats, as those are most likely to be copied in future format changes. That seems like a good place to leave this first PR.

@uschindler
Copy link
Contributor

I have a suggestion. I used the same approach in the expressions module to patch the stack trace and rethrowing exceptions (to include the source code of the exception into the stack trace). So basically at moment you always use code like this:

    } catch (Throwable t) {
      IOUtils.closeWhileSuppressingExceptions(t, docIn, posIn, payIn);
      throw t;
    }

It would be much nicer like this:

    } catch (Throwable t) {
      throw IOUtils.closeWhileSuppressingExceptions(t, docIn, posIn, payIn);
    }

This can be done by using generics on the method:

  public static <T extends Throwable> T closeWhileSuppressingExceptions(T ex, Iterable<? extends Closeable> objects) {
    for (Closeable object : objects) {
      try {
        if (object != null) {
          object.close();
        }
      } catch (Throwable e) {
        ex.addSuppressed(e);
      }
    }
   return ex;
  }

This allows the compiler to still take record of the exceptions and the code looks much better because you actually see that the method modifies the exception.

If you don't want to use the return type, you can still rethrow the conventional way.

@uschindler
Copy link
Contributor

P.S.: Here is the bytecode of the Expression patching: #14602 (comment)

Small note: As this is generated bytecode the generics on the static patcher method are not needed, so left out.

@thecoop
Copy link
Contributor Author

thecoop commented May 9, 2025

Unfortunately that doesn't work here - Java forgets the specific exception types thrown in the try when the exception goes in & out of the generic method - it infers T to Throwable, and now the method returns (and so throws) Throwable as the raw type

@uschindler
Copy link
Contributor

Unfortunately that doesn't work here - Java forgets the specific exception types thrown in the try when the exception goes in & out of the generic method - it infers T to Throwable, and now the method returns (and so throws) Throwable as the raw type

That is strange..... I have to dig further. I did this a while ago in a similar way.

@uschindler
Copy link
Contributor

uschindler commented May 9, 2025

Indeed that does not work. I was under the impression that the compiler should be able to see that the method parameter has same type as return type by the generics. Looks like that's not fully solved by the compiler people. Especally as it gets harder for multi-catch.

I thought it did not work with multi-catch, but that it works here. Of course you could make a more horrible hack, but some people don't like this in the Lucene community (its known as sneaky rethrow....).

@uschindler
Copy link
Contributor

uschindler commented May 9, 2025

The reason for that problem is in the JLS: https://docs.oracle.com/javase/specs/jls/se24/html/jls-11.html#jls-11.2.2

A throw statement whose thrown expression is a final or effectively final exception parameter of a catch clause C can throw an exception class E iff:

  • E is an exception class that the try block of the try statement which declares C can throw; and

  • E is assignment compatible with any of C's catchable exception classes; and

  • E is not assignment compatible with any of the catchable exception classes of the catch clauses declared to the left of C in the same try statement.

The problem is: When it passes through the method it is no longer final/effectivly final. The method could change the type of exception and return something else while still assignment compatible with regards to the generics.

@thecoop thecoop force-pushed the success-exception-handling branch from 5cf461f to e088619 Compare May 9, 2025 14:06
@thecoop thecoop requested a review from uschindler May 9, 2025 14:44
@mikemccand
Copy link
Member

+1 to eliminate the crazy success pattern! I hope we can also eliminate even the success2 cases :)

Thank you @thecoop for fixing this!

Copy link
Contributor

@dweiss dweiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pattern exists because the code predates try-with-resources, like Uwe mentioned. I don't mind changing it, especially where multi-statement try-with-resources can be used. Thank you.

Copy link
Contributor

@uschindler uschindler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi,
the general pattern looks fine, although we might think on improving it: The Java designers nowadays would not have added try-with-resources! Instead they possibly would have made it a method call to some tryWithResources utility method taking lambdas. We could implement something similar for a "try-cleanup-on-throwable" pattern as a mthod in IOUtils which takes a lambda for the main body. I have not yet thought closer about it, but might make the code look nicer and better express what it is doing.

I skimmed through the changes and did not find any bad things. Thanks for adding special handling of errors, I think that was the reason for the partly failing tests during development?!

+1 to merge this.

@rmuir
Copy link
Member

rmuir commented May 10, 2025

I thought it did not work with multi-catch, but that it works here. Of course you could make a more horrible hack, but some people don't like this in the Lucene community (its known as sneaky rethrow....).

But what if the horrible hack (sneakythrow) could be contained to just be inside these utility methods (IOUtils), to simplify exception handling elsewhere?

IMO this would be worth it, even if it saves just one line at these call sites. Historically the exception handling code has been problematic (easy to leak files or mask important information, accidentally suppress the wrong thing, etc etc)...

Maybe we should look into it as a followup?

@thecoop
Copy link
Contributor Author

thecoop commented May 12, 2025

I definitely agree that having to have a separate throw t as part of the exception handling is trappy (although generally ok due to flow-control changes breaking the compile without it), and I would prefer to have a single line or use try-resources with lambdas in some way, ideally without putting the whole try block in a lambda. I'll have a play around with it & see what happens.

@ChrisHegarty ChrisHegarty merged commit 7d3c7bb into apache:main May 13, 2025
8 checks passed
weizijun added a commit to weizijun/lucene that referenced this pull request May 16, 2025
* main: (31 commits)
  Fix termination condition in TestStressNRTReplication. (apache#14665)
  deps(java): bump com.gradle.develocity from 3.19 to 3.19.2 (apache#14662)
  Build: remove hard-coded Java versions from ecj.javadocs.prefs (apache#14651)
  Update verifier comment to show label (apache#14658)
  Catch and re-throw Throwable rather than using a success boolean (apache#14633)
  Mention label in changelog verifier comment (apache#14656)
  Enable PR actions in changelog verifier (apache#14644)
  Fix FuzzySet#getEstimatedNumberUniqueValuesAllowingForCollisions to properly account for hashCount (apache#14614)
  Don't perform additional KNN querying after timeout, fixes apache#14639 (apache#14640)
  Add instructions to help/IDEs.txt for VSCode and Neovim (apache#14646)
  build(deps): bump ruff from 0.11.7 to 0.11.8 in /dev-tools/scripts (apache#14603)
  deps(java): bump de.jflex:jflex from 1.8.2 to 1.9.1 (apache#14583)
  Use the preload hint on completion fields and memory terms dictionaries. (apache#14634)
  Clean up FileTypeHint a bit. (apache#14635)
  Expressions: Improve test to use a fully private class or method
  Remove deprecations in expressions (apache#14641)
  removing constructor with deprecated attribute 'onlyLongestMatch (apache#14356)
  Moving CHANGES entry for apache#14609 from 11.0 to 10.3 (apache#14638)
  Overrides rewrite in PointRangeQuery to optimize AllDocs/NoDocs cases (apache#14609)
  Adding benchmark for histogram collector over point range query (apache#14622)
  ...

# Conflicts:
#	lucene/CHANGES.txt
ChrisHegarty pushed a commit that referenced this pull request May 29, 2025
…dates (#14668)

Followon from #14633 - refactor uses of success and success2 booleans.
@thecoop thecoop deleted the success-exception-handling branch June 20, 2025 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants