The parameter is expectedFailuresCount and it should be 0. The callable's call() function should not be invoked.

I wonder how useful is this verification now as we are basically verifying the behavior of our own MockCallable. As you mentioned in other comments, we are always checking that the future is not done before submitting it to an retrying executor, so the current flakiness is actually expected since we are not checking if the future is done again in the submit method itself.
That being said, I don't think we want to mimic the behavior in MockCallable as that's outside the scope of the unit tests of AbstractRetryingExecutorTest. My suggestion is that we keep this test case but remove this line of verification if we don't find any other benefit.

MockCallable's tracer is to mimic the functionality inside AttemptCallable (which is what I believe FailingCallable is based on): https://github.com/googleapis/gapic-generator-java/blob/4bb0a5e3ce78878841e87a47cd457995ea9ed87e/gax-java/gax/src/main/java/com/google/api/gax/rpc/AttemptCallable.java#L78-L85

The tracer's functionality for the MockCallable can also be covered with the assertion below: assertEquals(0, future.getAttemptSettings().getAttemptCount());

so I'm fine with removing.

Is this a required change? If not, I would leave it out of this PR, and create another PR for updating the mockito version.

I believe this was failing the CI when running the integration tests. I can upgrade this in a separate PR.

If we don't verify the tracer anymore, do we still need this new MockCallable? Can we keep using FailingCallable? I think the flakiness was only from verifyNoMoreInteractions()?

Ah yep! Actually, on a second look I was thinking that we could add in a reference for the externalFuture for FailingCallable and check that the externalFuture's isDone() is true. It would keep it similar to how it's done for AttemptCallable: https://github.com/googleapis/gapic-generator-java/blob/99ade2be85d33e166ea31ea803dffcef2d402058/gax-java/gax/src/main/java/com/google/api/gax/rpc/AttemptCallable.java#L79-L81. However, this would add changes for all the other tests that use FailingCallable. We would be able to keep the verifyNoMoreInteractions() and ensure the tracer logic isn't run.

If this is too much, I can revert the previous commit and just remove in the verify() call. WDYT?

I like the idea, I see that you already updated with the new approach.

Can we still verify it as the last step like previously? Or it has to be verified earlier now?

I think it makes sense to verify it here (before the executor submits the future) to ensure that the callable's logic isn't invoked (not invoking == exiting once it realizes the external future is done) after it's been submitted for execution.

Otherwise we can use verifyNoInteractions(tracer) and put that at the end which requires an updating mockito version in dependencies.properties. Functionality-wise, they should be the same so I just used verifyNoMoreInteractions which already exists

Are they the same functional-wise? The callable would never run until executor.submit(future), and I think the purpose of this test is to verify that the inner callable would never run if the future is already cancelled, despite that the future is submitted. So verifying after executor.submit(future) makes more sense to me.

The inner callable does run, but does not execute any logic -- call() is invoked but then immediately exited as the external future is always done. Since this is the first attempt, the random retry delay is set to 0 and is scheduled to execute immediately. I was thinking that this would create a possible race between when the callable exits and when we run verifyNoMoreInteractions.

If call() runs before verifyNoMoreInteractions then there wouldn't be a point of running verifyNoMoreInteractions as the callable has already exited
If call() runs after verifyNoMoreInteractions, then this line would confirm the behavior.

(From my understanding) We can't guarantee when the scheduleExecutor executes the callable (either before or after verifyNoMoreInteractions in the test) so asserting verifyNoMoreInteractions before the execution would confirm that the call() ran and that the tracer never ran.

Which line would invoke call() immediately? I still think the callable is not invoked until executor.submit(future) based on the Javadoc

Er* It should be invoked immediately once executor.submit() runs:
https://github.com/googleapis/gapic-generator-java/blob/59efbbbd485282e37abbc5eed87b72a15a33240e/gax-java/gax/src/main/java/com/google/api/gax/retrying/ScheduledRetryingExecutor.java#L115-L119 as retryingFuture.getAttemptSettings().getRandomizedRetryDelay().toMillis() -> 0ms for the first attempt

I looked up the verifyNoMoreInteractions javadoc and I misunderstood the use case. I assumed it was a point-in-time verification to ensure there was no more interactions once it was invoked (the race logic I mentioned above), but it's meant to ensure that there were no more interactions after we've verified that things have already run/ executed/ invoked a certain number of times.

I think having either verifyNoInteractions or verifyNoMoreInteractions at the end of the file should work. May make more sense to have this be verifyNoInteractions instead of verifyNoMoreInteractions for clarity (tracer should have 0 interactions and no more implies more than 0) but that would require Mockito 4.x in the dependencies.properties file. Either one is fine with me and I'll just add a small comment in the test file.