Snapshot listener gets a regular callback with 'Received unexpected EOS on empty DATA frame from server' exception #822
Description
We are using Firestore snapshot listeners on documents and queries to receive callbacks whenever documents are updated.
The documentation states that "after an error, the listener will not receive any more events, and there is no need to detach your listener“. However, once a snapshot listener is created, we regularly (around every hour at least for documents that do not get any updates) receive an error callback, i.e. EventListener::onEvent with non-null second error argument that is of FirestoreException type that looks like a gRPC idle timeout or similar:
io.grpc.StatusRuntimeException: INTERNAL: Received unexpected EOS on empty DATA frame from server
at io.grpc.Status.asRuntimeException(Status.java:535)
... 10 common frames omitted
Wrapped by: com.google.api.gax.rpc.InternalException: io.grpc.StatusRuntimeException: INTERNAL: Received unexpected EOS on empty DATA frame from server
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:67)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60)
at com.google.api.gax.grpc.ExceptionResponseObserver.onErrorImpl(ExceptionResponseObserver.java:82)
at com.google.api.gax.rpc.StateCheckingResponseObserver.onError(StateCheckingResponseObserver.java:86)
at com.google.api.gax.grpc.GrpcDirectStreamController$ResponseObserverAdapter.onClose(GrpcDirectStreamController.java:149)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:557)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:69)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:738)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:717)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
... 3 common frames omitted
Wrapped by: com.google.api.gax.rpc.ApiException: com.google.api.gax.rpc.InternalException: io.grpc.StatusRuntimeException: INTERNAL: Received unexpected EOS on empty DATA frame from server
at com.google.cloud.firestore.Watch.lambda$closeStream$2(Watch.java:339)
... 6 common frames omitted
Wrapped by: com.google.cloud.firestore.FirestoreException: com.google.api.gax.rpc.InternalException: io.grpc.StatusRuntimeException: INTERNAL: Received unexpected EOS on empty DATA frame from server
at com.google.cloud.firestore.FirestoreException.forApiException(FirestoreException.java:104)
at com.google.cloud.firestore.Watch.lambda$closeStream$2(Watch.java:336)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:831)
If we do not "handle" these errors by recreating the snapshot listener, the callback would in most of the cases still be called on a snapshot listener upon update regardless of the fact that previously there was a callback with the EOS error invoked already invoked on the same listener. In other words, it seems that there is no need to treat EOS exceptions as errors upon which "the listener will not receive any more events“, but this neither documented nor expected behaviour.
Environment details
Platform: GKE
Firestore version: 3.0.7
Steps to reproduce
- Create a snapshot listener on a document
- Wait for 1 hour
- EventListener::onEvent callback is called with non-null error parameter that is FirestoreException wrapping an EOS situation