Skip to content

Flaky-test: AdminApi2Test.testDeleteNamespace #17111

@codelipenghui

Description

@codelipenghui

Search before asking

  • I searched in the issues and found nothing similar.

Example failure

https://github.com/apache/pulsar/runs/7849998786?check_suite_focus=true

Exception stacktrace

Error:  Tests run: 155, Failures: 1, Errors: 0, Skipped: 138, Time elapsed: 156.122 s <<< FAILURE! - in org.apache.pulsar.broker.admin.AdminApi2Test
  Error:  testDeleteNamespace(org.apache.pulsar.broker.admin.AdminApi2Test)  Time elapsed: 0.321 s  <<< FAILURE!
  org.apache.pulsar.client.admin.PulsarAdminException$ServerSideErrorException: 
  
   --- An unexpected error occurred in the server ---
  
  Message: java.util.concurrent.CompletionException: org.apache.pulsar.metadata.api.MetadataStoreException$NotFoundException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /managed-ledgers/test-tenant/test-ns/persistent/__change_events/__compaction
  
  Stacktrace:
  
  org.apache.bookkeeper.mledger.ManagedLedgerException$MetaStoreException: java.util.concurrent.CompletionException: org.apache.pulsar.metadata.api.MetadataStoreException$NotFoundException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /managed-ledgers/test-tenant/test-ns/persistent/__change_events/__compaction
  Caused by: java.util.concurrent.CompletionException: org.apache.pulsar.metadata.api.MetadataStoreException$NotFoundException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /managed-ledgers/test-tenant/test-ns/persistent/__change_events/__compaction
  	at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:332)
  	at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:347)
  	at java.base/java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:781)
  	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
  	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162)
  	at org.apache.pulsar.metadata.impl.ZKMetadataStore.handleDeleteResult(ZKMetadataStore.java:274)
  	at org.apache.pulsar.metadata.impl.ZKMetadataStore.lambda$batchOperation$6(ZKMetadataStore.java:186)
  	at org.apache.zookeeper.MockZooKeeper.multi(MockZooKeeper.java:1009)
  	at org.apache.pulsar.metadata.impl.ZKMetadataStore.batchOperation(ZKMetadataStore.java:160)
  	at org.apache.pulsar.metadata.impl.batching.AbstractBatchedMetadataStore.flush(AbstractBatchedMetadataStore.java:117)
  	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
  	at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
  	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
  	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
  	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
  	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
  	at java.base/java.lang.Thread.run(Thread.java:833)
  Caused by: org.apache.pulsar.metadata.api.MetadataStoreException$NotFoundException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /managed-ledgers/test-tenant/test-ns/persistent/__change_events/__compaction
  	at org.apache.pulsar.metadata.impl.ZKMetadataStore.getException(ZKMetadataStore.java:440)
  	... 12 more
  Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /managed-ledgers/test-tenant/test-ns/persistent/__change_events/__compaction
  	at org.apache.zookeeper.KeeperException.create(KeeperException.java:118)
  	at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
  	at org.apache.pulsar.metadata.impl.ZKMetadataStore.getException(ZKMetadataStore.java:434)
  	... 12 more
  
  	at org.apache.pulsar.client.admin.PulsarAdminException.wrap(PulsarAdminException.java:252)
  	at org.apache.pulsar.client.admin.internal.BaseResource.sync(BaseResource.java:316)
  	at org.apache.pulsar.client.admin.internal.NamespacesImpl.deleteNamespace(NamespacesImpl.java:230)
  	at org.apache.pulsar.broker.admin.AdminApi2Test.testDeleteNamespace(AdminApi2Test.java:1437)
  	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
  	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
  	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
  	at org.testng.internal.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:45)
  	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:73)
  	at org.testng.internal.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
  	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
  	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
  	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
  	at java.base/java.lang.Thread.run(Thread.java:833)
  	Suppressed: org.apache.pulsar.client.admin.PulsarAdminException$ServerSideErrorException: 
   --- An unexpected error occurred in the server ---
  
  Message: java.util.concurrent.CompletionException: org.apache.pulsar.metadata.api.MetadataStoreException$NotFoundException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /managed-ledgers/test-tenant/test-ns/persistent/__change_events/__compaction
  
  Stacktrace:
  
  org.apache.bookkeeper.mledger.ManagedLedgerException$MetaStoreException: java.util.concurrent.CompletionException: org.apache.pulsar.metadata.api.MetadataStoreException$NotFoundException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /managed-ledgers/test-tenant/test-ns/persistent/__change_events/__compaction
  Caused by: java.util.concurrent.CompletionException: org.apache.pulsar.metadata.api.MetadataStoreException$NotFoundException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /managed-ledgers/test-tenant/test-ns/persistent/__change_events/__compaction
  	at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:332)
  	at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:347)
  	at java.base/java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:781)
  	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
  	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162)
  	at org.apache.pulsar.metadata.impl.ZKMetadataStore.handleDeleteResult(ZKMetadataStore.java:274)
  	at org.apache.pulsar.metadata.impl.ZKMetadataStore.lambda$batchOperation$6(ZKMetadataStore.java:186)
  	at org.apache.zookeeper.MockZooKeeper.multi(MockZooKeeper.java:1009)
  	at org.apache.pulsar.metadata.impl.ZKMetadataStore.batchOperation(ZKMetadataStore.java:160)
  	at org.apache.pulsar.metadata.impl.batching.AbstractBatchedMetadataStore.flush(AbstractBatchedMetadataStore.java:117)
  	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
  	at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
  	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
  	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
  	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
  	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
  	at java.base/java.lang.Thread.run(Thread.java:833)
  Caused by: org.apache.pulsar.metadata.api.MetadataStoreException$NotFoundException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /managed-ledgers/test-tenant/test-ns/persistent/__change_events/__compaction
  	at org.apache.pulsar.metadata.impl.ZKMetadataStore.getException(ZKMetadataStore.java:440)
  	... 12 more
  Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /managed-ledgers/test-tenant/test-ns/persistent/__change_events/__compaction
  	at org.apache.zookeeper.KeeperException.create(KeeperException.java:118)
  	at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
  	at org.apache.pulsar.metadata.impl.ZKMetadataStore.getException(ZKMetadataStore.java:434)
  	... 12 more
  
  		at org.apache.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:237)
  		at org.apache.pulsar.client.admin.internal.BaseResource$3.failed(BaseResource.java:202)
  		at org.glassfish.jersey.client.JerseyInvocation$1.failed(JerseyInvocation.java:882)
  		at org.glassfish.jersey.client.JerseyInvocation$1.completed(JerseyInvocation.java:863)
  		at org.glassfish.jersey.client.ClientRuntime.processResponse(ClientRuntime.java:229)
  		at org.glassfish.jersey.client.ClientRuntime.access$200(ClientRuntime.java:62)
  		at org.glassfish.jersey.client.ClientRuntime$2.lambda$response$0(ClientRuntime.java:173)
  		at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
  		at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
  		at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
  		at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
  		at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
  		at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:288)
  		at org.glassfish.jersey.client.ClientRuntime$2.response(ClientRuntime.java:173)
  		at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$apply$1(AsyncHttpConnector.java:246)
  		at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
  		at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
  		at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
  		at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147)
  		at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$retryOperation$4(AsyncHttpConnector.java:288)
  		at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
  		at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
  		at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
  		at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147)
  		at org.asynchttpclient.netty.NettyResponseFuture.loadContent(NettyResponseFuture.java:222)
  		at org.asynchttpclient.netty.NettyResponseFuture.done(NettyResponseFuture.java:257)
  		at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.finishUpdate(AsyncHttpClientHandler.java:241)
  		at org.asynchttpclient.netty.handler.HttpHandler.handleChunk(HttpHandler.java:114)
  		at org.asynchttpclient.netty.handler.HttpHandler.handleRead(HttpHandler.java:143)
  		at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:78)
  		at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
  		at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
  		at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
  		at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
  		at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
  		at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
  		at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
  		at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
  		at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:327)
  		at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:299)
  		at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
  		at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
  		at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
  		at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
  		at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:[1410](https://github.com/apache/pulsar/runs/7849998786?check_suite_focus=true#step:10:1411))
  		at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
  		at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
  		at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
  		at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
  		at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)
  		at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)
  		at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)
  		at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
  		at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:995)
  		at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
  		at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
  		... 1 more
  	Caused by: javax.ws.rs.InternalServerErrorException: HTTP 500 ? --- An unexpected error occurred in the server ---??Message: java.util.concurrent.CompletionException: org.apache.pulsar.metadata.api.MetadataStoreException$NotFoundException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /managed-ledgers/test-tenant/test-ns/persistent/__change_events/__compaction??Stacktrace:??org.apache.bookkeeper.mledger.ManagedLedgerException$MetaStoreException: java.util.concurrent.CompletionException: org.apache.pulsar.metadata.api.MetadataStoreException$NotFoundException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /managed-ledgers/test-tenant/test-ns/persistent/__change_events/__compaction?Caused by: java.util.concurrent.CompletionException: org.apache.pulsar.metadata.api.MetadataStoreException$NotFoundException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /managed-ledgers/test-tenant/test-ns/persistent/__change_events/__compaction?	at java.base/java.util.concurrent.Com
  		at org.glassfish.jersey.client.JerseyInvocation.convertToException(JerseyInvocation.java:960)
  		at org.glassfish.jersey.client.JerseyInvocation.access$700(JerseyInvocation.java:82)
  		... 54 more
  Caused by: [CIRCULAR REFERENCE: javax.ws.rs.InternalServerErrorException: HTTP 500 ? --- An unexpected error occurred in the server ---??Message: java.util.concurrent.CompletionException: org.apache.pulsar.metadata.api.MetadataStoreException$NotFoundException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /managed-ledgers/test-tenant/test-ns/persistent/__change_events/__compaction??Stacktrace:??org.apache.bookkeeper.mledger.ManagedLedgerException$MetaStoreException: java.util.concurrent.CompletionException: org.apache.pulsar.metadata.api.MetadataStoreException$NotFoundException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /managed-ledgers/test-tenant/test-ns/persistent/__change_events/__compaction?Caused by: java.util.concurrent.CompletionException: org.apache.pulsar.metadata.api.MetadataStoreException$NotFoundException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /managed-ledgers/test-tenant/test-ns/persistent/__change_events/__compaction?	at java.base/java.util.concurrent.Com]

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions