Skip to content

KAFKA-12284: increase request timeout to make tests reliable#10547

Merged
chia7712 merged 4 commits into
apache:trunkfrom
showuon:KAFKA-12284
Apr 28, 2021
Merged

KAFKA-12284: increase request timeout to make tests reliable#10547
chia7712 merged 4 commits into
apache:trunkfrom
showuon:KAFKA-12284

Conversation

@showuon

@showuon showuon commented Apr 16, 2021

Copy link
Copy Markdown
Member

The MirrorConnectorsIntegrationTests recently failed with

TimeoutException: The request timed out.

quite frequently. After investigation, the reason is the createTopic in the server doesn't complete within request.timeout.ms (default to 30 seconds). It makes sense that the server is pretty slow during 2 connector cluster (primary and backup) started at the same time. What I did are:

  1. increase the timeout value to 60 seconds to make the tests reliable.
  2. wait for internal topics created after connector cluster started, and then try to create our test topics.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@showuon

showuon commented Apr 16, 2021

Copy link
Copy Markdown
Member Author

@mimaison @ning2008wisc , I've updated the PR, please help review it. Thank you.

@showuon showuon mentioned this pull request Apr 16, 2021
Closed
3 tasks
@showuon

showuon commented Apr 16, 2021

Copy link
Copy Markdown
Member Author

Failed tests are all un-related. (Not in MirrorMakerIntegrationTest)

)

    Build / JDK 8 and Scala 2.12 / kafka.server.RaftClusterTest.testCreateClusterAndCreateAndManyTopicsWithManyPartitions()
    Build / JDK 15 and Scala 2.13 / org.apache.kafka.connect.integration.ConnectorClientPolicyIntegrationTest.testCreateWithNotAllowedOverridesForPrincipalPolicy
    Build / JDK 15 and Scala 2.13 / kafka.admin.DeleteOffsetsConsumerGroupCommandIntegrationTest.testDeleteOffsetsNonExistingGroup()
    Build / JDK 15 and Scala 2.13 / kafka.admin.ResetConsumerGroupOffsetTest.testResetOffsetsByDurationToEarliest()
    Build / JDK 15 and Scala 2.13 / kafka.admin.ResetConsumerGroupOffsetTest.testResetOffsetsExistingTopicSelectedGroups()
    Build / JDK 15 and Scala 2.13 / kafka.admin.ResetConsumerGroupOffsetTest.testResetOffsetsToZonedDateTime()
    Build / JDK 15 and Scala 2.13 / kafka.server.RaftClusterTest.testCreateClusterAndCreateAndManyTopicsWithManyPartitions()
    Build / JDK 15 and Scala 2.13 / org.apache.kafka.streams.integration.KTableKTableForeignKeyInnerJoinMultiIntegrationTest.shouldInnerJoinMultiPartitionQueryable

@ning2008wisc

ning2008wisc commented Apr 19, 2021

Copy link
Copy Markdown
Contributor

thanks for the PR :) change seems reasonable and lgtm (if consistent test results can be verified by multiple runs). The final review will be done by the committer.

@showuon showuon mentioned this pull request Apr 20, 2021
Merged
3 tasks
@showuon

showuon commented Apr 20, 2021

Copy link
Copy Markdown
Member Author

Thanks @ning2008wisc !
@mimaison , see if you have any comments. Thanks.

@chia7712

chia7712 commented Apr 22, 2021

Copy link
Copy Markdown
Member

@showuon Could you merge trunk to trigger QA again?

@showuon

showuon commented Apr 22, 2021

Copy link
Copy Markdown
Member Author

Done. Let's wait and see :)

@showuon

showuon commented Apr 22, 2021

Copy link
Copy Markdown
Member Author

Failed tests are

    Build / JDK 8 and Scala 2.12 / org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationSSLTest.testReplication()
    Build / JDK 8 and Scala 2.12 / kafka.server.RaftClusterTest.testCreateClusterAndCreateAndManyTopicsWithManyPartitions()
    Build / JDK 15 and Scala 2.13 / org.apache.kafka.connect.integration.RebalanceSourceConnectorsIntegrationTest.testDeleteConnector
    Build / JDK 15 and Scala 2.13 / org.apache.kafka.streams.integration.KTableKTableForeignKeyInnerJoinMultiIntegrationTest.shouldInnerJoinMultiPartitionQueryable

The failed MirrorConnectorsIntegrationSSLTest.testReplication() is not request timeout anymore.

org.opentest4j.AssertionFailedError: Condition not met within timeout 20000. Offsets not translated downstream to primary cluster. ==> expected: <true> but was: <false>

chia7712
chia7712 approved these changes Apr 27, 2021
View reviewed changes

@chia7712 chia7712 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@showuon thanks for this patch. LGTM. one small comment below.

Comment thread connect/runtime/src/test/java/org/apache/kafka/connect/util/clusters/EmbeddedKafkaCluster.java Outdated
@showuon

showuon commented Apr 28, 2021

Copy link
Copy Markdown
Member Author

Failed tests are all un-related. (Not in MirrorMakerIntegrationTest)

    Build / JDK 11 and Scala 2.13 / kafka.network.ConnectionQuotasTest.testListenerConnectionRateLimitWhenActualRateAboveLimit()
    Build / JDK 11 and Scala 2.13 / kafka.server.RaftClusterTest.testCreateClusterAndCreateAndManyTopics()
    Build / JDK 8 and Scala 2.12 / org.apache.kafka.streams.integration.StoreQueryIntegrationTest.shouldQueryOnlyActivePartitionStoresByDefault
    Build / JDK 15 and Scala 2.13 / kafka.api.TransactionsTest.testCommitTransactionTimeout()
    Build / JDK 15 and Scala 2.13 / org.apache.kafka.streams.integration.KTableKTableForeignKeyInnerJoinMultiIntegrationTest.shouldInnerJoinMultiPartitionQueryable

@chia7712 chia7712 merged commit ab06aef into apache:trunk Apr 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chia7712 chia7712 chia7712 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@showuon @ning2008wisc @chia7712