Skip to content

Integration tests: fix flaky test_dictionaries_update_and_reload::test_reload_after_fail_by_timer#67793

Merged
fm4v merged 2 commits intomasterfrom
flaky-test_reload_after_fail_by_timer
Aug 7, 2024
Merged

Integration tests: fix flaky test_dictionaries_update_and_reload::test_reload_after_fail_by_timer#67793
fm4v merged 2 commits intomasterfrom
flaky-test_reload_after_fail_by_timer

Conversation

@fm4v
Copy link
Copy Markdown
Member

@fm4v fm4v commented Aug 5, 2024

Expect both statuses ["FAILED", "FAILED_AND_RELOADING"] and ["LOADED", "LOADED_AND_RELOADING"]

According to fails for the last 90 days:
3 fails in second and third asserts:
1 2 3
2 fails in first assert:
1 2

Changelog category (leave one):

  • Not for changelog (changelog entry is not required)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

...

Documentation entry for user-facing changes

  • Documentation is written (mandatory for new features)

Information about CI checks: https://clickhouse.com/docs/en/development/continuous-integration/

CI Settings (Only check the boxes if you know what you are doing):

  • Allow: All Required Checks
  • Allow: Stateless tests
  • Allow: Stateful tests
  • Allow: Integration Tests
  • Allow: Performance tests
  • Allow: All Builds
  • Allow: batch 1, 2 for multi-batch jobs
  • Allow: batch 3, 4, 5, 6 for multi-batch jobs

  • Exclude: Style check
  • Exclude: Fast test
  • Exclude: All with ASAN
  • Exclude: All with TSAN, MSAN, UBSAN, Coverage
  • Exclude: All with aarch64, release, debug

  • Do not test
  • Woolen Wolfdog
  • Upload binaries for special builds
  • Disable merge-commit
  • Disable CI cache

@robot-clickhouse-ci-2
Copy link
Copy Markdown
Contributor

robot-clickhouse-ci-2 commented Aug 5, 2024

This is an automated comment for commit bd6c7a5 with description of existing statuses. It's updated for the latest CI running

❌ Click here to open a full report in a separate page

Check nameDescriptionStatus
Flaky testsChecks if new added or modified tests are flaky by running them repeatedly, in parallel, with more randomization. Functional tests are run 100 times with address sanitizer, and additional randomization of thread scheduling. Integration tests are run up to 10 times. If at least once a new test has failed, or was too long, this check will be red. We don't allow flaky tests, read the doc❌ failure
Successful checks
Check nameDescriptionStatus
Integration testsThe integration tests report. In parenthesis the package type is given, and in square brackets are the optional part/total tests✅ success
Style checkRuns a set of checks to keep the code style clean. If some of tests failed, see the related log from the report✅ success

Copy link
Copy Markdown
Member

@azat azat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P.S. description is outdated

@azat
Copy link
Copy Markdown
Member

azat commented Aug 5, 2024

@fm4v also the test is not idempotent and should be rewritten completely otherwise the flaky check will fail - https://s3.amazonaws.com/clickhouse-test-reports/67754/e6921bfc595937ac6d2d448ac46cf2778119d509/integration_tests_flaky_check__asan_.html

The best way to do this is to use DDL for creating dictionaries instead of XML

@alexey-milovidov alexey-milovidov added the 🍃 green ci 🌿 Fixing flaky tests in CI label Aug 5, 2024
@fm4v fm4v added this pull request to the merge queue Aug 7, 2024
Merged via the queue into master with commit 755b06b Aug 7, 2024
@fm4v fm4v deleted the flaky-test_reload_after_fail_by_timer branch August 7, 2024 21:51
@robot-ch-test-poll3 robot-ch-test-poll3 added the pr-synced-to-cloud The PR is synced to the cloud repo label Aug 7, 2024
@azat
Copy link
Copy Markdown
Member

azat commented Aug 8, 2024

@fm4v it has been merged with flaky check failed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🍃 green ci 🌿 Fixing flaky tests in CI pr-not-for-changelog This PR should not be mentioned in the changelog pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants