Skip to content

[Test][Autoscaler] deflaky autoscaler idle timeout e2e tests by a longer timeout#3727

Merged
kevin85421 merged 1 commit intoray-project:masterfrom
rueian:e2eautoscaler-deflaky-idle-timeout
Jun 1, 2025
Merged

[Test][Autoscaler] deflaky autoscaler idle timeout e2e tests by a longer timeout#3727
kevin85421 merged 1 commit intoray-project:masterfrom
rueian:e2eautoscaler-deflaky-idle-timeout

Conversation

@rueian
Copy link
Copy Markdown
Collaborator

@rueian rueian commented Jun 1, 2025 •
edited
Loading

Give the test case more time to scale down.

The original flaky rate of this test is about 10% on my mac. Now it can pass 100 times without flaky.

▶ while go test -count=1 ./... -run=TestRayClusterAutoscalerV2IdleTimeout; do :; done

ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	96.222s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.784s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.683s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.702s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.944s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.898s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.874s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	95.137s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.967s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.632s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.951s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.076s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.656s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.324s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.028s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.483s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.926s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.547s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.739s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.546s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.204s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.733s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.741s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.087s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.840s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	95.212s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.311s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.097s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.837s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.502s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.673s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.628s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.149s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.693s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.718s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.852s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	100.106s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	99.967s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.824s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.861s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.826s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	99.891s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	100.224s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	99.747s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.722s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.875s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.979s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.820s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.729s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.135s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.008s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.684s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.573s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.000s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.948s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.640s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.807s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.770s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.443s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.611s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.000s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.444s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.981s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.819s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.534s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.819s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.978s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.461s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.839s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.908s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.694s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.981s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.709s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.628s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.826s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.665s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.121s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.974s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.413s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.389s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.958s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.449s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.588s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.958s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.440s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	99.825s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.965s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.219s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.679s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.074s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.842s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.743s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.833s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.164s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	99.851s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	99.330s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	94.670s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.857s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	100.949s
ok  	github.com/ray-project/kuberay/ray-operator/test/e2eautoscaler	93.538s

Why are these changes needed?

Related issue number

Closes #3701

Checks

  • I've made sure the tests are passing.
  • Testing Strategy
    • Unit tests
    • Manual tests
    • This PR is not tested :(

@rueian
[Test][Autoscaler] deflaky autoscaler idle timeout e2e tests by a lon…
c03fdd7 
…ger timeout

Signed-off-by: Rueian <rueiancsie@gmail.com>
@rueian rueian marked this pull request as ready for review June 1, 2025 19:15
@rueian rueian mentioned this pull request Jun 1, 2025
Closed
2 tasks
@kevin85421 kevin85421 merged commit 764971f into ray-project:master Jun 1, 2025
25 checks passed
@davidxia davidxia mentioned this pull request Jun 5, 2025
Closed
4 tasks
pawelpaszki pushed a commit to opendatahub-io/kuberay that referenced this pull request Jun 10, 2025
@rueian @pawelpaszki
[Test][Autoscaler] deflaky autoscaler idle timeout e2e tests by a lon…
2d0823c 
…ger timeout (ray-project#3727)

Signed-off-by: Rueian <rueiancsie@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kevin85421 kevin85421 kevin85421 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[CI] Deflaky Autoscaler V2 e2e tests

2 participants

@rueian @kevin85421