fix: fix failing Slack e2e tests#5295
Conversation
✅ Deploy Preview for asyncapi-website ready!Built without sensitive environment variables
To edit notification comments on pull requests, go to your Netlify project configuration. |
📝 WalkthroughWalkthroughA single line was added to the Cypress test setup in Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~2 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #5295 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 22 22
Lines 830 830
Branches 159 159
=========================================
Hits 830 830 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
⚡️ Lighthouse report for the changes in this PR:
Lighthouse ran on https://deploy-preview-5295--asyncapi-website.netlify.app/ |
There was a problem hiding this comment.
🧹 Nitpick comments (3)
cypress/slackworkspace.cy.js (2)
22-24: Test name mismatch with actual assertion.Similar to above—the test claims to verify "links for Privacy, Contact Us, and Region Change" but
verifyAllFooterLinks()only checks the URL containsslack.com. Consider updating the test name or adding a comment documenting this intentional simplification.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@cypress/slackworkspace.cy.js` around lines 22 - 24, The test title "Should show links for Privacy, Contact Us, and Region Change" is misleading because verifyAllFooterLinks() only asserts the URL contains "slack.com"; update the test to either rename the it() description to reflect the actual check (e.g., "Should verify footer links point to slack.com") or modify verifyAllFooterLinks() to perform the fuller assertions for Privacy, Contact Us, and Region Change; alternatively add a concise inline comment next to the call to verifyAllFooterLinks() documenting that this is an intentional simplified URL-only check.
11-19: Test name may not reflect actual verification.The test is titled "Should show all login methods when the Slack invite link is active", but
verifyAllLoginMethods()in the page object now only assertscy.url().should('include', 'slack.com'). Consider renaming to better reflect what's actually verified, or add a code comment explaining the trade-off between test stability and verification depth.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@cypress/slackworkspace.cy.js` around lines 11 - 19, The test title claims it verifies "all login methods" but the page object method verifyAllLoginMethods only asserts the URL contains 'slack.com', so update the test to accurately reflect what's being checked: either rename the it(...) description to something like "navigates to Slack when invite link is active" or expand verifyAllLoginMethods to include the additional assertions you want; alternatively add a clear inline comment next to verifyAllLoginMethods (and mention checkLinkStatus/verifyInactiveLinkMessage) explaining that the implementation intentionally only asserts the redirect (cy.url().should('include', 'slack.com')) for stability. Ensure the chosen change updates the test name or the method so behavior and description match.cypress/pages/slack.js (1)
25-31: Methods are identical and don't verify what their names suggest.Both
verifyAllLoginMethods()andverifyAllFooterLinks()have identical implementations that only checkcy.url().should('include', 'slack.com'). This doesn't actually verify login methods or footer links exist.I understand this is a pragmatic trade-off to fix flaky tests caused by Slack's changing DOM. However:
- The duplicate code should be consolidated or the methods should be honest about what they verify.
- Consider adding a comment explaining the intentional simplification.
Option 1: Consolidate into a single honest method
- verifyAllLoginMethods() { - cy.url().should('include', 'slack.com'); - } - - verifyAllFooterLinks() { - cy.url().should('include', 'slack.com'); - } + // Verifies we're on a Slack page. Granular UI element checks were removed + // because Slack's DOM structure changes frequently, causing test flakiness. + verifyOnSlackPage() { + cy.url().should('include', 'slack.com'); + }Option 2: Keep separate methods with documentation
verifyAllLoginMethods() { + // Note: Simplified to URL check only. Granular login button assertions + // (Google, Apple, email) were removed due to Slack DOM instability. cy.url().should('include', 'slack.com'); } verifyAllFooterLinks() { + // Note: Simplified to URL check only. Granular footer link assertions + // (Privacy, Contact Us, Region Change) were removed due to Slack DOM instability. cy.url().should('include', 'slack.com'); }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@cypress/pages/slack.js` around lines 25 - 31, The two methods verifyAllLoginMethods() and verifyAllFooterLinks() are identical and only assert the URL includes 'slack.com', which is misleading; either consolidate them into a single honest helper (e.g., verifySlackRedirect()) and have both callers use that, or keep both method names but update their bodies to call a shared helper and add an explanatory comment stating the intentional simplification due to Slack's flaky DOM; update or add tests/call sites to use the new helper or the commented methods so intent is clear.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@cypress/pages/slack.js`:
- Around line 25-31: The two methods verifyAllLoginMethods() and
verifyAllFooterLinks() are identical and only assert the URL includes
'slack.com', which is misleading; either consolidate them into a single honest
helper (e.g., verifySlackRedirect()) and have both callers use that, or keep
both method names but update their bodies to call a shared helper and add an
explanatory comment stating the intentional simplification due to Slack's flaky
DOM; update or add tests/call sites to use the new helper or the commented
methods so intent is clear.
In `@cypress/slackworkspace.cy.js`:
- Around line 22-24: The test title "Should show links for Privacy, Contact Us,
and Region Change" is misleading because verifyAllFooterLinks() only asserts the
URL contains "slack.com"; update the test to either rename the it() description
to reflect the actual check (e.g., "Should verify footer links point to
slack.com") or modify verifyAllFooterLinks() to perform the fuller assertions
for Privacy, Contact Us, and Region Change; alternatively add a concise inline
comment next to the call to verifyAllFooterLinks() documenting that this is an
intentional simplified URL-only check.
- Around line 11-19: The test title claims it verifies "all login methods" but
the page object method verifyAllLoginMethods only asserts the URL contains
'slack.com', so update the test to accurately reflect what's being checked:
either rename the it(...) description to something like "navigates to Slack when
invite link is active" or expand verifyAllLoginMethods to include the additional
assertions you want; alternatively add a clear inline comment next to
verifyAllLoginMethods (and mention checkLinkStatus/verifyInactiveLinkMessage)
explaining that the implementation intentionally only asserts the redirect
(cy.url().should('include', 'slack.com')) for stability. Ensure the chosen
change updates the test name or the method so behavior and description match.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: dcd33521-34c5-44f9-a9c0-8d5fa05ccaba
📒 Files selected for processing (2)
cypress/pages/slack.jscypress/slackworkspace.cy.js
|
|
||
| beforeEach(() => { | ||
| slackPage.visitSlack(); | ||
| slackPage.waitForPageLoad(); |
There was a problem hiding this comment.
The absence of this line is the valid reason for failing as far as I've tested. I maybe wrong.
@wei123-web Please try this:
- Revert to old code
- Add this line and run the test
- Tell us what you observe
There was a problem hiding this comment.
@animeshk923 I tested your suggestion locally you were right!
Adding only waitForPageLoad() to the old code also fixes the failing test:
- Should show all login methods when the Slack invite link is active
- Should show links for Privacy, Contact Us, and Region Change
2 passing, 0 failing
However, I kept my current approach (simplified methods) because: - The Slack invite link is expired and may never show login buttons
- Old methods like
verifyGoogleLoginButton()will fail again
when Slack changes their UI - Simplified URL checks are more stable long term
Please let me know which approach you prefer and I'll update accordingly!
There was a problem hiding this comment.
@wei123-web if that is the case, then can you please update the PR.
@animeshk923 thanks for your effort!
There was a problem hiding this comment.
@wei123-web you don't need to remove the tests from the codebase.
- Restore the previously present code.
- Add that one line mentioned above
- Push changes
That's it!
PS: update the PR body message too after following the above steps...
There was a problem hiding this comment.
hey @animeshk923 @princerajpoot20
I ran both approaches today and here are the results:
Minimal fix (only adding waitForPageLoad()):
Still failing - same error as original CI failure
Yesterday minimal fix → 2 passing
Today same code → 1 failing
There was a problem hiding this comment.
@animeshk923 @princerajpoot20
Updated the PR with minimal fix only:
- Restored original code
- Added only
waitForPageLoad()inbeforeEach - Both tests passing locally
Sorry for the confusion earlier!
and thanks @animeshk923
There was a problem hiding this comment.
@wei123-web great work! 👏
ps: don't forget to update the pr description...
There was a problem hiding this comment.
thanks @animeshk923 for your help and suggestions!
There was a problem hiding this comment.
thanks @animeshk923 for your help and suggestions!
@wei123-web anytime!
3a84db2 to
c4cfda4
Compare
c4cfda4 to
ece5ffd
Compare
|



Description
waitForPageLoad()inbeforeEachto ensure the pageis fully loaded before running tests
ready when assertions ran
Related issue(s)
Fixes #5281
Summary by CodeRabbit