Skip to content

Fix chrome tests: textParagraphNavigation and styleNav#16701

Merged
seanbudd merged 1 commit into
nvaccess:masterfrom
mltony:fixChromeTests
Jun 17, 2024
Merged

Fix chrome tests: textParagraphNavigation and styleNav#16701
seanbudd merged 1 commit into
nvaccess:masterfrom
mltony:fixChromeTests

Conversation

@mltony

@mltony mltony commented Jun 16, 2024 •
edited by coderabbitai Bot
Loading

Copy link
Copy Markdown
Contributor

Link to issue number:

N/A

Summary of the issue:

Chrome tests textParagraphNavigation and styleNav are failing.

Description of user facing changes

N/A

Description of development approach

Minor changes to make these two tests to pass.

Testing strategy:

runsystemtests --include chrome --test "textParagraphNavigation"  2>&1 >H:\test.txt & notepad++ -n9 H:\test.txt & "C:\Program Files (x86)\NVDA\nvda.exe"
runsystemtests --include chrome --test "styleNav"  2>&1 >H:\test.txt & notepad++ -n9 H:\test.txt & "C:\Program Files (x86)\NVDA\nvda.exe"

Known issues with pull request:

Tests are flaky - not these two tests specifically, but it seems that all Chrome tests are. While testing this PR I encountered multiple times these flaky error messages:

  • Unable to locate 'before sample' marker. See NVDA log for full speech.
  • Unable to find Window in task switcher matching: re.compile('NVDA Browser Test Case .1021090117.')

Just wanted to flag this.

Code Review Checklist:

  • Documentation:
    • Change log entry
    • User Documentation
    • Developer / Technical Documentation
    • Context sensitive help for GUI changes
  • Testing:
    • Unit tests
    • System (end to end) tests
    • Manual testing
  • UX of all users considered:
    • Speech
    • Braille
    • Low Vision
    • Different web browsers
    • Localization in other languages / culture than English
  • API is compatible with existing add-ons.
  • Security precautions taken.

Summary by CodeRabbit

  • Tests
    • Updated text in test_textParagraphNavigation and test_styleNav for accuracy.
    • Removed unnecessary assertions in test_styleNav.
    • Removed [Tags] metadata from specific test cases to include them in the build process.

@mltony mltony marked this pull request as ready for review June 16, 2024 01:39
@mltony mltony requested a review from a team as a code owner June 16, 2024 01:39
@mltony mltony requested a review from michaelDCurran June 16, 2024 01:39
@coderabbitai

coderabbitai Bot commented Jun 16, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

Walkthrough

The changes involve updates to the test cases in chromeTests.py and chromeTests.robot. Specifically, string modifications and logic adjustments were made to the test_textParagraphNavigation and test_styleNav functions in chromeTests.py. Additionally, certain test case tags were removed in the chromeTests.robot file, likely to include them in the build process.

Changes

File Change Summary
tests/system/robot/chromeTests.py Modified string and assertions in test_textParagraphNavigation and test_styleNav functions.
tests/system/robot/chromeTests.robot Removed [Tags] metadata excluded_from_build from test_textParagraphNavigation and test_styleNav.
Recent review details

Configuration used: .coderabbit.yml
Review profile: CHILL

Commits

Files that changed from the base of the PR and between ad9ed73 and ab0d522.

Files selected for processing (2)
  • tests/system/robot/chromeTests.py (3 hunks)
  • tests/system/robot/chromeTests.robot (1 hunks)
Additional context used
Path-based instructions (2)
tests/system/robot/chromeTests.robot (1)

Pattern **/*: Focus on code smells, logic errors, edge cases, missing test cases, security flaws and serious issues. Avoid commenting on minor issues such as linting, formatting and style issues. This project uses tabs instead of spaces, do not suggest usage of spaces over tabs. Are there any 'red flags' in this code that might warrant closer investigation from a security standpoint? Explain what makes them suspicious.

tests/system/robot/chromeTests.py (2)

Pattern **/*: Focus on code smells, logic errors, edge cases, missing test cases, security flaws and serious issues. Avoid commenting on minor issues such as linting, formatting and style issues. This project uses tabs instead of spaces, do not suggest usage of spaces over tabs. Are there any 'red flags' in this code that might warrant closer investigation from a security standpoint? Explain what makes them suspicious.

Pattern **/*.py: _, pgettext, ngettext, and ngettext are defined globally, errors for this being undefined can be ignored.

Additional comments not posted (6)
tests/system/robot/chromeTests.robot (3)

Line range hint 1-1: Confirm that the removal of the exclamation mark in "我不会说中文！" to "我不会说中文" does not affect any dependent tests or logic.

Verification successful

The search for the new string "我不会说中文" also did not produce any output, indicating that it might not be used elsewhere in the test suite. Given that both the old and new strings are not found elsewhere, it appears that this change is isolated to the modified test and does not affect other tests or logic.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify if the change in the string affects other tests.

# Test: Search for occurrences of the old string in the test suite.
rg --type robot $'我不会说中文！'

Length of output: 105

Script:

#!/bin/bash
# Description: Verify if the change in the string affects other tests.

# Test: Search for occurrences of the new string in the test suite.
rg --type robot $'我不会说中文'

Length of output: 93

Line range hint 1-1: Confirm the implications of removing the excluded_from_build tags. Ensure that these tests are stable enough to be included in builds.

Line range hint 1-1: Ensure that the updated assertion "End of document. After Test Case Marker" correctly reflects the intended test outcome and does not introduce flakiness.

tests/system/robot/chromeTests.py (3)

2511-2511: The change in the Chinese string from "我不会说中文！" to "我不会说中文" is correctly implemented and aligns with the PR's goal to fix the test flakiness.

2579-2579: The update in the assertion text from "End of document." to "End of document. After Test Case Marker" is correctly implemented and helps in making the test outcomes more predictable and specific.

Line range hint 1-2579: The rest of the file appears stable and consistent with the expected functionalities. No unintended changes seem to have been introduced.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

coderabbitai[bot]
coderabbitai Bot reviewed Jun 16, 2024
View reviewed changes

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range and nitpick comments (1)
tests/system/robot/chromeTests.robot (1)

Line range hint 1-1: Ensure the copyright year is updated to include 2023 if changes were made this year.

@seanbudd

seanbudd commented Jun 17, 2024

Copy link
Copy Markdown
Member

Thanks @mltony

@seanbudd seanbudd merged commit 65e6620 into nvaccess:master Jun 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@seanbudd seanbudd seanbudd approved these changes

@michaelDCurran michaelDCurran Awaiting requested review from michaelDCurran michaelDCurran is a code owner automatically assigned from nvaccess/developers

+1 more reviewer

@coderabbitai coderabbitai[bot] coderabbitai[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@mltony @seanbudd