Emoji panel and clipboard history system tests by seanbudd · Pull Request #12487 · nvaccess/nvda

seanbudd · 2021-05-31T08:37:31Z

Link to issue number:

None

Summary of the issue:

To aid reviewing future work on the emoji panel and clipboard history and #11485, system tests for these Windows features are needed.

Description of how this pull request fixes the issue:

Introduces system tests which (covers most of the manual testing in #11485):

Opening clipboard history and navigating it
Opening emoji panel, ensure the first item read is an emoji. (Note: excluded, see known issues)
Toggling between emoji panel and clipboard history without closing them. (Note: excluded, see known issues)
Searching for emoji items, navigating them. (Note: excluded, see known issues)

Testing strategy:

See PR builds. Run system tests locally as well as windows versions implement these features differently.

Known issues with pull request:

As discussed on #11485 (comment), Windows versions implement these features differently and Microsoft plans to update them further. As a result, the AppVeyor tests may need to be updated or excluded if they start to fail and work on fixing the emoji panel or clipboard history can be prioritised. Additionally, these tests ideally should be run on various Windows versions to ensure nothing has broken across implementations.

For example: searching for emojis is not implemented on the AppVeyor implementation of the emoji panel. As such, this test fails and needs to be excluded.
https://ci.appveyor.com/project/NVAccess/nvda/builds/39392852/tests

Emoji related tests need to be excluded until #11485, as the emoji panel does not always announce an emoji when opening the
emoji panel. This can be observed with this build, where 2 tests involve opening the emoji panel and expecting an emoji announced but only one passes. https://ci.appveyor.com/project/NVAccess/nvda/builds/39404151/tests

Change log entries:

None needed

Code Review Checklist:

Pull Request description is up to date.
Unit tests.
System (end to end) tests.
Manual testing.
User Documentation.
Change log entry.
Context sensitive help for GUI changes.
UX of all users considered:
- Speech
- Braille
- Low Vision
- Different web browsers

josephsl · 2021-05-31T08:47:04Z

CC @michaelDCurran as it could open up a possibility to test IME candidates support in the future (IME candidates also covers hardware keyboard input suggestions, but not sure how to get that going in the future).

Thanks.

AppVeyorBot · 2021-05-31T08:52:43Z

PASS: Translation comments check.
PASS: Unit tests.
PASS: Lint check.
FAIL: System tests. See test results for more information.
Build (for testing PR)

See test results for failed build of commit 82c07fa303

AppVeyorBot · 2021-06-01T03:01:35Z

PASS: Translation comments check.
PASS: Unit tests.
PASS: Lint check.
FAIL: System tests. See test results for more information.
Build (for testing PR)

See test results for failed build of commit 011f36421f

LeonarddeR · 2021-06-01T05:54:05Z

Could we ensure that these tests never run on older versions, i.e. Windows 8.1 and older? This doesn't matter for appveyor, but for manual testing, exclusion would be appreciated.

josephsl · 2021-06-01T05:57:10Z

Hi, I think perhaps a condition flag should be specified in Robot somehow that will exclude certain tests from being performed if winVersion.getWinVer() </>= winVersion.WIN*. For Windows Insider Preview, we need to rely on winVersion.getWinVer().build as we won’t know what the final build number will be until it ships to beta channel. Thanks.

seanbudd · 2021-06-03T00:36:13Z

Could we ensure that these tests never run on older versions, i.e. Windows 8.1 and older? This doesn't matter for appveyor, but for manual testing, exclusion would be appreciated.

Would it be troublesome to just include --exclude windowsTests if you wish to ignore these? Having expected failures locally shouldn't really be an issue if it is clear why they are failing. I think ideally we should make a separate robot args files for appveyor and various developer environments. As it stands now, system tests are used mostly by appveyor. Locally, I tend to only run tests relevant to the work I'm doing.

seanbudd · 2021-06-03T00:44:35Z

I've added some more explicit tags so that you can --exclude emojiPanel and --exclude clipboard instead if preferred.

AppVeyorBot · 2021-06-03T01:00:49Z

PASS: Translation comments check.
PASS: Unit tests.
PASS: Lint check.
FAIL: System tests. See test results for more information.
Build (for testing PR)

See test results for failed build of commit 7a878dfa7d

seanbudd · 2021-06-14T09:07:19Z

I want to change this to use our notepad system tests when that is merged in on #11856

AppVeyorBot · 2021-07-07T07:03:04Z

PASS: Translation comments check.
PASS: Unit tests.
PASS: Lint check.
FAIL: System tests. See test results for more information.
Build (for testing PR): https://ci.appveyor.com/api/buildjobs/pjpen15h5031f31e/artifacts/output/nvda_snapshot_pr12487-23310,dba58804.exe

See test results for failed build of commit dba58804c7

AppVeyorBot · 2021-07-07T08:59:44Z

PASS: Translation comments check.
PASS: Unit tests.
PASS: Lint check.
FAIL: System tests. See test results for more information.
Build (for testing PR): https://ci.appveyor.com/api/buildjobs/dwyxktrx0pv1i283/artifacts/output/nvda_snapshot_pr12487-23314,1097d6ed.exe

See test results for failed build of commit 1097d6edff

AppVeyorBot · 2021-07-08T01:05:38Z

PASS: Translation comments check.
PASS: Unit tests.
PASS: Lint check.
FAIL: System tests. See test results for more information.
Build (for testing PR): https://ci.appveyor.com/api/buildjobs/s8j7r7rj8y0ss0x4/artifacts/output/nvda_snapshot_pr12487-23317,ec35121e.exe

See test results for failed build of commit ec35121eda

AppVeyorBot · 2021-08-26T03:32:19Z

PASS: Translation comments check.
PASS: Unit tests.
PASS: Lint check.
FAIL: System tests. See test results for more information.
Build (for testing PR): https://ci.appveyor.com/api/buildjobs/k4jn27bj4odl4m9c/artifacts/output/nvda_snapshot_pr12487-23650,b7c1fa0a.exe

See test results for failed build of commit b7c1fa0af4

AppVeyorBot · 2021-08-26T03:52:41Z

PASS: Translation comments check.
PASS: Unit tests.
PASS: Lint check.
FAIL: System tests. See test results for more information.
Build (for testing PR): https://ci.appveyor.com/api/buildjobs/o5mktg70loelfvkh/artifacts/output/nvda_snapshot_pr12487-23651,acf63d4b.exe

See test results for failed build of commit acf63d4b71

AppVeyorBot · 2021-09-09T07:01:02Z

PASS: Translation comments check.
PASS: Unit tests.
PASS: Lint check.
FAIL: System tests. See test results for more information.
Build (for testing PR): https://ci.appveyor.com/api/buildjobs/q1mqyqf52pkjyosa/artifacts/output/nvda_snapshot_pr12487-23714,21685cfe.exe

See test results for failed build of commit 21685cfecd

seanbudd added 3 commits May 31, 2021 17:16

emoji panel and clipboard system tests

784f085

fix teardown

07b943e

exclude panel search from appveyor build

d93967b

seanbudd requested a review from a team as a code owner May 31, 2021 08:37

seanbudd requested a review from michaelDCurran May 31, 2021 08:37

seanbudd changed the title ~~Emoji panel system tests~~ Emoji panel and clipboard history system tests May 31, 2021

seanbudd marked this pull request as draft May 31, 2021 08:57

fix behaviour to be more expected

c5c353b

exclude emoji test from build

7dd9fa6

seanbudd marked this pull request as ready for review June 1, 2021 03:22

josephsl mentioned this pull request Jun 1, 2021

Emoji panel and clipboard history: locate actual items when traversing the elements for element selected event #11485

Merged

more explicit tags for exclusion

faf01ff

seanbudd force-pushed the emojiPanelSystemTests branch from d3ad0a3 to faf01ff Compare June 3, 2021 00:45

use run dialog instead of search bar

af7b38e

seanbudd marked this pull request as draft June 14, 2021 09:06

seanbudd added 3 commits July 7, 2021 16:11

Merge branch 'master' into emojiPanelSystemTests

f754239

use notepad for tests

014ab6f

fix timeout syntax

a7d3f31

make better use of notepad lib

9aec4b6

seanbudd force-pushed the emojiPanelSystemTests branch from 373848a to 9aec4b6 Compare July 7, 2021 07:53

seanbudd added 2 commits July 7, 2021 18:33

use a regex match for emojis

3ec5c53

fix typos, expand regex

83fe261

improve clipboard history focus

00758f3

seanbudd added 2 commits August 26, 2021 13:34

Merge branch 'master' into emojiPanelSystemTests

42b7481

update command ref

da7997f

seanbudd closed this Sep 13, 2021

Uh oh!

Conversation

seanbudd commented May 31, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Link to issue number:

Summary of the issue:

Description of how this pull request fixes the issue:

Testing strategy:

Known issues with pull request:

Change log entries:

Code Review Checklist:

Uh oh!

josephsl commented May 31, 2021

Uh oh!

AppVeyorBot commented May 31, 2021

Uh oh!

AppVeyorBot commented Jun 1, 2021

Uh oh!

LeonarddeR commented Jun 1, 2021

Uh oh!

josephsl commented Jun 1, 2021 via email

Uh oh!

seanbudd commented Jun 3, 2021

Uh oh!

seanbudd commented Jun 3, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AppVeyorBot commented Jun 3, 2021

Uh oh!

seanbudd commented Jun 14, 2021

Uh oh!

AppVeyorBot commented Jul 7, 2021

Uh oh!

AppVeyorBot commented Jul 7, 2021

Uh oh!

AppVeyorBot commented Jul 8, 2021

Uh oh!

AppVeyorBot commented Aug 26, 2021

Uh oh!

AppVeyorBot commented Aug 26, 2021

Uh oh!

AppVeyorBot commented Sep 9, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

seanbudd commented May 31, 2021 •

edited

Loading

seanbudd commented Jun 3, 2021 •

edited

Loading