Keyman for iPhone and iPad Acceptance Test Procedures

1. These test procedures are to be run before moving from alpha to beta, or beta to stable, or before PRs are merged into stable branches.
2. Copy these checklists of tests into a new issue (for tier transitions) or a comment on the PR.
3. When annotating results, use > at the start of a new line under the checkbox to note the result. (For an example, see test(mac): Keyman for mac 14.0 beta acceptance test #3973).

Visual Inspection of master branch

• Verify latest CI build of master is successful
• Verify ios/history.md contains all the current changes

Nope. Now in the repo's root history.md.

• Verify that all unit tests for iOS are currently passing. (Newly added for 14.0)

There seems to be a bit of instability with two unit tests: testCancellationWithAssociations and testFickleSelection.
They usually pass, but may occasionally fail during test runs.

• testCancellationWithAssociations seems to only fail on the initial run of a test suite after its 'host app' has been installed. Re-runs won't fail - it's just the first run.
  • So, while fine for now... it'll definitely need to be fixed whenever we integrate the unit tests into iOS's CI.
• testFickleSelection interestingly only fails for the inProgress case, which is fortunately not actually used within the iOS app.

Gather Assets for Testing

• Record device's iOS version
• Ensure the device is properly configured for testing and is registered with TestFlight.

The tests below were performed targetting iOS 14.2 on a Simulated iPhone 11 Pro Max and iPhone SE (2nd gen).

At this time, Keyman for iOS does not support external keyboard input.

Setup Steps

• Uninstall previous version of Keyman on the device
• Install test build IPA
• On device, disable internet connectivity
• Start Keyman and select Add a keyboard for your language
• Verify error message "Could not reach Keyman server!" is displayed

No useful feedback was displayed. Addressed by #4693.

• Click OK to dismiss. Close Keyman app
• On device, enable internet connectivity
• Start Keyman
• On initial Get Started menu, select "Enable Keyman as system-wide keyboard" and do any necessary system steps to enable Keyman

Menu Functionality Tests

Get Started

• Verify touching Add a keyboard for your language pulls up the Add New Keyboard menu

Launches Settings > Installed Languages, then the keyboard search... which is basically 14.0's "Add New Keyboard" menu, so "success."

• Verify touching Enable Keyman as system-wide keyboard pulls up accurate instructions.

On 13.0+ devices, may appear as "Set up Keyman as system-wide...". Clicking that pulls up the proper location within the iOS settings menu, so "success". The "Keyboards > Enable Keyman" subtext for the needed drilldown is 100% accurate.

• Verify touching More info brings up an accurate version and "Getting Started" page.

With the change to app help, it's a little different now, but is still pretty accurate.

• Set Don't show again to checked, close and re-open Keyman, and verify Get Started menu does not appear

Note that the stated version will be inaccurate if run within the Simulator instead of on a physical device.

No longer true! Yay!

Add New Keyboard

This menu is accessed via Get Started menu or Settings menu

• On the device, enable internet connectivity
• Verify a new keyboard can be downloaded and selected
• Return to the text input screen and confirm that it operates correctly.
• Without changing selected language, confirm that it can be cleanly deleted via the Settings menu.
• Confirm that another keyboard is correctly selected afterward.

Share

• Type and verify text can be copied and shared to external app

Keyman Browser

• On default page, click on text field and set system keyboard to Keyman
• Verify can type with Keyman as system keyboard

Bonus step:

• Verify that predictive suggestions are properly applied when selected

• Close and reopen Keyman app

Text Size

• Verify text can be rescaled from Text Size 16 to 72

Clear Text

• Verify text can be cleared

Info

• Verify that the Keyman iOS version briefly appears and record the version
• Verify that the Info menu options matches that of the More Info... option from the Get Started menu option.
• Verify that the Keyman iOS version matches the version string seen at (Home Screen) > [iOS] Settings > Keyman.

Settings

• Installed Languages ... Add Language
• Select an installed language ... Add (another) Keyboard
• Select an installed language ... Dictionary
• Turn on/off 'Enable Predictions' and validate each
• Turn on/off 'Enable Corrections' and validate each
• Validate that when both predictions and corrections are off, banner is not visible
• Add/Remove dictionaries - validate
• If multiple dictionaries are available, test swapping between them

Uh-oh. The toggles aren't having the proper effect. The banner remains active and predictions/corrections occur... for... the first word of the context? After that, the suggestions are static until the context is cleared or the previous newline is reached.

Keyboard Tests

• Confirm all keyboard tests below have been performed in both portrait and landscape modes.

English (Eurolatin2) in-app

• In portrait orientation, verify that the OSK appears and fills the width the bottom of the screen
• Verify long-press q key works
• Verify long-press p key works

• Issue: when using 's' as a base key for popups, then selecting the subkey that overlays the base 'w' key, predictive text will offer suggestions starting with a 'w'.
  The touch-probability generation algorithm is using the base layer's geometry, not the subkey "layer" geometry, which is less-than-ideal here.

• Verify uppercase layer can be selected via SHIFT
• Verify number layer can be selected via 123
• Verify long-press 1 key works
• Verify long-press 0 key works
• Verify backspace, space, and enter keys work

Note that the shown EuroLatin2 is a highly customized keyboard explicitly designed for mobile.

English: US Basic in-app

• Verify that the US Basic keyboard can be downloaded through Settings > Installed Languages > English > +
  • This should automatically bring up search results for English keyboards.
  • When the package-installer launches, English should be the one pre-selected language.
• Install the package for both English and Tagalog.
• Set English: US Basic as the active in-app keyboard.
• Verify that predictive text suggestions appear.
• Set Tagalog: US Basic as the active in-app keyboard.
• Verify that predictive text suggestions do not appear.
• Verify that long-press 123 key works, presenting options for a currency layer and a symbol layer.
• Select the currency layer, then press the £ key.
• Verify that the keyboard outputs correctly and returns to the default layer.
• Go to the symbol layer and press the © key.
• Verify that the keyboard outputs correctly and remains on the symbol layer.
• Revert to the default layer and ensure basic key inputs work.
• Long-press e on the default layer and select a subkey.
• Verify that the selected key produces the correct output.
• From the Settings > Languages > English > US Basic menu, follow the "Help link" and ensure it displays appropriate help.
  • (Should work both online and offline)
• From the Settings > Languages > English > US Basic menu, scan the QR code with a phone and test that it links to the current version of that keyboard's public download page on keyman.com.

The QR code is not displaying during the app's initial launch. If the app is closed and then relaunched, the QR code will display at that time. (Probably due to query-caching behavior, with the initial query having occurred before 'US Basic' was installed.)

• Returning to the app, delete the US Basic keyboard via Settings for Tagalog.
• Set English: US Basic as the active in-app keyboard.
• Verify that the keyboard appears correctly and ensure basic key inputs work.
  • This is to ensure that uninstalling for one language doesn't adversely affect installations of the same resource for another language.
• Delete the US Basic keyboard via Settings for English.

What's New Tests

• Refer to the new changes in ios/history.md and verify functionality

Nope. Now in the repo's root history.md file, along with all the other platforms' changes.