Skip to content

Edge WebView2: add possibility to create app modules for apps hosting WebView2 interface#16717

Merged
seanbudd merged 6 commits into
nvaccess:masterfrom
josephsl:msedgewebview2
Jun 19, 2024
Merged

Edge WebView2: add possibility to create app modules for apps hosting WebView2 interface#16717
seanbudd merged 6 commits into
nvaccess:masterfrom
josephsl:msedgewebview2

Conversation

@josephsl

@josephsl josephsl commented Jun 18, 2024

Copy link
Copy Markdown
Contributor

Link to issue number:

Closes #16705

Summary of the issue:

Apps hosting Edge WebView2 show up as websites. However, in apps such as new Outlook (olk.exe), it is desirable to disable browse mode by default or offer custom commands and workarounds.

Description of user facing changes

NVDA will recognize and work with app modules for apps hosting Edge WebView2 controls such as new Outlook.

Description of development approach

Similar to wwahost and javaw, a new host app module (msedgewebview2) is created to fetch actual names of aps hosting the webview controls. The module borrows heavily from javaw app module except it is limited to fetching parent process information. Instead of importing appModuleHandler.AppModule, module-level getattr is defined to return the default app module.

Testing strategy:

Manual testing:

  1. Download and install an app that embeds WebView2 controls such as new Outlook (olk) and LinkedIn, or if using Windows 11, launch Widgets (Windows+W).
  2. Without the PR applied, generate develoepr info (NVDA+F1) and observe the appModule line (it will say "msedgewebview2" for app name).
  3. With the PR applied, generate developer info, and observe that the parent process is now the app name ("olk" for new Outlook, for example).
  4. Create an app module for one of the host apps such as olk.py for new Outlook.
  5. Restart NVDA, switch to the app for te just-created app module, then generate developer info (app name and app module name will say the actual app name).

Known issues with pull request:

None

Code Review Checklist:

  • Documentation:
    • Change log entry
    • User Documentation
    • Developer / Technical Documentation
    • Context sensitive help for GUI changes
  • Testing:
    • Unit tests
    • System (end to end) tests
    • Manual testing
  • UX of all users considered:
    • Speech
    • Braille
    • Low Vision
    • Different web browsers
    • Localization in other languages / culture than English
  • API is compatible with existing add-ons.
  • Security precautions taken.

Summary by CodeRabbit

  • New Features

    • Added support for applications utilizing the Edge WebView2 runtime interface, enhancing app module creation capabilities.
  • Documentation

    • Updated developer guide with additional details and examples for creating app modules for apps using Edge WebView2. Clarifications on naming conventions and subclassing were also included.

Thanks.

josephsl added 4 commits June 17, 2024 21:12
…hosting WebView2 runtime to display ther interfaces. Re nvaccess#16705.

Similar approach to javaw.py: look up the parent process for WebView2 app and return the parent process name for use by NVDA when loading app modules for apps hosting WebView2 runtime.
…is loaded. Re nvaccess#16705.

Using module-level __getattr__, load the default app module class if no custom app module is found.
@josephsl josephsl requested a review from a team as a code owner June 18, 2024 03:31
@josephsl josephsl requested a review from seanbudd June 18, 2024 03:31
@coderabbitai

coderabbitai Bot commented Jun 18, 2024

Copy link
Copy Markdown
Contributor

Walkthrough

The changes introduce support for applications using the Microsoft Edge WebView2 runtime interface by allowing NVDA to create app modules for such apps. These modifications include mechanisms to handle the Edge WebView2 runtime, such as identifying parent processes and handling specific features in NVDA's app module interface.

Changes

Files/Paths Change Summaries
source/appModules/msedgewebview2.py Introduced support for apps using the Edge WebView2 runtime.
user_docs/en/changes.md Documented the support for creating app modules for Edge WebView2.
projectDocs/dev/developerGuide/... Clarified and extended guidelines for app modules using WebView2 runtime.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant NVDA
    participant App
    participant WebView2
    User->>App: Open App
    App->>NVDA: Notification of Interface
    NVDA->>WebView2: Identify WebView2 Runtime
    WebView2-->>NVDA: Provide Interface Data
    NVDA->>NVDA: Load Corresponding App Module
    NVDA-->>User: Provide Interaction Interface
Loading

Assessment against linked issues

Objective Addressed Explanation
Add support for loading app modules for apps with Edge WebView2 runtime as user interface (#16705)
Enable browsing mode by default for Edge WebView2 runtime (#16705)
Use parent process detection similar to javaw.exe strategy (#16705)

Recent review details

Configuration used: .coderabbit.yml
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 376b9f4 and ec66ad9.

Files selected for processing (1)
  • projectDocs/dev/developerGuide/developerGuide.md (2 hunks)
Additional context used
Path-based instructions (1)
projectDocs/dev/developerGuide/developerGuide.md (2)

Pattern **/*: Focus on code smells, logic errors, edge cases, missing test cases, security flaws and serious issues. Avoid commenting on minor issues such as linting, formatting and style issues. This project uses tabs instead of spaces, do not suggest usage of spaces over tabs. Are there any 'red flags' in this code that might warrant closer investigation from a security standpoint? Explain what makes them suspicious.


Pattern **/*.md: Focus on grammar, spelling, and punctuation. Also consider readability and clarity of contents. Ensure that changes follow the style of one sentence per line, suggest changes if this is not met.

Additional comments not posted (2)
projectDocs/dev/developerGuide/developerGuide.md (2)

475-478: Clarification on executables hosting interfaces enhances understanding for developers.

The additional details about msedgewebview2.exe provide clearer guidance on how to handle app modules for applications using the Edge WebView2 runtime. This aligns with the PR's objective to support such applications in NVDA.


510-523: Excellent addition of a practical example for app modules using WebView2.

The example clearly demonstrates how to disable browse mode by default for applications using the WebView2 runtime, which is a key feature for enhancing accessibility in such applications. This example is practical and directly supports the objectives described in the PR.


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Outside diff range and nitpick comments (1)
source/appModules/msedgewebview2.py (1)

13-20: Ensure proper error handling for undefined parent process ID.

Consider adding a comment explaining the use of the walrus operator (:=) for clarity, especially for developers unfamiliar with this newer Python syntax.

Comment thread user_docs/en/changes.md
* You can use `scheduleThread.scheduleDailyJobAtStartUp` to automatically schedule a job that happens after NVDA starts, and every 24 hours after that.
Jobs are scheduled with a delay to avoid conflicts.
* `scheduleThread.scheduleDailyJob` and `scheduleJob` can be used to schedule jobs at custom times, where a `JobClashError` will be raised on a known job scheduling clash.
* It is now possible to create app modules for apps hosting Edge WebView2 (msedgewebview2.exe) controls. (#16705, @josephsl)

@coderabbitai coderabbitai Bot Jun 18, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider enhancing the clarity and detail of this entry.

- * It is now possible to create app modules for apps hosting Edge WebView2 (msedgewebview2.exe) controls. (#16705, @josephsl)
+ * NVDA now supports the creation of app modules specifically for applications utilizing the Edge WebView2 interface (msedgewebview2.exe). This enhancement facilitates better integration with such applications, enabling custom command sets and improved navigation. Refer to issue #16705 and PR contributor @josephsl for more details.

This change provides more context and emphasizes the impact of the new feature, aligning with the detailed style of documentation.

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
* It is now possible to create app modules for apps hosting Edge WebView2 (msedgewebview2.exe) controls. (#16705, @josephsl)
* NVDA now supports the creation of app modules specifically for applications utilizing the Edge WebView2 interface (msedgewebview2.exe). This enhancement facilitates better integration with such applications, enabling custom command sets and improved navigation. Refer to issue #16705 and PR contributor @josephsl for more details.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost - perhaps use only the first sentence as per other changes.md entries (remember to put the issue number and GitHub username mentions for third-party contributors).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@josephsl, based on your feedback, here's the revised entry for the changes.md document:

- * It is now possible to create app modules for apps hosting Edge WebView2 (msedgewebview2.exe) controls. (#16705, @josephsl)
+ * NVDA now supports creating app modules for applications using the Edge WebView2 interface (msedgewebview2.exe). (#16705, @josephsl)

This version maintains conciseness and aligns with the style of other entries. Please let me know if this meets your expectations or if further adjustments are needed.



If you found this review helpful, would you consider giving us a shout-out on X?
Thank you for using CodeRabbit!

@josephsl

josephsl commented Jun 18, 2024

Copy link
Copy Markdown
Contributor Author

@coderabbitai, read my initial comment again as the walkthrough is not equal to what I actually wrote. For reference, it does opposite to what you think the PR is doing.

@josephsl

josephsl commented Jun 18, 2024

Copy link
Copy Markdown
Contributor Author

@coderabbitai summarize the pull request based on my latest comment about the walkthrough

@seanbudd seanbudd changed the title Edge WebView2: add possibility to create app modules for aps hosting WebView2 interface Edge WebView2: add possibility to create app modules for apps hosting WebView2 interface Jun 18, 2024
@LeonarddeR

Copy link
Copy Markdown
Collaborator

I think I'm missing how this appModule ought to be used. May be add an example first, e.g. for Outlook?

@josephsl

josephsl commented Jun 18, 2024 via email

Copy link
Copy Markdown
Contributor Author

@josephsl

Copy link
Copy Markdown
Contributor Author

Hi,

One more thing I should have done: add notes about msedgewebview2.exe in dev guide - I'll take care of this soon.

Thanks.

…vaccess#16705.

Note from Leonard de Ruijter: give an example of an app hosting WebView2 runtime. For this one, a third app module example is defined to showcase WebView2 and disabling browse mode by default (example: olk.exe for new Outlook).
Comment thread projectDocs/dev/developerGuide/developerGuide.md Outdated
Comment thread projectDocs/dev/developerGuide/developerGuide.md Outdated
Comment thread projectDocs/dev/developerGuide/developerGuide.md Outdated
Comment thread projectDocs/dev/developerGuide/developerGuide.md Outdated
@seanbudd

Copy link
Copy Markdown
Member

@coderabbitai review

@seanbudd seanbudd merged commit 49a9757 into nvaccess:master Jun 19, 2024
@josephsl josephsl deleted the msedgewebview2 branch June 19, 2024 04:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for loading app modules for apps with Edge WebView2 runtime as user interface

3 participants