Skip to content

Finish WinMM Removal#17678

Merged
seanbudd merged 5 commits into
masterfrom
finishWinmmRemoval
Feb 14, 2025
Merged

Finish WinMM Removal#17678
seanbudd merged 5 commits into
masterfrom
finishWinmmRemoval

Conversation

@SaschaCowley

@SaschaCowley SaschaCowley commented Feb 6, 2025

Copy link
Copy Markdown
Member

Link to issue number:

None

Summary of the issue:

There are still some remnants of WinMM left in NVDA, as well as some no-longer used code.

Description of user facing changes

None.

Description of development approach

Removed nvwave.getOutputDevices, nvwave.OutputDeviceIdToName and nvwave.outputDeviceNameToId, as these are no longer used internally.

Renamed nvwave.WasapiWavePlayer to WavePlayer, and removed the WavePlayer variable that proxied WasapiWavePlayer.

Removed the underscore prefixes from utils.mmdevice._AudioOutputDevice and utils.mmdevice.getOutputDevices.

Testing strategy:

Unit and system tests.

Known issues with pull request:

None known.

Code Review Checklist:

  • Documentation:
    • Change log entry
    • User Documentation
    • Developer / Technical Documentation
    • Context sensitive help for GUI changes
  • Testing:
    • Unit tests
    • System (end to end) tests
    • Manual testing
  • UX of all users considered:
    • Speech
    • Braille
    • Low Vision
    • Different web browsers
    • Localization in other languages / culture than English
  • API is compatible with existing add-ons.
  • Security precautions taken.

@coderabbitai summary

@SaschaCowley

Copy link
Copy Markdown
Member Author

@seanbudd, per the PR description

There is now no part of the public API that allows enumerating audio output devices.

As I can see add-ons wanting to enumerate output audio endpoints, should I make utils.mmdevice._getOutputDevices part of the public API? Or should we just wait and see if people complain?

@SaschaCowley

Copy link
Copy Markdown
Member Author

@jcsteh sorry to ping you on this, but you've commented on the WinMM removal project before: are you aware of any other remnants of WinMM that are yet to be removed (excluding the SAPI4 code, which is scheduled for removal next year)?

@jcsteh

jcsteh commented Feb 6, 2025

Copy link
Copy Markdown
Contributor

Nothing else I'm aware of or can spot on a brief look. My only open question is our use of winsound.MessageBeep. That uses MessageBeep in user32.dll, which isn't specifically WinMM, but:

  1. Does the simple act of importing winsound cause us to load winmm.dll (because of PlaySound)? Is that a problem for us if it does, even if it's unused?
  2. Does MessageBeep in user32.dll use WinMM indirectly? If so, what are we supposed to use instead in modern Windows development?
    • I'm guessing it doesn't because I'm fairly sure MessageBox, etc. use MessageBeep.

@gexgd0419

Copy link
Copy Markdown
Contributor

I think that this is more about removing WinMM usage from NVDA's code, rather than getting rid of WinMM entirely.

Surprisingly, MessageBeep doesn't load any DLL at all. From its documentation, I guess that the sound is played by the "SystemSoundService", instead of the process calling MessageBeep. MessageBeep merely "queues the sound".

Also, although the SAPI5 synth driver was changed to use WASAPI, it still needs a default wave format as the stream format. The way I'm using is to get the format from the default audio output device of SAPI5, which loads winmm.dll indirectly.

Come to think of it, I realized that the default audio output of SAPI5 is not necessarily the audio device chosen by the user, so this logic may apply wave formats across different devices. I'll see what I can do to fix this.

@SaschaCowley SaschaCowley added this to the 2025.1 milestone Feb 7, 2025
@seanbudd

seanbudd commented Feb 7, 2025

Copy link
Copy Markdown
Member

@SaschaCowley - if there is a need for add-ons it's a good idea to do it. Do we think the proposed API is particularly risky or likely to change?

@SaschaCowley

Copy link
Copy Markdown
Member Author

I don't think it's particularly risky, nor do I think it's likely to change, unless we wanted to move source/utils/mmdevice.py elsewhere

@seanbudd

seanbudd commented Feb 7, 2025

Copy link
Copy Markdown
Member

I think creating an API is worthwhile then

@jcsteh

jcsteh commented Feb 7, 2025

Copy link
Copy Markdown
Contributor

Also, although the SAPI5 synth driver was changed to use WASAPI, it still needs a default wave format as the stream format. The way I'm using is to get the format from the default audio output device of SAPI5, which loads winmm.dll indirectly.

Hmm. It'd really be a lot nicer if we could get the format that the particular speech engine uses natively (or prefers). That's how this works everywhere else: we let Windows resample as needed, rather than making the synth resample.

Failing that, I wonder whether we should just always specify 48k 16 bit mono.

@gexgd0419

This comment was marked as off-topic.

@SaschaCowley

Copy link
Copy Markdown
Member Author

@jcsteh

  1. Does the simple act of importing winsound cause us to load winmm.dll (because of PlaySound)? Is that a problem for us if it does, even if it's unused?

It looks like winsound is implemented purely in c. As far as I can see, it doesn't include any WinMM code, directly. However, it does include mmsystem.h, which is where some functions in winmm.dll are declared. I don't know enough about the minutiae of how Windows loads such things to say whether that's a problem.
https://github.com/python/cpython/blob/45db419c3104a14007ea9efbc4bff03aef8ed10c/PC/winsound.c#L38-L40

  1. Does MessageBeep in user32.dll use WinMM indirectly? If so, what are we supposed to use instead in modern Windows development?
    • I'm guessing it doesn't because I'm fairly sure MessageBox, etc. use MessageBeep.

No idea on that one

@SaschaCowley SaschaCowley marked this pull request as ready for review February 7, 2025 07:08
@SaschaCowley SaschaCowley requested a review from a team as a code owner February 7, 2025 07:08
@SaschaCowley SaschaCowley requested a review from seanbudd February 7, 2025 07:08
@jcsteh

jcsteh commented Feb 7, 2025

Copy link
Copy Markdown
Contributor

It looks like winsound is implemented purely in c. As far as I can see, it doesn't include any WinMM code, directly. However, it does include mmsystem.h, which is where some functions in winmm.dll are declared. I don't know enough about the minutiae of how Windows loads such things to say whether that's a problem. https://github.com/python/cpython/blob/45db419c3104a14007ea9efbc4bff03aef8ed10c/PC/winsound.c#L38-L40

It would need to link against winmm.lib and would thus load winmm.dll. The question is whether it tells the linker to delay load the dll or not. I also don't know how much NV Access cares about winmm.dll just being loaded if it isn't used; I'm just flagging it in case you do.

@jcsteh

jcsteh commented Feb 7, 2025

Copy link
Copy Markdown
Contributor

@gexgd0419, it might be worth it, but as you say, it does carry some risk. Honestly, I'm inclined to suggest we should just always pass a constant format; e.g. 48k 16 bit mono. The Windows mixer probably runs at 48k on most systems these days except for higher end audio interfaces where it might run higher, though that is anecdotal based on what I've seen; I have no data to reference.

@jcsteh

jcsteh commented Feb 7, 2025

Copy link
Copy Markdown
Contributor

I guess using the audio device's format is better than hard-coded; you resample once (or not at all) instead of potentially twice. I imagine there are WASAPI calls to get that info, but I haven't looked at how they work. But even then, if the default device changes or a device is disconnected, you're now potentially using a different format without knowing and now you're resampling anyway.

There are no good answers. :(

@SaschaCowley

Copy link
Copy Markdown
Member Author

It would need to link against winmm.lib and would thus load winmm.dll. The question is whether it tells the linker to delay load the dll or not.

It doesn't look like they do.
https://github.com/python/cpython/blob/45db419c3104a14007ea9efbc4bff03aef8ed10c/PCbuild/winsound.vcxproj#L95-L99

I also don't know how much NV Access cares about winmm.dll just being loaded if it isn't used; I'm just flagging it in case you do.

Neither do I. @seanbudd, @gerald-hartig?

@seanbudd

Copy link
Copy Markdown
Member

We don't want to load the DLL. The security risk comes from loading it

@jcsteh

jcsteh commented Feb 10, 2025

Copy link
Copy Markdown
Contributor

In that case, you'll probably want to replace calls to winsound.MessageBeep with (likely indirect) calls to ctypes.windll.user32.MessageBeep.

@gexgd0419

Copy link
Copy Markdown
Contributor

@seanbudd I could prevent SAPI5 from loading winmm.dll by not touching any part related to SAPI5's support of audio devices. However, winmmbase.dll would still be loaded when using a system built-in SAPI5 voice.

Is loading winmmbase.dll also a security risk?

If so, then I guess that using any system built-in SAPI5 voice would become a security risk, because MSTTSEngine_OneCore.dll has dependency on winmmbase.dll. From its name, I would guess that all OneCore voices would be affected as well.

@seanbudd seanbudd added the conceptApproved Similar 'triaged' for issues, PR accepted in theory, implementation needs review. label Feb 10, 2025
@seanbudd

Copy link
Copy Markdown
Member

@SaschaCowley - could you resolve merge conflicts when you can?

# Conflicts:
#	source/nvwave.py
#	user_docs/en/changes.md
@SaschaCowley

Copy link
Copy Markdown
Member Author

@seanbudd done

@AppVeyorBot

Copy link
Copy Markdown

See test results for failed build of commit 876001990e

@seanbudd seanbudd merged commit c00a14f into master Feb 14, 2025
@seanbudd seanbudd deleted the finishWinmmRemoval branch February 14, 2025 04:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api-breaking-change conceptApproved Similar 'triaged' for issues, PR accepted in theory, implementation needs review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants