[WIP] A11Y for blind players#3207
Conversation
…on for accessible actors. Refactor.
…random artifacts.
…com/caturria/shipwright into actor-accessibility-experiments
…terrain detection features.
…com/caturria/shipwright into actor-accessibility-experiments
…com/caturria/shipwright into actor-accessibility-experiments
…com/caturria/shipwright into actor-accessibility-experiments
also adds stick burning warning and more actors
bomb stairs room
…com/caturria/shipwright into actor-accessibility-experiments
|
I think it's worth discussing a path forward for this. This PR is gigantic ( I see 2 paths forward for this:
I personally feel that option 2 may be the better choice here. GitHub supports forking repos within the same org, so we could even discuss it being an "official" fork. I'm looking forward to hearing what others think about this. |
…com/caturria/shipwright into actor-accessibility-experiments
| // actors. Manually reset by aim assist provider. | ||
| f32 pitch; // Used to report whether Link is aiming higher or lower than the actor. | ||
| u8 frequency; // How often the sound will be played. Lower frequencies indicate that Link's vertical aim is | ||
| // closer to the actor. } aimAssist; |
There was a problem hiding this comment.
| // closer to the actor. } aimAssist; | |
| // closer to the actor. |
|
@briaguya-ai I've created #5435 & have attempted to simplify the change / cleanup the code, while at same time extending cues into adult phase of game. It only depends on miniaudio as a submodule. I think this PR can be closed in favor of mine & I'd like to resume the discussion of a path forward here I do think there are some blockers to merging, particularly the generation step needs to be improved (ideally it'd be a button, rather than having this reset-rewrite dance) |
|
closing as work has continued in #5435 |
This module provides accessibility features which can be used by blind players to navigate the game world independently. It is implemented by way of a "virtual actor" system. Virtual actors are spatialized sound sources which can be placed at arbitrary locations in a given scene. They can map directly to real actors or be entirely custom objects which identify a location to the player via sound.
Each virtual actor includes a policy which defines its behaviour (which sound effect it plays, how often, how far away it can be heard from, etc).
This mechanism is almost entirely decoupled from the game; there is just enough boiler plate to allow it to access information about actors and include extra audio in the game's output.
In addition to actors, the module also reports the location of specific terrain features (such as walls, hills, ledges and water), also via the virtual actor system.
Due to the limitations imposed on the number of concurrent sounds supported by the game's sound engine, a custom engine was developed (on top of MiniAudio), as well as a tool which extracts all of the sounds from the game and packs them into a new OTR file. Once extracted, the module can freely render the game's own sound effects without interfering with normal operation. It can also render the game's raw audio samples.
To maximize portability, MiniAudio is heavily stripped down: in particular, its I/O and threading features are excluded. Instead of opening a secondary audio handle, it simply mixes its audio in with the game's own audio output before it is sent out to Libultraship.
At this time, the game is playable up to the end of the Deku Tree using only sounds. Several behaviours have been defined for actors in other areas, however more are required in order to make subsequent content fully playable.
To use this module, do the following:
we decided to make this public now as we have finished the first major hurdle: terrain
detection. With terrain implemented as well as a powerful framework for defining accessible behaviours for actors, much of the project going forward will be adding definitions to new actors and brainstorming solutions to specific puzzles and minigames which will pose unique challenges.
If you would like to get involved, please get in touch with us.
Build Artifacts