Skip to content

feat(audio): add MiniAudio backend as alternative to OpenAL+FFmpeg#159

Merged
fbraz3 merged 19 commits into
mainfrom
opencode/tidy-orchid
Jun 16, 2026
Merged

feat(audio): add MiniAudio backend as alternative to OpenAL+FFmpeg#159
fbraz3 merged 19 commits into
mainfrom
opencode/tidy-orchid

Conversation

@fbraz3

@fbraz3 fbraz3 commented Jun 12, 2026

Copy link
Copy Markdown
Owner

Description

Implement miniaudio (miniaud.io) as a single-header audio backend that replaces both OpenAL and FFmpeg for audio decoding/playback. FFmpeg is still used for video frame decoding (FFmpegVideoPlayer).

Based on implementation by @feliwir https://github.com/feliwir/CnC_Generals_Zero_Hour

Changes

New Files

  • cmake/miniaudio.cmake — FetchContent dependency (single-header approach)
  • Core/GameEngineDevice/Include/MiniAudioDevice/MiniAudioManager.h — Header (derived from feliwir's fork)
  • Core/GameEngineDevice/Source/MiniAudioDevice/MiniAudioManager.cpp — Full AudioManager implementation (~1960 lines)
  • Core/GameEngineDevice/Source/MiniAudioDevice/MiniAudioStream.h — Video audio stream header
  • Core/GameEngineDevice/Source/MiniAudioDevice/MiniAudioStream.cpp — Video audio stream implementation

Modified Files

  • CMakeLists.txt — Added miniaudio.cmake include
  • CMakePresets.json — linux64-deploy/macos-vulkan now default to MiniAudio; added linux64-openal/macos-openal legacy presets
  • Core/GameEngineDevice/CMakeLists.txt — Added SAGE_USE_MINIAUDIO block
  • GeneralsMD/Code/GameEngineDevice/CMakeLists.txt — Added miniaudio_lib link
  • Generals/Code/GameEngineDevice/CMakeLists.txt — Added miniaudio_lib link
  • GeneralsMD/.../SDL3GameEngine.cpp — Added MiniAudio instantiation
  • Generals/.../SDL3GameEngine.cpp — Added MiniAudio instantiation
  • GeneralsMD/.../FFmpegVideoPlayer.cpp — Added MiniAudio video audio support
  • Core/.../WWAudio/WWAudio.h — Fixed MilesStub guard for MiniAudio
  • Core/.../WWAudio/Utils.h — Fixed MilesStub guard for MiniAudio
  • Core/.../WWAudio/SoundBuffer.h — Fixed MilesStub guard for MiniAudio
  • Core/.../WWAudio/AudibleSound.h — Fixed MilesStub guard for MiniAudio
  • Core/.../WWAudio/MilesStub.h — Updated guard to include SAGE_USE_MINIAUDIO

Architecture

Game Audio (WAV/MP3/FLAC) → miniaudio decodes natively → playback
Video Frames              → FFmpeg decodes video       → display
Video Audio               → FFmpeg extracts audio      → MiniAudioStream → playback

Build

# Default (miniaudio)
cmake --preset linux64-deploy    # Linux
cmake --preset macos-vulkan      # macOS

# Legacy (OpenAL)
cmake --preset linux64-openal    # Linux
cmake --preset macos-openal      # macOS

Copyright Attribution

Testing

  • Zero Hour compiles (macOS arm64)
  • Generals base compiles (macOS arm64)
  • Game launches and audio works (MacOS)
  • Zero Hour compiles (Linux)
  • Generals base compiles (Linux)
  • Game launches and audio works (Linux)

fbraz3 added 11 commits June 12, 2026 01:32
Implement miniaudio (miniaud.io) as a single-header audio backend that
replaces both OpenAL and FFmpeg for audio decoding/playback. FFmpeg is
still used for video frame decoding (FFmpegVideoPlayer).

Key changes:
- MiniAudioManager: full AudioManager implementation using miniaudio
  engine with custom VFS for game file system
- MiniAudioStream: video audio streaming support
- cmake/miniaudio.cmake: FetchContent dependency (single-header)
- SAGE_USE_MINIAUDIO build flag (mutually exclusive with SAGE_USE_OPENAL)
- Updated MilesStub.h guards to work with MiniAudio backend
- linux64-deploy and macos-vulkan presets now default to MiniAudio
- linux64-openal and macos-openal presets added as legacy fallback

Based on implementation by Stephan Vedder (feliwir):
https://github.com/feliwir/CnC_Generals_Zero_Hour

Copyright 2026 Stephan Vedder (MiniAudioManager derived code)
Copyright 2025 Electronic Arts Inc.
The SAGE_USE_MINIAUDIO define was not reaching WW3D2 and other Core
libraries because it was only added to miniaudio_lib target. Add it
to core_config (same as SAGE_USE_OPENAL) so it propagates globally.
- Move MiniAudioStream.h to Include/ (matches OpenALAudioStream pattern)
- Fix UnsignedIntPtr -> UnsignedInt in MiniAudioManager signatures
- Fix BitTest -> BitIsSet macro name
- Fix getUninterruptable -> getUninterruptible method name
- Fix getFileLengthMS to use ma_decoder_init_vfs (API compat)
- Disable CoreAudio backend on macOS to avoid Byte typedef conflict
  between MacTypes.h (UInt8) and BaseTypeCore.h (char)
Match game's Byte typedef to macOS MacTypes.h (unsigned char instead
of char) on Apple platforms to avoid redefinition error when miniaudio
includes CoreAudio headers.

Also add AUDIO log line on backend load showing version, device, sample
rate, channels and playback device count.
ma_engine struct does not expose channels/sampleRate members in this
miniaudio version. Log only version, device name, and playback count.
Replace placeholder MiniAudioStream with working implementation:
- Accumulate decoded audio in buffer via bufferData()
- On play(), decode from memory via ma_decoder_init_memory()
- Create ma_sound from decoder for actual audio output

Also add diagnostic logging to playAudioEvent for troubleshooting.
miniaudio's built-in MP3 decoder hangs when streaming through the VFS.
Use FFmpeg (already linked for video) to decode all audio files into
PCM, then feed to miniaudio via ma_decoder_init_memory.

This matches the OpenAL implementation pattern (OpenALAudioCache) and
guarantees compatibility with all game audio formats (WAV, MP3, etc).
ma_sound_init_from_data_source does NOT copy data - it holds a reference.
Using ma_decoder + uninit destroyed the data source before playback.

Fix: use ma_audio_buffer_init which copies PCM data into internal storage.
The sound then owns its own copy and plays correctly after decoder cleanup.

Also store audioBuffer pointer in PlayingAudio for proper cleanup.
@fbraz3 fbraz3 marked this pull request as draft June 15, 2026 00:34
fbraz3 added 8 commits June 14, 2026 21:37
ma_audio_buffer_init() does NOT copy data - it only references external
memory. When the pcmData vector goes out of scope, the audio callback
reads from freed memory causing SIGSEGV on the CoreAudio IOThread.

Fix: use ma_audio_buffer_init_copy() which allocates and copies data
into internal storage. Also set sampleRate on the buffer ref.

Also: remove debug fprintf statements from playAudioEvent.
Replace placeholder MiniAudioStream with working implementation:
- Use ma_audio_buffer_init_copy to properly copy PCM data
- rebuildSound() recreates sound when new audio data arrives
- update() detects new data and triggers rebuild
- Proper cleanup of sound and audio buffer in reset()

This fixes video audio being silent (decoder data was freed before
playback completed).
Add fprintf logs at each step of sound creation:
- FFmpeg decode result
- Audio buffer creation result
- Sound init result
- Sound start result with volume value

This will help diagnose why sounds are created but not heard.
bytesPerSample was from the original format (e.g. 4 for float32) but
after float→s16 conversion the actual data is 16-bit. This caused
miniaudio to interpret PCM16 data as float32, producing extreme
volume and distortion.

Use outputBitsPerSample which tracks the actual output format.
play() now only sets a flag - actual sound creation is deferred to
update() which waits until enough audio data has accumulated (buffer
stops growing or ~30 frames elapsed). This prevents playing only
the first second of audio before the rest has been decoded.
The video player's update() was a no-op for miniaudio - MiniAudioStream::
update() was never called, so the deferred sound creation never happened.
Also fix static variable issue in update() and simplify data detection.
MiniAudioManager.cpp:
- Fix stopAllAudioImmediately: erase before release to avoid iterator
  invalidation when releasing NULL elements
- Fix processFadingList: same iterator fix
- Fix processPlayingList: now respects m_requestStop flag and updates
  3D position every frame for PAT_3DSample
- Fix releasePlayingAudio: only notify SoundManager for PAT_Sample
  and PAT_3DSample, not PAT_INVALID or PAT_Stream
- Fix closeDevice: stop all audio first to prevent use-after-free
- Fix playAudioEvent: always delete ffmpegFile after decode (memory leak)
- Fix playAudioEvent: guard against ma_format_unknown (unsupported bps)
- Fix playAudioEvent: assign audio->m_sound/m_audioBuffer before switch
  to avoid null if early return inside switch (indentation/scope bug)
- Fix playAudioEvent: guard null pos before calling set_position
- Remove all fprintf debug spam from production code paths
- getHandleForBink: provide engine to MiniAudioStream via setEngine()
  so stream doesn't need to call TheAudio->getDevice() each play()

MiniAudioStream.h/.cpp:
- Add setEngine() to receive engine reference from MiniAudioManager
- Add m_soundCreated flag to prevent double-init
- Add m_stableFrameCount: wait 2 frames of no new data before creating
  sound (more robust than single-frame check)
- Rename rebuildSound() to createSound() - called only once per play()
- Remove TheAudio dependency from stream (engine now injected)
- Cleaner reset() that preserves m_engine (externally owned)
…deo streaming

Refactor MiniAudioStream to use a ma_data_source instead of accumulating all PCM data before playing. This resolves video audio silence on both base Generals and Zero Hour because video audio is decoded frame-by-frame and the buffer is constantly growing. Added std::mutex for thread-safe FIFO buffer access, and automatic memory reclamation after 256 KB has been read. Declared the SAGE_USE_MINIAUDIO option in config-build.cmake.
@fbraz3 fbraz3 marked this pull request as ready for review June 15, 2026 23:09
@fbraz3 fbraz3 merged commit 25e6162 into main Jun 16, 2026
8 checks passed
@fbraz3 fbraz3 deleted the opencode/tidy-orchid branch June 16, 2026 17:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant