Audio enhancements#76
Conversation
stevesims
left a comment
There was a problem hiding this comment.
some notes on the implementation
| #define AUDIO_CMD_PLAY 0 // Play a sound | ||
| #define AUDIO_CMD_STATUS 1 // Get the status of a channel | ||
| #define AUDIO_CMD_VOLUME 2 // Set the volume of a channel | ||
| #define AUDIO_CMD_FREQUENCY 3 // Set the frequency of a channel | ||
| #define AUDIO_CMD_WAVEFORM 4 // Set the waveform type for a channel | ||
| #define AUDIO_CMD_SAMPLE 5 // Sample management | ||
| #define AUDIO_CMD_ENV_VOLUME 6 // Define/set a volume envelope | ||
| #define AUDIO_CMD_ENV_FREQUENCY 7 // Define/set a frequency envelope | ||
| #define AUDIO_CMD_ENABLE 8 // Enables a channel | ||
| #define AUDIO_CMD_DISABLE 9 // Disables (destroys) a channel | ||
| #define AUDIO_CMD_RESET 10 // Reset audio channel |
There was a problem hiding this comment.
This is the audio command set. As noted in the PR description, all of these commands are implemented except for AUDIO_CMD_ENV_FREQUENCY.
| #define AUDIO_WAVE_DEFAULT 0 // Default waveform (Square wave) | ||
| #define AUDIO_WAVE_SQUARE 0 // Square wave | ||
| #define AUDIO_WAVE_TRIANGLE 1 // Triangle wave | ||
| #define AUDIO_WAVE_SAWTOOTH 2 // Sawtooth wave | ||
| #define AUDIO_WAVE_SINE 3 // Sine wave | ||
| #define AUDIO_WAVE_NOISE 4 // Noise (simple, no frequency support) | ||
| #define AUDIO_WAVE_VICNOISE 5 // VIC-style noise (supports frequency) | ||
| #define AUDIO_WAVE_SAMPLE 8 // Sample - values 8+ give sample number plus 8 |
There was a problem hiding this comment.
I changed the default wave sound to be a square wave. That's more typical of 8-bit era machines and, well, sounds a bit better than a sawtooth. 😁
The wave types exposed here are essentially just those offered by fab-gl. I am considering implementing a noise generator that implements noise types similar to the SN76489 used in the BBC Micro and various other computers and arcade machines of the 80s.
| #define AUDIO_STATUS_ACTIVE 0x01 // Has an active waveform | ||
| #define AUDIO_STATUS_PLAYING 0x02 // Playing a note (not in release phase) | ||
| #define AUDIO_STATUS_INDEFINITE 0x04 // Indefinite duration sound playing | ||
| #define AUDIO_STATUS_HAS_VOLUME_ENVELOPE 0x08 // Channel has a volume envelope set | ||
| #define AUDIO_STATUS_HAS_FREQUENCY_ENVELOPE 0x10 // Channel has a frequency envelope set |
There was a problem hiding this comment.
audio status can be queried using the AUDIO_CMD_STATUS command. The value returned is a byte with bits set indicating the current status of the channel being queried
| #define AUDIO_STATE_IDLE 0 // Channel is idle/silent | ||
| #define AUDIO_STATE_PENDING 1 // Channel is pending (note will be played next loop call) | ||
| #define AUDIO_STATE_PLAYING 2 // Channel is playing a note (passive) | ||
| #define AUDIO_STATE_PLAY_LOOP 3 // Channel is in active note playing loop | ||
| #define AUDIO_STATE_RELEASE 4 // Channel is releasing a note | ||
| #define AUDIO_STATE_ABORT 5 // Channel is aborting a note |
There was a problem hiding this comment.
The audio channel implementation essentially internally implements a state machine, of which these are the different possible states.
| debug_log("audio_driver: loop %d\n\r", this->_channel); | ||
| } | ||
| switch (this->_state) { | ||
| case AUDIO_STATE_PENDING: |
There was a problem hiding this comment.
The loop essentially is the main driver of the audio channel state machine.
iThe "pending" state is broadly equivalent to the previous code that looked for the flag value being set, in that it kicks off note playback. From there it will set the state either to "playing" or "play loop" depending on how the channel has been set up.
| } else { | ||
| this->_state = AUDIO_STATE_PLAYING; | ||
| // if delay value is negative then this delays for a super long time | ||
| vTaskDelay(pdMS_TO_TICKS(this->_duration)); |
There was a problem hiding this comment.
call here to vTaskDelay is using the pdMS_TO_TICKS helper function, as vTaskDelay expects to be given a number of ticks to wait....
usually a tick is 1ms, but that's not guaranteed. NB the SOUND command implementation in BASIC is already assuming that the duration it provides is in ms.
| } | ||
| break; | ||
| } | ||
| case AUDIO_STATE_ABORT: |
There was a problem hiding this comment.
the abort state is used when something is done that needs to cause playback to abort, such as swapping the waveform on a channel that is playing a note.
| debug_log("audio_driver: ADSRVolumeEnvelope: attack=%d, decay=%d, sustain=%d, release=%d\n\r", this->_attack, this->_decay, this->_sustain, this->_release); | ||
| } | ||
|
|
||
| byte ADSRVolumeEnvelope::getVolume(byte baseVolume, word elapsed, long duration) { |
There was a problem hiding this comment.
the getVolume method of a volume envelope is used inside the loop to update the current volume level of a note.
when there is no envelope applied to a channel, a note of duration zero will be completely silent, as there is no time for a sound to be made.
when an envelope has been set, the attack and decay phases will always make a sound, but the release phase can be interrupted.
this means that the minimum duration of a note is the total of the attack+decay times, so attempting to play a zero duration note will sound.
to interrupt a note during the attack or decay phases you should clear the envelope.
| @@ -0,0 +1,321 @@ | |||
| // | |||
There was a problem hiding this comment.
to aid readability/maintainability I split the audio VDU commands out into a separate file
| // Returns: | ||
| // - Value (0 to 16777215) if 3 bytes read, otherwise -1 | ||
| // | ||
| int read24_t() { |
There was a problem hiding this comment.
used for uploading samples, as a 16-bit length value only allows for a 4s sample
our available memory is only 4MB so a 24-bit value is more than enough
|
Sample playback is now supported. Sample numbers have been changed to be negative values, and the "set waveform" command adjusted so that a negative number provided for a waveform indicates a sample. For example, this sets channel 1 to use sample -1: The sample waveform player does not understand frequency, so the frequency will be ignored on a channel playing a sample. At present, samples will always loop when they reach the end of the sample. There is no ability to control looping, i.e. no way to specify where a sample should loop back to. Sample playback is compatible with volume envelopes. This is still a WIP as there's a few things to improve. Firstly the sample waveform player continues where it left off when it is enabled/disabled. This isn't desirable, as we want samples to play from the beginning when they're started. This can be worked around by always setting the channel to the waveform just before playing a note. Secondly ideally we should provide a direct/simple way to play a sample without looping. This can currently only be done by explicitly playing a note for the duration of the sample (in ms). I'm considering allowing note playback with duration=0 to indicate a sample should be played to its end without looping. Alternatively/additionally I may enhance the sample waveform player to allow control over the looping behaviour. Implementing playing without looping is complicated by volume envelope support, as a sample could be shorter than the minimum envelope duration, and how the release phase is handled also needs some more thought. If a volume envelope is set that has a decay phase then playing a sample specifying the exact duration of the sample would result in the sample looping at the beginning of the decay phase... probably not what we want. |
|
Sample playback now improved. Samples always restart on new play. Behaviour around deleting samples that were assigned to channels does what you'd expect. Groundwork laid to support enhanced looping behaviour, but that will likely be left to a future PR - for now if it's desired to play a sample for only the specified duration then just specify the appropriate duration in the "play" call (subtracting "release" length if a volume envelope has been set) |
|
@breakintoprogram I've marked this PR as ready for review as I believe it's now in a state that can be merged. All the functionality I'd planned is now implemented with the sole exception of frequency envelope support. That need not be a show-stopper though and can be added in a subsequent PR. The only other thing that's missing is documentation and examples. As the docs are currently a read-only wiki I'm not sure how to go about doing that. For now I will write a markdown file and raise a separate PR with that. |
|
I've now implemented frequency envelopes. Rather than increasing the size of this already large PR and making reviewing the code even harder, I've raised a PR on my own repo stevesims#12 with just those additions. @breakintoprogram if you'd prefer I just merged that in so it's just one single big PR I can do that - just let me know |
also refactors audio VDU handler code into separate file
supports indefinite (duration 0) playback
audio channel loop can now handle volume envelopes an ADSRVolumeEnvelope class is supported indefinite duration handling now uses a duration value of -1 (65535 as a plain 16-bit value), allowing a zero-duration note will now sound when an envelope is set setting volume to zero on a channel that is playing an indefinite duration note and has a volume envelope will “release” the note/channel rather than abruptly muting the channel future commit will add support for setting volume envelope via a VDU command
also fix setting volume envelope on an active channel so it swaps to “looping” behaviour, so envelope gets instantly applied properly
status now returns -1 if the channel doesn’t exist preliminary work towards configurable channels
(no support yet for assigning a sample to a channel)
use C++ vectors and smart pointers
a sample can be applied as the waveform to use for a channel sample numbers are now negative values, reducing the ambiguity when using the “sample” audio command, and allowing the “waveform” command to accept the sample number in the waveform byte replacing or clearing a sample will remove it from any channel that may have been playing it, resetting that channel to use the default waveform
adds EnhancedSamplesGenerator which is used in place of the fab-gl SamplesGenerator this accepts a audio_sample object for its sample data. the setFrequency method is used to allow us to indicate to the generator that the index should be reset, as we can’t override the enable call this approach will allow us to add looping information to the audio_sample object, and to use that in the EnhancedSamplesGenerator deleting a sample removes it from any channels it is active on.
b9a8ee7 to
81241e5
Compare
|
Hi @stevesims I'm currently QAing the changes on my Agon, and have noted the comment re changes to BBC BASIC. |
|
@breakintoprogram as noted above, I have a branch with frequency envelope support ready to roll. I have made some further enhancements to this work, but back-porting to this codebase will be rather tricky. the API however hasn't changed - the enhancements to the audio code are mostly around slight improvements to memory management and ensuring more data is stored in psram. I have not yet implemented a volume envelope type that can supports the "enhanced ADSR" style that Acorn implemented. it is on my TODO list tho, and I'll hopefully get to that soon. given the way that envelopes are supported, back-porting should be very doable. |
Alternative audio enhancements, based in part on the work done by @HeathenUK in #31
Taking inspiration from #31 this code re-purposes the unused "waveform" byte of VDU 23,0,&85 and uses it as a "command" indicator. This allows for a wide variety of enhancements to be added to the audio system in an extensible way. Compatibility with existing code is maintained by assuming that calls to VDU 23,0,&85 will have been made with the waveform byte set to zero.
The command set implemented is quite extensive, but essentially only offers facilities that one may have expected to find on 8-bit era machines.
Indefinite note playback is supported. This can be achieved either by giving a duration of
-1to the "play note" command, or by using the "volume" command to set the volume on a silent channel. A channel playing an indefinite duration note is silenced by setting the volume to zero.(NB BBC BASIC will need to be changed to support this, as when a duration value of -1 is set on a
SOUNDcommand it sends over a duration of 12750.)Commands are present to allow for direct adjustment of volume and frequency on a channel. This could potentially allow for complex audio envelopes to be managed by code running on the z80, potentially allowing BBC BASIC to implement the
ENVELOPEcommand... Rather than burdening the z80 with such things though there is support for envelopes within the VDP.The "volume envelope" command applies an envelope to a channel, so all subsequent notes played on that channel will use the same envelope, much like on a synthesiser. Only one envelope type, simple ADSR, is currently implemented. This is similar to the ADSR support in #31 but in this version the "Release" phase of the envelope is interruptible, as that is typically how volume envelopes work. The system can be quite easily extended to support other envelope types, and I may look to implement a type that would allow for functionality closer to the original BBC Micro's
ENVELOPEfunctionality which allowed for the volume level during the "sustain" phase to gradually change.Commands are also present (and implemented) to enable and disable audio channels, reset a channel, get the status of a channel, and also upload 16khz 8-bit PCM samples to the VDP.
This PR is in draft as there are two areas of functionality not yet finished. Firstly the "frequency envelope" support has not yet been written - this will allow for changing frequencies like the invaders style sound @HeathenUK mentioned against his complexity 2 example in #31. Secondly whilst samples can be uploaded to the VDP (and cleared), assigning them to a channel for playback has not yet been implemented. I expect to implement sample playback by this time tomorrow.
I'll write up some documentation with examples soon.