Category Archives: JavaScript
why you should not use AudioWorklet
Note: The below text is rather technical and meant for Web developers/architects that have an interest in respective WebAudio related infrastructure.
I have been successfully using the old (meanwhile “deprecated”) ScriptProcessorNode infrastructure since around 2012. Since 2018 there is a “modern” replacement API which supposedly solves all the “problems” that Google’s browser developers (and their friends in the Mozilla subsidiary) had identified in the old API. The “modern” API has now been around for 5 years and one would think that any teething troubles of that “new” API should meanwhile be resolved. Therefore everybody should replace his old ScriptProcessorNode garbage with the cool “modern” stuff immediately, right?
One of the main flaws of the old ScriptProcessorNode infrastructure is that it “lives” in cohabitation with the browser’s main UI thread, i.e. it competes for the same CPU core that is used for “all” the remaining UI code. This means that if the CPU is “too slow”, then the UI code may interfere with the music playback and vice versa the music logic may negatively impact the responsiveness of the UI. How relevant that concern is will depend on the actual hardware of the end user (and the amount of work that the web page has to perform). I am the proud owner of a 10 year old Intel i7-4790K desktop processor (which I used for all the measurements in the later parts of this post) but users of modern smartphones may obviously be less fortunate.
For context here some measurements with my respective webSID player: The computational needs of that player are somewhat proportional to the number of SID chips (see “MOS 8580″) that have to be emulated. Below a visualization of the impact an increase from 1 to 2 and then to 3 SID chips has on the browser’s UI thread utilization (see quota needed for”Scripting”). In the webSID example the player requires about +6% (on my machine) of the available processing capacity for each additional SID chip that is emulated.

On my machine and for the above example (which doesn’t use much UI logic) there still seems to be quite a hefty reserve of “idle” CPU time. But that picture can change if the UI also starts to use the CPU. Below the same 3-SID emulation from above but this time the UI uses the streamed data of the 12 SID channels (4 per SID) to draw some graphs on HTML Canvas. It is obvious that as a function of rising UI CPU consumption there is some point at which it would be nice to offload the music related CPU use to somewhere else so that the UI has the complete capacity of the CPU core available to itself.
Let’s try out the AudioWorklet infrastructure and see what happens! For the sake of the experiment I am using my old ScriptProcessorNode based “V2” music player (the old version can the tested here: https://www.wothke.ch/webV2M/). You can try out yourself how well or how badly it fares on the device of your choice. (Let me know your experience/results in the comments!)
I’ve created a new AudioWorklet based version of that player that you can test here: https://www.wothke.ch/webV2M_worklet/ 
The page actually delivers on the “move CPU load elsewhere” goal: Below on the left the old ScriptProcessorNode based implementation and on the right the new AudioWorklet based implementation (these are measurements from the two pages liked above, i.e. the UI needs some CPU to draw the 16 voice streams). On the hardware that I am using, the new implementation frees up 15.6% “Scripting” of the available CPU capacity to whatever else the UI might want to do with it (at the same time there seems to be a +0.7% increase of “System” CPU use – something that one might want to keep an eye on in specific use case scenarios).
So it’s “mission accomplished”, right? Let’s go for it! Or why not? Unfortunately there are some severe trade-offs.. and once again history seems to be repeating.
You remember that incident when the Google “hotshots” (and their vassals) set out to protect us users from pages to play unwanted audio some years ago? Where they then came up with that brilliant “solution” that forced developers to rewrite their existing code to add “user clicking” facilities to be used before any audio could ever be played (uselessly wasting endless amounts of the developer’s time all over the world just to make sure that the user now has to first click the “claim your free i-Phone” button – before the page screams “welcome to porn-emporium” from your office cubicle)? Well done geniuses…
Working with the AudioWorklet infrastructure I often got the impression that once again it was the clueless intern that was tasked to design the new API.. 😦 Below some additional observations I made in the brief period that I prototyped my respective V2 player.
1) The new API adds multiple levels of asynchronous behavior during the construction phase of the AudioWorkerNode. In scenarios where the new code (AudioWorkerNode) is supposed to replace existing old code (ScriptProcessorNode) in the context of existing surrounding code this may be really annoying.
In the old ScriptProcessorNode model there already was some async handing needed if you were using WebAssembly, i.e. the ScriptProcessorNode needed to use some kind of callback to report when the WebAssembly part was actually ready to be used. (If your existing code doesn’t already use that approach.. then life will be even more interesting for you.)
In the new model a “processor” first has to be loaded asynchronously and this is the 1st callback (Promise) that you’ll need to handle before you can even create the AudioWorkletNode. By default that load will report back before the WebAssembly code that it may be using is actually ready to be used. And the only way to get information about what the state on the “processor” side is, is via asynchronous messaging (both ways). Or alternatively you might use the SharedArrayBuffer construct – which comes with severe limitations of its own. And all of this adds more levels of asynchronous callbacks to your code. In my use-case the “processor” obviously isn’t ready to play anything before it has the actual music file that should be played, so that has to again be initialized via asynchronous messaging (two ways before your UI side player code actually knows what the current state is). Other players may need to load additional chains of additional files (e.g. instruments) before they are ready to play – adding more respective roundtrips.
The added complexity obviously affects the whole software development process (from the design of the more complex solution to the debugging of potential problems).
2) To make the above “more interesting”, the designers of the API have added some “random” limitations into the mix. For example there actually are “Atomics” APIs that are designed for “concurrency” scenarious, and which allow to let one thread wait for some shared memory to be updated from some other thread – which makes sense since it potentially avoids wasteful “busy waits”. But the designers blocked use of these APIs from the UI thread – since it “would potentially be bad for user experience” (e.g. if a baboon doesn’t know what he is doing while using it).. so now it is “busy waiting” for everybody instead. Or you can completely rewrite your existing code from the ground up to embrace the new model – which you probably had intended to do from the start, right? “welcome to porn-emporium”!
3) SharedArrayBuffer seems to be one of the nicer and more useful features in the above context. But after the turmoil surrounding the Spectre exploit a few years ago, commonly used browsers had already temporarily blocked that feature completely in panic. So if you invest your time to write code based on the “modern” infrastructure be aware that this rug might be pulled from under your feet at the blink of an eye. Certain APIs apparently are deemed to be “dangerously powerful” and I am amazed to see how the geniuses at Google address these threats. In the case of SharedArrayBuffer you now have to make specific webserver configurations just to get Chrome into the mode which should actually be the default. “welcome to porn-emporium”!
4) I found tool support for the AudioWorklet infrastructure (e.g. Chrome’s DevTools, etc) ridiculously lacking – considering that it is a “feature” that has been officially released 5 years ago. I find it quite strange that respective flaws have apparently gone largely unnoticed – and one explanation that comes to mind is that the feature might not actually be used to the extent that its proponents want to make us believe. Examples: Chrome (v117) does not even show in the “Network” tab what “processor” files it has actually loaded and worse it keeps loading old cached versions of “processor” files instead of the updated versions of those files (not always) – even after shift-reload and browser restart (eventhough Chrome does load the correct version of the exact same file if it is directly opened via its URL). Imagine the fun that you’ll have when trying to figure out why some of your users see a different versions of your code than what you have actually deployed! Even diagnosing simple syntax errors in the “processor” file becomes unnecessarily challenging when Chrome doesn’t give a proper error message and basically only tells you that it could not use the respective code due to “some problem in your ‘index’ file” (that behavior actually seems to depend on the specific type of syntax error and a stray ‘.’ will break the debug info where a stray ‘a’ might not.. LOL). “welcome to porn-emporium”!
5) Google uses AudioWorklet to push its https agenda – eventhough there is absolutely no rational explanation why people should not be allowed to load music playing pages by whatever means THEY chose to use. But Google always knows better what is best and will protect you from yourself if necessary! Therefore pages that use the “modern” AudioWorklet cannot play music when loaded via http. “welcome to porn-emporium”!
6) It is an obvious precondition for realtime music players that the logic producing the music must not take longer than it takes to play the produced audio data. Violations of this precondition will potentially interrupt the continuous playback and lead to clicking or stuttering noises. In the case of the old ScriptProcessorNode it may typically be a 2048 (it may range from 256 to 16k) sized output that is produced, corresponding to 0.046 seconds of audio (at the typical sample rate of 44100Hz). For AudioWorklet that output is always 128 sized, i.e. the interval is typically 16x shorter (and may be up to 128x shorter as compared to the ScriptProcessorNode implementation). This means that in the ScriptProcessorNode case, variations in the actual processing needs can have temporary peaks (e.g. GC kicking in or waveforms that are more expensive to calculate than others) without hurting the playback – as long as the average processing time fits into the available longer time window. Whereas AudioWorklet is much more sensitive to variations – which must average out within each of the much shorter 128 -samples windows. If respective processing time variations are an issue in your existing audio generation logic then you might need yet additional workarounds (e.g. separate Worker to produce larger buffers in the background) when migrating the code to AudioWorklet.
7) I don’t have a smartphone to use my pages on, so I did the “next best” thing to at least get an indication for what to expect by running Chrome-DevTool’s “Lighthouse” analyzer. (Based on past experience respective Lighthouse reports have to be taken with a large pile of salt.) But I was interested to see how the two version of my V2M player supposedly score for Desktop as compared to Mobile devices in the categories “Performance” and “Best practices”. The 3 year old Chrome version that I used first reported 100/100 for both versions and all scenarios except for the old version (ScriptProcessorNode) on Mobile that scored “only” 98/100.
I then repeated the same test on the recent version of Chrome (v117) . Here the old version scored 99/100 on Desktop and 76/100 on Mobile. While the new (AudioWorklet) version scored 97/100 on Desktop and 76/100 on Mobile (i.e. performance supposedly worse on Desktop than the old version..). So these test results suggest to me that for this page the end user experience will be pretty identical for both versions of the page.
I did reach a point where the new version actually performed better than the old version when I simulated a “6x CPU slowdown” via Chrome’s DevTools. However the prospect of having code still work on older/slower devices may be deceiving: Users of respective old devices may often chose to not update their OS and/or browser to the latest versions and other features that you might want to use on your pages (e.g. ES6 classes, WEBGL2, etc) would prevent them from even using your page anyway.
8) From an audio visualization perspective, AudioWorklet’s 128-samples output is an advantage. You no longer need the hacks needed in the ScriptProcessorNode scenario in order to show the currently played data with a precision that exceeds the playback length of the used audio buffer.
9) Be careful to not prematurely jump at any conclusion with regard to some audio playback issue that you might be having. At some point I found that one of my pages produced ugly clicks in Chrome during its audio playback for no good reason: When analyzing the CPU load in the DevTools, respective overall load was well below 50%. But it got really interesting when I then tried to “Record” a “Performance” trace using the DevTools. As soon as the DevTools were in “Record” mode the same song that glitched in “normal” mode suddenly played flawlessly (and it restarted to glitch as soon as I stopped to “Record”). When the browser implemention is garbage you might end up getting the short end of the stick regardless of which particular feature you are using. “welcome to porn-emporium”!
So to return to the clickbait title.. should you use AudioWorklet (as a replacement for existing legacy ScriptProcessorNode implementations or in general)?
I’d say only if you verified that you actually need its potential benefits. The extra price you pay for using the “modern” API is quite high (in terms of extra software development, testing, deployment and maintenance cost, unnessessary lack of http support, extra technological risks, cross-browser issues) and often the “old” API is totally good enough and much easier to use. If AudioWorklet solves a problem that you actually have (only) then you should go for it.
“Google’s” approach of “deprecating” the ScriptProcessorNode and pushing the use of AudioWorklet resembles the move of an imaginary “handyman’s guild” to “deprecate” the use of nails and pushing the use of screws.. “welcome to porn-emporium”!
an old player revisited
Ten years after my first sc68 web port I thought it might be time to update the code base to the latest sc68 dev version. This new version supposedly has improved SNDH playback capabilities and I used the occasion to play with WEBGL and also add a visualization of the emulation’s internal voice streams. A live demo can be found here: http://www.wothke.ch/webSC68
PS: I updated PlayMOD to now also use this version.
PlayMOD online chiptune music player
My voyage into the realm of legacy computer music had originally started with my webSID music player and later continued with my extended webUADE version of the uade Amiga music emulator.
I still have fond childhood memories of my respective C64 and Amiga home computers since these devices ultimately triggered my career in software engineering. Whereas most of the capabilities of respective 40 years old home computers obviously look quite lacking from today’s perspective, their audio features have aged rather gracefully and I feel that the audio stuff is much better suited to preserve the nostalgia – e.g. as compared to the blocky pixel graphics or underwhelming computing power.
I later learned that even though the above devices where obviously the best that ever existed (cough), other people share similar nostalgia but with regard to other devices. In many cases emulators for respective devices already existed on some platform and all that was missing were respective ports so that it would be possible to use them on a Web page. Since this is basically the same thing that I had already done for my webSID and webUADE players I started to also port some of the existing 3rd party emulators (many of which I somewhat enhanced in the process).
Over the years the number of respective JavaScript/WebAssembly based music emulators in my toolbox has grown to around 30 and it was time to put them to good use: PlayMOD combines all of “my” Web emulators within one UI to provide “all-in-one” online browsing and music playback for some of the largest “legacy computer music” collections available on the Internet:
The modland.com collection contains about 455’000 music files from various legacy home computer and game consoles and the vgmrips.net collection adds another 62’000 primarily arcade system songs. The PlayMOD project derives its name from “module files” (MOD) music – which signifies computer music that has been created using some kind of tracker software (the term “tracker” goes back to Karsten Obarski’s Ultimate SoundTracker on the Commodore Amiga from 1987). However, in addition to actual MOD-files the used collections also provide a large number of other music formats, e.g. many of the older stuff would be usually referred to as “chiptune music” today. You may use the Basics tab directly on the PlayMOD page for more background information.
When looking for a MOD or chiptune player, PlayMOD provides the best coverage available due to its combined use of different emulators/player engines. PlayMOD is probably the only comprehensive cross-platform online player in existence today.
There are hundreds of different legacy music file formats involved and the emulators available in PlayMOD currently allow to play more than 99.9% of what is available in the two collections. This avoids having to manually find and install a suitable player for each exotic format (which otherwise may be a tedious task, and a player may not even exist for the platform of your choice, e.g. see Ixalance).
The PlayMOD web page allows to browse the folder/files available in the respective collections but it does not host any of the music files. In order to play a song, the user’s browser will directly retrieve the selected file from the respective collections (see links above) and then play it. Consequently the page will only be able to play the music while the ‘modland’ and ‘vgmrips’ servers are available.
The respective modland and vgmrips collections document the evolution of computer music during the past 40+ years. Having everything consolidated in one place allows to easily compare the capabilities of respective legacy sound systems (e.g. by comparing how the compositions of the same composer sounded on different platforms) or to just indulge in reminiscences.

The PlayMOD user interface is based on the design/codebase originally created by JCH for his DeepSID. I wasn’t keen on creating a UI from scratch so I am glad that I could reuse JCH’s already existing stuff – eventhough it had to be heavily modified. The PlayMOD UI is still in a prototype/proof-of concept stage and the quality of the used meta data (e.g. composer information) leaves a lot to be desired due to it having been automatically generated based on questionable quality raw data.
Obviously, legacy computer music could also be preserved by just creating some recording from the original hardware, and as can be seen on youtube, many people already use that approach. Indeed the approach of using an emulator will not always be as accurate as the use of the original HW (on the other hand recordings may suffer from lossy data compression – a problem that an emulation does not have). As compared to the original music files, recordings may use up much more memory and consequently network bandwidth, but today that isn’t the issue that it might have been 10 years ago. However emulation avoids the additional recording/publishing step and new music files can immediately be listened to – without having to wait for somebody with the suitable infrastructure to provide such a recording. (There actually are still “scenes” where people create new music for legacy systems today.)
From a “legacy computer music preservation” perspective the emulation approach also has the benefit that it not only preserves the end result but also the steps taken to achieve it. It allows for interactions that would not be possible with a simple recording. (Mileage may vary depending on the “original” music file format.)
Example: The “Scope” tab in the below screenshot shows the output of the different channels that some “Farbrausch V2” song internally uses to create its stereo output, i.e. an emulation approach allows to look at the “magic” that is happening behind the scenes.

Similarly a respective emulation could still be tweaked during playback, e.g. by turning certain features on/off, or by using different chip models.
269 Life
In addition to showing some cool ray-marching based realtime graphics, my latest web page is dedicated to “269 Life” and the matching title is meant to attract some extra attention to that meaningful movement (see http://www.269life.com).
The realtime WEBGL web page can be found here:  https://www.wothke.ch/269life/. You’ll need some sort of 3D graphics accellerator and a Chrome browser to use it. Or you can have a look at a youtube recording here.
I am not repeating the information that can already be found in the comment of the youtube video here. Instead I’ve added some background information regarding the techniques that I used in the page.
All the fractal graphics are created using Knightly’s “pseudo kleinian” algorithm (see example code in “Fragmentarium”) as a base and parameterizing it with various “distance estimate” functions. An “orbit trap” based implementation is used to color the result. Depending on the specific “scene” a number of reflections is calculated (up to three). The “Phong Blinn” shading model is finally used in combination with a standard “ambient occlusion” implementation to render the fractal background (basically the same impls that I had previously used in “modum panem”).
Three different approaches are used to display text elements:
- “Flat” texts are created by using a font texture that is then displayed via simple triangles (two per character).
- Texts like the title or the ones in the “greetings” section are then based on extruded 3d fonts (see standard THREE.js examples).
- Finally there are the “particle based” texts that explode in the “greetings” section – which are created using regular canvas text rendering.
A “bokeh” post-processing is applied to the resulting page to create a “depth of field” effect. (The respective implementation is derived from Dave Hoskins work.) The “bokeh” post-processing is also used to create some interesting distortion effects on the overlayed title text (which is not using the same z-buffer).
Finally the “greetings” scene showcases the combination of “standard” THREE.js elements (particles, extruded texts, etc) with the shader generated fractal background: By having the fractal shader propagate its z-buffer information, the regular THREE.js overlays are later clipped correctly (thanks to Marius for the respective depth calculation – see boxplorer2).
The “neon signs” here are created via a post-processing pass that adds a “glow” as well as a “god’s ray” effect. A simple random noise based shader is used to create the purple “northlights” on the horizon and 20’000 confetti particles provide for some action.
Thanks again to Wolf Budgenhagen and LMan for letting me use their music.
modum panem

Obviously there was something missing in my previous mandelbox experiment… exactly, some means of transportation so you can explore the scenery!

I therefore extended the original version such that “regular” 3D stuff can be mixed with the ray-marching based fractal landscape.

also I added a bit of “collision detection” so that the landscape can be moved “out of the way” when your transport gets too close..

so enjoy your ride: https://www.wothke.ch/modum/
or have a look at the youtube recording.

Notice: Unfortunately Chrome seems to be the only browser that currently properly supports WEBGL_draw_buffers. (Firefox also claims to support the feature but in fact it is completely messing up the respective color attachments..
update: It seems that meanwhile the Firefox clowns have managed to get their shit together and the page should now also work in that browser. Note that the first load of the page may be slow.
the beauty of numbers..
I just did a little revival of my old WebGL fractal stuff. The two below screens use the latest version of my Commodore C64 emulator to play two fitting music creations by Markus Klein.
vu meters?
Just a little experiment for how to synchronize visualization of additional data streams with the the playback of WebAudio music: The music samples are generated on the fly using a ScriptProcessorNode emulating some legacy AtariST. In addition to the stereo output the respective ScriptProcessorNode also generates three streams containing “playback volume” for the AtariST’s internal sound-chip voices:
just for fun a more psychedelic WebGL based rendering of the same data (the WebGL here combines an orbit trap fractal with an inverse distortion, and the “music volume” is used to parameterize the rendering):
unaligned “packed structs”…
are certainly not a good idea if a program is supposed to be portable. Unfortunately that is exactly what ZXTune is using to parse the different binary music files.
“One of the rules of packed structs is that you have to access them directly through the packed struct type. In other words, you can’t, in general, take the address of a packed field and then access the field through that pointer, because the pointer type won’t reflect the packed attribute.” (sunfishcode)
Unfortunately ZXTune used boost::array instances within the various packed structs.. Problem: when methods are invoked on boost::array (or std::array, etc). The ‘this’ argument to the boost::array functions may be misaligned, but the boost::array functions themselves don’t know this.
On CPUs which don’t mind unaligned memory access you may get away within without realizing that there is a problem.. and in this case it was my attempt to cross-compile the program using Emscripten that revealed the issue. Not wanting to rewrite too much of the foreign code I opted for a quick-fix: replacing the boost::array with a built-in array fixed the immediate problem…
Naturally a clean implementation should better refrain from depending on unaligned memory access at all… not all the CPUs are as forgiving as Emscripten.
(click on the below image for a live demo).
AdLibido – the wonders of early PC music ;-)
It was back “in the old days” and I remember my relief when some day I found out that all PCs were not necessarily mute: Thanks to some “walking frame” called “AdLib” they could actually make sounds… and a bit later things became pretty neat with the rise of Sound Blaster…
AdPlug plays sound data, originally created for the AdLib (OPL2) and Sound Blaster (Dual OPL2/OPL3) audio boards, directly from its original format on top of an emulator.
My latest bit of Xmas tinkering is a HTML5/WebAudio version of AdPlug (Thanks to Simon Peter and the other authors of AdPlug.). For a live demo click on the below image..
update: The respective page has meanwhile been updated and the used link therefore no longer corresponds to the original screenshot (you’ll need a browser with WEBGL support to use it).
68000, Paula & Co.
Yet another bit of home computer music nostalgia. This time it is the Commodore Amiga that is emulated using UADE. The original Unix Amiga Delitracker Emulator is based on a two process design where the core Amiga emulator engine and the music player’s frontend are separate processes that communicate via IPC. Using a regular Amiga emulator then may be tricky because depending on the song it expects to synchronously load additional data files. Obviously this is on a collision course with the concepts available for an HTML5/JavaScript page.
Bringing this one to the web therefore required some redesign of the original UADE code base. Once that had been done Emscripten again did a splendid job translating the C code into JavaScript and linking it to the manually written JavaScript callbacks (see live demo here).
This experiment once again confirmed my earlier observations that the debugger support built into today’s web browsers is utterly useless (but for the most trivial scenarios). So this not only was a travel back in time with regard to home computer music but also with regard to the modern development tools that I had gotten used to: bye bye IDE – welcome back debug/trace output.





