Skip to content

Add support for audio streaming (RAOP) #1059

@postlund

Description

@postlund

TL;DR Audio streaming (AirPlay) is coming to pyatv. Keep on reading if you are interested (instructions at the end).

So... I'm pretty excited about this! Soon after opening this issue, I will push a PR (#1060) with an MVP implementation of Remote Audio Output Protocol (RAOP) support in pyatv. For those not familiar with RAOP, it's the audio streaming part of AirPlay (previously also called AirTunes). It's basically what makes it possible to stream audio to an AirPlay receiver, e.g. an AirPort Express or a HomePod. Something able to send a stream to a receiver is called a sender, so this makes pyatv an AirPlay sender.

There are several AirPlay senders out there today, both open source and proprietary. One example of a great open source sender is owntone (formerly known as forked-daapd). A lot of inspiration has been taken from owntone and also other open source senders when creating this, so a big thank you to all involved! A list of sources is available at the end.

Support for streaming audio to AirPort Expresses and HomePods have been requested for a while. As there doesn't seem to exist any working RAOP implementations in python, I have deferred that until later as there's a lot of work involved in writing that code. Python is maybe not the ideal language either, mainly for performance reasons. But I took on the challenge and I have written a tiny implemetation to get started. There are lots and lots of issues and lack in functionality, so don't expect to have anything useful in a while. But I decided that I want to merge something and put it out there for people to try out. So please, give it a spin and let me know if it works for you. It's interesting to know what kind of AirPlay receivers you stream to and what kind of device you stream from (performance reasons).

My main driver (and what I'm prioritizing) is support for the HomePod mini. Basically, getting good enough support in pyatv so that it can be used for TTS in Home Assistant. I was generously donated a HomePod mini (thank you!) for this particular usecase, so I'm gonna honor that and do what I can to make it happen! This doesn't mean that it won't work with other receivers (it should), but it's not something I will give additional time investigating for now. I have tried my current implementation with success on my HomePod mini and Yamaha RX-V773 receiver.

As I mentioned, the support is currently very limited, the design isn't very good and there are lots of issues beneath the surface. But better with something than nothing. I'm gonna use this issue as sort of an "epic" for work to be done. Below is a list of limitations and other tasks that needs to be dealt with. The main idea is to create a new issue with a more detailed description for each task as work progresses. I'm very open to external help! If you would like to help out, feel free to pick something below, open and issue and send a PR. Any questions can be asked here, I'll try to answer as good as I can. Streaming and working with audio (i.e. signal processing) is completely new to me, so I'm learning as I go along. Likely lots of beginner mistakes. So please challenge me!

Current state

  • As you might have already guessed, it's now possible to scan and connect to pure AirPlay speakers/receivers.
  • All audio sent to the receiver is sent in raw format (L16). It is however encapsulated in an ALAC frame. Some overhead comes with this as ALAC has a 23bit frame size (i.e. poor alignment). The bitarray library is used to build the ALAC frame as it's optimized in C. Futher down the road, proper ALAC support is deseriable.
  • A simple wrapper is built in for the pyminiaudio library. This means that all media files supported by that library can be streamed, which currently means wav, flac, vorbis and mp3.
  • Only AirPlay 1 receivers not requiring any encryption or password is supported. I believe the HomePod only requires pairing and encryption when leveraging AirPlay 2 features, so that can be skipped until later.
    • Everything related to time and synchronization is pretty messy. I'm mostly inspired by RAOP-Player for this. Probably needs some clean up.

Things to do

  • Compensate with additional packets when being late during streaming #1061: The audio will only play for a short while, typically around 10s on my computer. This is basically bescause my "send loop" isn't compensating for being late (i.e. it takes too much time to process audio frames). Each second, sample rate (typically 441000) amount of packets containing 352 frames are to be sent (each frame contains two samples in case of stereo). I divide the time into 1/44100 slots, send 352 frames and then sleep until the next slot. This however adds a bit of extra time for each slot, so it takes longer than one second to send all packets making the receiver underflow and stops playing. Depending on the computer performance, this might differ a bit though. 
  • Send metadata updates periodically during streaming #1062: Some times, playback actually works longer than I described above. If that's the case, then playback will stop around 30s in. This seems to be a known behavior with the Apple TV and HomePod and a workaround is to send metadata every 25s. See here and here.
  • There are some minor distortions in the sound that can be heard sometimes. Usually at the end and with sound files that are mostly quiet (I have some random test clips I found on the net with various musical instruments). Not sure why that is. I believe this has to do with the source file and not my implementation.
  • Support retransmission of lost packets when streaming #1079: Packets are just blindly sent out. A ring buffer should be added and packets buffered in case retranmission is needed.
  • Read metadata from audio file and send that to receiver #1075: Metadata (title, album and artist) are just hardcoded to dummy values. Ideally, this would be fetched from the audio file but also support manual override. miniaudio however does not support reading metadata, so that would mean an additional library is needed. Maybe worth it?
  • Track length is always set to three seconds when streaming with RAOP #1094: Media length is hardcoded to 3s (as demo). When playing, it seems like the play state remains in "pause" and I'm not entirely sure how to fix that.
  • Support remote control on AirPlay receivers #1068: Media controls doesn not work. Maybe this?
  • Respect audio properties provided by receiver and transcode input audio to match #1067: The code is written to respect sampling rate, sample size and number of channels of the source file as much as possible. I have however only succeeded to play anything in 44100/2/16bit, so that's what is considered supported.
  • To help out with troubleshooting, it would be great with some kind of performance/statistics during and after streaming. Some numbers are already printed, but this can be extended. Maybe even introduce a public interface?
  • As a test, a new method called stream_file has been added to the Stream interface. This kind of interface is however very limited as it does not give any real opportunities to the user for further interaction (e.g. play something and still allow pause, rewind, etc) without the use of a task. Maybe spawn in background and allow some kind of listener to call back when playback ends? Needs a bit of thinking. It's still possible to keep a simple method like this for conveience and add a more flexible solution later.

Limitations and out of scope

Try it out

Works more or less the same as play_url:

$ atvremote -s 10.0.0.4 stream_file=some_file.mp3

Easy as that.

References

Some references I've used, but probably missed some:

https://github.com/philippe44/RAOP-Player
https://github.com/owntone/owntone-server
https://openairplay.github.io/airplay-spec/introduction.html
https://emanuelecozzi.net/docs/airplay2
https://stackoverflow.com/questions/34584522/airplay-protocol-how-to-use-raw-pcm-instead-of-alac
https://nto.github.io/AirPlay.html

Metadata

Metadata

Assignees

Labels

featureraopRemote Audio Output Protocol

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions