{"id":3057,"date":"2018-10-18T22:47:11","date_gmt":"2018-10-19T02:47:11","guid":{"rendered":"https:\/\/webrtchacks.com\/?p=3057"},"modified":"2018-10-20T09:19:25","modified_gmt":"2018-10-20T13:19:25","slug":"sfu-load-testing","status":"publish","type":"post","link":"https:\/\/webrtchacks.com\/sfu-load-testing\/","title":{"rendered":"Breaking Point: WebRTC SFU Load Testing (Alex Gouaillard)"},"content":{"rendered":"

If you plan to have multiple participants in your WebRTC calls then you will probably end up using a Selective Forwarding Unit (SFU)<\/a>.\u00a0 Capacity planning for SFU\u2019s can be difficult \u2013 there are estimates to be made for where they should be placed, how much bandwidth they will consume, and what kind of servers you need.<\/p>\n

To help network architects and WebRTC engineers make some of these decisions, webrtcHacks contributor<\/a> Dr. Alex Gouaillard and his team at CoSMo Software<\/a> put together a load test suite to measure load vs. video quality. They\u00a0published<\/a> their results for all of the major open source WebRTC SFU\u2019s. This suite based is the Karoshi Interoperability Testing Engine (KITE)<\/a> Google funded and uses on webrtc.org to<\/a> show interoperability status. The CoSMo team also developed a machine learning based video quality assessment framework optimized for real time communications scenarios.<\/p>\n

First an important word of caution \u2013 asking what kind of SFU is the best is kind of like asking what car is best. If you only want fast then you should get a Formula 1 car but that won\u2019t help you take the kids to school. Vendors never get excited about these kinds of tests because it boils down their functionality into just a few performance metrics. These metrics may not have been a major part of their design criterion and a lot of times they just aren\u2019t that important. For WebRTC SFU\u2019s in particular, just because you can load a lot of streams on an SFU, there may be many resiliency, user behavior, and cost optimization reasons for not doing that. Load also tests don\u2019t take a deep look at the end-to-end user experience, ease of development, or all the other functional elements that go into a successful service. Lastly, a published report like this represents a single point in time – these systems are always improving so result might be better today.<\/p>\n

That being said, I personally have had many cases where I wish I had this kind of data when building out cost models. Alex and his team have done a lot of thorough work here and this is great sign for maturity in the WebRTC open source ecosystem. I personally reached out to each of the SFU development teams mentioned here to ensure they were each represented fairly. This test setup is certainly not perfect, but I do think it will be a useful reference for the community.<\/p>\n

Please read on for Alex\u2019s test setup and analysis summary.<\/p>\n

{\u201ceditor\u201d: \u201cchad hart<\/a><\/u>\u201c}
\n\"overloaded<\/a>