WebRTC Video Codecs performance evaluation

The standard and most popular codecs in WebRTC are VP8 and H.264 but those are not the only options we have.   VP9 has been available for a while and some big services are using it and AV1 support has been recently added to Chrome.

When comparing codecs there are interesting considerations like interoperability and licensing but probably the most important factors are how good the codec is in terms of compression and how cheap the codec is in terms of cpu&memory usage.

The compression ratio is usually the first thing we look at and there are many comparisons available for that, but the resource consumption is equally important if we want to be able to use the codecs for real time use cases.

Given that AV1 is available in the Chrome Canary versions I decided to run some tests to get estimation of where we stand in terms of cpu usage for the 4 available codecs in WebRTC ecosystem.   The idea of the test is to compare the whole video pipeline with those 4 codecs and not just the codecs in isolation.

Testing environment

The tests were done with a simple web page establishing a connection between 2 PeerConnections (one sending and the other receiving).  The test page is here in case you want to run the tests yourself:

https://jsfiddle.net/tvo7czxs/

The tests with that page were performed changing 3 variables:  

• Codec: VP8, VP9, H264, AV1
• Resolution: HD, VGA, QVGA
• Bitrate: 200Kbps, 800Kbps, 2Mbps

If you look at the test page it is easy to change those 3 parameters to run the tests with other configurations or in other devices.

The Chrome version used was the latest one synced from git this week (1/2/21) and the tests were ran in a MacBook Pro (2.4 GHz 8-Core Intel Core i9).   

To check the CPU usage I just looked at the average CPU consumed by the Chrome process in the system Activity Monitor after waiting 30 seconds to give time for the stabilisation of bandwidth estimation and resolution/framerate adaption inside WebRTC.  When the results below say 100% that means 1 full core of the machine.

Nothing too fancy but hopefully easy enough to understand.

In that environment I ran the 36 combinations of the parameters a couple of times, averaged the results and summarised them in the following sections: 

QVGA results

For QVGA resolution the results were as expected, with VP9 requiring a bit more CPU than VP8 and AV1 requiring almost 2x CPU as VP8.   H.264 was the one requiring less CPU usage as it is using hardware acceleration for that.

 

% of CPU usage with different codecs
QVGA	200Kbps	800Kbps	2Mbps
VP8	18	22	28
VP9	20	28	33
H264	10	14	15
AV1	36	46	50

VGA results

For VGA the results were not very different but at low bitrates only VP9 was able to maintain the resolution and AV1 was using more than 1 core when increasing the bitrate limit to 2 Mbps.   Quality of H.264 at 200Kbps was really bad with very low framerate and lot of blocking as adaptation is apparently not working well in this case in Chrome.

% of CPU usage with different codecs
VGA	200Kbps	800Kbps	2Mbps
VP8	18	30	46
VP9	26	38	40
H264	8	13	16
AV1	33	80	150

HD (1280x720) results

HD results were similar to VGA but AV1 was not able to encode the original resolution and downscaled it in the tests with all the bitrates.  H264 also behaved very poorly at low bitrate and the difference between VP8 and VP9 cost was much higher than with VGA.

% of CPU usage with different codecs
HD	200Kbps	800Kbps	2Mbps
VP8	17	30	65
VP9	30	50	80
H264	8	12	16
AV1	35	48	90

(In addition AV1 with HD resolution crashed often with a memory issue in a Mac related code, but maybe the bug is fixed at the time you read this)

Encoding vs Decoding cost

I did one more test to split the cost between encoding (Sender side) and decoding (Receiver side).   That test was performed for VGA at 800 Kbps and the results were the next ones for the four codecs under consideration.

The results are not very different but the decoding in case of VP9 and AV1X was relatively cheaper compared with the encoding.

Comparing just the decoding cost with different codecs it looks like AV1 is around 2x more expensive than the others.   VP9 is slightly more expensive than VP8 and VP8 slightly more expensive than H264 but without big differences between those three.

Conclusions

Having new codecs is amazing and there is no doubt AV1 is the future of real-time video communications but it looks like we should wait a bit to be able to use it for general purpose videoconferencing applications.   In the mean time we can probably use it for specific use cases like broadcasting, dedicated powerful devices or to encode the low resolution versions of the video streams when using simulcast.

For other use cases VP8 and VP9 look like still the best options unless you don't worry that much about low bitrate cases or you are using high resolutions and battery/cpu consumption is a big concern and you can consider H264.

Also there is apparently a new libaom patch coming that could improve the performance by a 15% so it would be good to repeat these tests with future versions of Chrome and different devices (AV1 could be potentially more optimized for ARM CPUs).

If you want to replicate the test and share your data I will update these tables/graphs with the average of the measurements. And as usual feedback is more than welcomed either here or in Twitter.

Thx a lot for the early feedback from @murillo, @elminiero, @dan_jenkins, @maniksach.