Have you ever wondered how video conferencing apps (like Zoom, Google Meet or Slack Huddles) work?
A while ago I was wondering about it, and found out it was all built on something called WebRTC.
Here’s a description of WebRTC by MDN:
WebRTC (Web Real-Time Communication) is a technology that enables Web applications and sites to capture and optionally stream audio and/or video media, as well as to exchange arbitrary data between browsers without requiring an intermediary. The set of standards that comprise WebRTC makes it possible to share data and perform teleconferencing peer-to-peer, without requiring that the user install plug-ins or any other third-party software. WebRTC consists of several interrelated APIs and protocols which work together to achieve this.
The interesting thing about WebRTC is that it’s a peer-to-peer protocol, meaning your computer and my computer would be talking to each other directly without needing server in the middle! (well, we need one temporarily which I will discuss later)
I searched for resources about how to use WebRTC to build a video conferencing app from scratch, but found no good resources.
My search criteria was:
- Built on WebRTC from scratch without any packages/services.
- Written in TypeScript.
- Uses modern browser APIs.
All tutorials started with “Just use this package/service which handles everything for you”, which is useful if you want to build the app quickly, but not very useful if you want to build it yourself to learn how it really works.
Then I learned about how WebRTC really worked when I was reading the High Performance Browser Networking book (which I highly recommend for anyone interested in computer networking), and it even had a working example!
But the code was written in JavaScript (not TypeScript) and used browser APIs which are now deprecated, so I decided to it myself.
I want go into explaining WebRTC as it’s already explained well by book here https://hpbn.co/webrtc, but I have a couple of notes.
Firstly, we do need what’s called a Signaling Server which is what would be used by the peers to exchange their IPs and other metadata about their capabilities (like which video quality the camera allows), but technically speaking you can use pigeons or WhatsApp to exchange this data with the peers, although that’s not very practical.
Secondly, check this section in the book https://hpbn.co/webrtc/#audio-opus-and-video-vp8-bitrates
Audio (OPUS) and Video (VP8) Bitrates When requesting audio and video from the browser, pay careful attention to the size and quality of the streams. While the hardware may be capable of capturing HD quality streams, the CPU and bandwidth must be able to keep up! Current WebRTC implementations use Opus and VP8 codecs: The Opus codec is used for audio and supports constant and variable bitrate encoding and requires 6–510 Kbit/s of bandwidth. The good news is that the codec can switch seamlessly and adapt to variable bandwidth. The VP8 codec used for video encoding also requires 100–2,000+ Kbit/s of bandwidth, and the bitrate depends on the quality of the streams:
As a result, a single-party HD call can require up to 2.5+ Mbps of network bandwidth. Add a few more peers, and the quality must drop to account for the extra bandwidth and CPU, GPU, and memory processing requirements.
So while the the video call can indeed by peer-to-peer, it’s not very practical if the number of peers in the call start to exceed 4 or 5, so what apps like Zoom or Google Meet do is use a SFU (Selective Forwarding Unit), which receives all traffic from all peers and forwards them to the rest of the peers.
So instead of you sending and receving streams to and from 10 peers, you only do it with the SFU, and it forwards it to the rest of the peers, singinficantly reducing the needed bandwidth, and the SFU can apply all of sort of media optimizations to the streams.
This was out of scope for me when implementing my app, so I want with the true peer-to-peer approach.
And I implemented the Signaling Server using WebSockets via Node.js and socket.io.
It felt really good to make a video call with my friend and see his face via my own video conferencing web app!
Here’s the repo: https://github.com/adhamsalama/webrtc.
And here are the resources I found useful: