How are Images/Videos sent in WhatsApp?

I've been involved in the development on different IM&P services in the past and always one of the core features was the ability to share media files (audio, video, pdf...) with other users or groups.


Probably the most popular of these services was TU Me that was based on SIP but used HTTP to upload and download the files from a central storage. SIP was used to send the message with the url to the other end so that he can download the media file.

In cases where I had to use XMPP there were typically two approaches at least if you need to provide offline messaging:
  • Some inefficient inband alternatives where the media files are sent using the XMPP connection.
  • Custom (or not widely supported like xep-0363) extensions based on HTTP Uploads.
But today I was curious on how this is done in WhatsApp.  I found this opensource python implementation (yowsup) that looks great and got an idea on how it could be done but I wanted to confirm what is the protocol used by the most recent WA clients and also get some captures of the messages to see how everything looks like.

Obviously the easiest way was to use to WA Web Client because you have the Chrome developer tools in Chrome so I tried sending and receiving some files from the browser.

Sending a file

1/ Request a url to upload the media file

The first thing the client does when it needs to send a media file is to request an upload url from the server.  This is done using the signaling channel over WebSockets with a message with the action "encr_upload", the type of the content ("image" or "video") and an long id that looks like a hash of the file to be sent.


The server replies with a "status=200" and a dedicatdd url where the file has to be uploaded.  In my case the url was: https://mmg.whatsapp.net/u/f/CIX33...._kOA/Ajlc....78Xf

2/ Upload the media file

The file is uploaded to the url received in the previous message using a HTTP POST as you can see in the next capture:


It looks like if the file is uploaded encrypted (appart from using HTTPS) probably as part of the E2E encryption feature in WhatsApp.

3/ Send a message to the recipient of the media file

After the file is uploaded to the mmg.whatsapp.net server the client sends a message using the WebSocket connection to send the actual message to the other participant (yellow line in the first capture).  This message was bigger than expected (2-10K) so I thought it was very likely to include a thumbnail/preview.

I confirmed that in the receiver side.   The size of the preview image in base64 shown in the UI was more or less the size of that signaling message.

Comments

  1. NipuJuly 5, 2020 at 7:49 AM

    WhatsApp constantly keeps updating its software to include a slew of features to provide smooth messaging and call experience to its users.

    ReplyDelete
  2. Noah LiamDecember 6, 2021 at 11:44 AM

    Cómo hackear whatsapp Online Gratis, Puede monitorear y ver de forma remota todos los SMS enviados y recibidos, llamadas, GPS y rutas, conversaciones de Whatsapp, Instagram y otros datos en cualquier teléfono Android.
    Hackear Whatsapp Online Gratis es una aplicación de espionaje invisible para teléfonos celulares que monitorea los datos en el dispositivo deseado, brindándole una idea real de quién está hablando con su hijo o empleado.

    ReplyDelete
  3. INFYCLE TECHNOLOGIESDecember 14, 2021 at 2:18 AM

    Don’t follow your role model. Be the Role model person for others. But it's so simple by getting Hadoop training in Chennai. Because it is an assurance course to bounce back from a double salary. For joining call 7502633633.

    ReplyDelete

Post a Comment

Popular posts from this blog

Bandwidth Estimation in WebRTC (and the new Sender Side BWE)

Image
Bandwidth estimation is probably the most critical component in the video engine of WebRTC. The bandwidth estimation (BWE) module is responsible for deciding how much video* traffic you can send without congesting the network to prevent degradation of the video quality. In the past bandwidth estimation algorithms used to be very rudimentary and based mostly on packet loss. Basically we used to start increasing slowly the video bitrate until we detected packets being lost.  To detect the packet loss you use the standard RTCP feedback mechanisms where the receiver side reports packet loss periodically using RTCP Receiver Report (RR) messages. Modern bandwidth estimation algorithms are more advanced and try to detect congestion before it is bad enough to make routers discard packets. These algorithms predicts congestion analyzing the delay between packets. The idea is that when you start having some congestion, the buffers in the routers will start filling and the delay will be
Read more

Improving Real Time Communications with Machine Learning

Image
When we talk about the applications of Artificial Intelligence / Machine Learning (AI/ML) for Real Time Communications (RTC) we can group them in two different planes: Service Level : There are many features that can be added to a videoconference service, for example identification of the participants, augmented reality, emotion detection, speech transcription or audio translation .  These features are usually based on image and speech recognition and language processing. Infrastructure Level : There are many ways to apply ML that do not provide new features but improve the quality and/or reliability of the audio/video transmission. Service level applications are fun, but they are more for Product Managers and I like technology more, so in the next sections I will try to describe possible applications of AI/ML for Real Time Communications at Infrastructure Level organizing those ideas in five different categories. Optimizing video quality Some of the ML algorithms used fo
Read more

Controlling bandwidth usage in WebRTC (and how googSuspendBelowMinBitrate works)

Image
There are cases when we would like to limit the maximum bitrate being transmitted by WebRTC to avoid wasting resources in the user endpoints or save money reducing the bandwidth usage in our servers.   This is because the maximum bitrate by default in Chrome is around 2Mbps and for many use cases a much lower bitrate provides still pretty good quality.   BTW using a lower bitrate can also help with stability of quality in multiparty scenarios by reducing the amount of competition among different streams. There is no simple API to configure the maximum bitrate in WebRTC (although there is one in ORTC) but there are 3 ways to do this by mangling the SDP. 1. U se the standard  b=AS:BITRATE (Chrome) or b=TIAS:BITRATE  (Firefox) attributes in the SDP for the audio or video channel[1] 2. Use codec specific attributes (this work at least for opus audio codec with maxaveragebitrate property) [2] 3. Use the proprietary x-max-bitrate attribute in the video channel of the SDP answer.  For e
Read more