OpenAI WebRTC API Review
There is a new interface added to OpenAI RealTime models. Now it supports WebRTC! Given the people working on it I'm sure it has to be great so as usual let’s take a look and see what is under the hood in terms of audio transmission. Signalling or Establishment of the connection There are two options for the establishment of a RealTime session with the OpenAI servers: WebSocket signalling : much nicer API without ugly SDPs involved but less suited for public networks. HTTP/WebRTC signalling : has an uglier API including SDP offer/answer negotiations but can work well in real networks that is critical for most of the use cases. In the rest of the post we will focus only in the later (HTTP/WebRTC) that is the most interesting one. Authentication The first step to use these RealTime APIs sending audio data directly from clients to OpenAI servers is to obtain an ephemeral key using you OpenAI API Secret. This is a simple HTTP request that for testing you can do from the command line: `...