UDP vs TCP, what is best for Microsoft Teams Voice?
Originally posted on gsx.com
This entry is part of our series on Microsoft Teams Performance monitoring.
As most of you already know, MS Teams works with two main protocols: UDP and TCP. Microsoft and most other VOIP solution providers recommend using Teams with UDP rather than TCP. The purpose of this article is to understand why and in which condition it matters.
We’ve configured multiple Martello Robot Users, using Teams in different simulated network conditions.
You can read more about how the Robot Users are testing MS Teams and other Microsoft 365 services in this blog article.
So, when are UDP or TCP used? Why does Teams sometimes use TCP? Does Microsoft recommend using UDP?
UDP performance is fastest at delivering packets on the network for many reasons. The first is that it doesn’t check if the packet is fully received. Another is that this protocol is not sensitive to packet reorder. This means that UDP can deal with it without considering it as packet loss. The problem is that you must open many more ports on your network for it to work. And that is why, sometimes UDP is not recommended for communication outside of the company network.
In these cases, the protocol used is TCP. This protocol is much more secure because it needs to open significantly fewer ports. On top of that, TCP makes sure that all packets are received at the destination, and if not, it will send them again. However, all that processing takes time, so TCP is supposedly slower.
Let’s see if this is true!
First, we conducted our tests in a normal environment, and then we played with it to stress the network in order to see how the Teams end-user experience would change, depending on the protocol used. We experimented with low available bandwidth, a congested network, packet loss and reorder. We will explore the results in the second part of this article.
First experiment: Basic environment – Stable network and bandwidth
This is an environment that has been designed to be stable from a network perspective.
Here, we are looking at data we gathered for about a week, from 2 Robot Users:
- One using TCP (green)
- The other one is using UDP (grey)
Every 5 minutes they perform a 15-second call to Microsoft 365 and gather the quality and network metrics during the call.
Let’s examine at the critical statistics here.
First, if you want a complete description of all the metrics displayed on the dashboard, you can read this article >>
This explains why each of these metrics are important and what their impact is on the MS Teams end-user experience.
Let’s focus on the MOS.
As you can see here, the Robot User on UDP performance and the one on TCP perform almost identically. Of course, there is some variance due to normal network life and traffic.
Average MOS, healed ratio, and most of the statistics that show voice degradation are almost the same.
To complete these tests in a normal/optimized environment, we did the same tests in a completely isolated lab and what we saw is that TCP and UDP were again, statistically-speaking, identical.
We had expected UDP performance to be much better. But on a high speed, low latency network, the two are identical. That is why we decided to tweak the network to see if the end-user experience would be affected with UDP or with TCP protocol.
Second Experiment: UDP vs TCP on 15% Packet Loss Network
Here we can see the results of the voice quality in a network with 15% of packet loss injected.
We can see right away that the MOS dropped sometimes, even to zero, because calls were failing.
This was expected because packet loss is included in the computation of the MOS score.
The healed ratio changed a lot, that was also expected.
Just like any voice product, MS Teams must do some sample repair because of network instability. The Teams owner anticipated some jitter and packet loss, and therefore have algorithms that can deal with these problems. These statistics are critical to understanding voice quality, and only Martello Robot Users can show you these statistics.
For more information about healed ratio (also called ratio concealed sample average) please take a look at this article >>
Comparing calls using UDP and TCP, here we can see that there is a tremendous amount of healing on UDP performance , which does not have a built-in mechanism for making sure that the packets arrive at the destination. On TCP, it is different because a retransmission takes place to automatically resend the information.
As you can see, the Jitter is impacted too.
This is directly related to the nature of the protocol, as UDP doesn’t really mind the order of the packets. It will send all the packets regardless, whereas TCP will.
Also, you need to consider the codec. The codec 104 is called SILK, which has built-in repair capabilities to try to smooth things over. MS Teams automatically chooses SILK when the network is in trouble because if its capabilities to repair samples.
The value of that healed ratio goes up to 10 and we know that once it passes 7, the users become unhappy.
Obviously, this test was conducted in an extreme environment, as 15% packet loss is drastic. However, it is useful to understand how Teams is repairing calls and how UDP or TCP are impacted by the quality of the network.
After having described the effect of packet loss on Voice quality for UDP and TCP, let’s see what latency brings to the table.
Third experiment: UDP vs TCP with High Latency Network (200ms)
The Microsoft recommendation about latency between the MS Teams clients and Microsoft Edge is 100ms. In order to show you the impact, we have put that figure to 200ms.
First thing you see here is the MOS, which stayed stable, above 4.0.
Again, you can see the jitter, especially for the TCP Robot User and the healed ratio that is way higher for the UDP protocol.
A robotic sound or any low-quality voice is certainly disturbing, but it may also have more impact. For example, when you are having a discussion with someone that uses your language as a second or third language. In this case, it can be complex to understand an accent and the conversation, even if the MOS is above 4.
A last point to notice here is that the latency starts to impact packet reorder. The UDP protocol doesn’t care about the packet order but the TCP protocol does. This is another reason why UDP performance is generally recommended.
Now, let’s focus on these packet reordering statistics.
Fourth Example: UDP vs TCP with 10% Packet Reorder Ratio
It is interesting to see the MOS more stable on the UDP (purple) protocol than the TCP one (red) that shows many more drops.
Drops usually cause user complaints.
Once more, we see how UDP doesn’t care much about the reordering ratio and how it doesn’t impact the call as much. As explained earlier, it is one of the main reasons why UDP is recommended by Microsoft.
The MOS is in high fluctuation, which is why it is important to look at the variance. If we compute the average, it might look fine, but in reality, users would likely complain about half of these calls.
Another interesting point is the healed ratio here. It stays somewhat below the 2% mark that we know as a good limit. It is impressing to see how the Teams client is resilient and able to cover the packet reorder.
Now that we have seen the performance impact of these critical statistics on a UDP Teams call or TCP Teams call, let’s now see how MS Teams deals with a congested network.
Fifth Example: UDP vs TCP with a Congested Network
At your client companies, the network is used for a variety of reasons (large transfer, numbers of emails, multiple calls and presentations at the same time, YouTube, etc.) so that the available bandwidth is often not the maximum possible.
We tested the impact low bandwidth has on voice quality, with both protocols.
Here, you can see the UDP performance in red and in green, the TCP. The differences are truly critical.
The MOS score for the UDP stays in the 4 range. For TCP, it stays between 0 and 3.
The down count is also impressive, meaning a lot of calls would be impossible on the TCP side.
Here is the reality of the situation: you will have a hundred or more people on the same internet in a building, surfing YouTube or streaming Spotify while working with 30 or 40 Teams related calls at the same time. So low available bandwidth will affect your MS Teams client.
That is where you really see the UDP protocol shining.
All the benefits of TCP from a delivery standpoint won’t help in a congested environment. They don’t bring additional value for “smoothing” the sound in a nice fashion into someone else’s headset.
This is the main reason why Microsoft and other VOIP want you on UDP.
Here, TCP is losing data and suffering from the jitter.
Even some MOS scores fall all the way down to zero because the calls dropped.
We can clearly line the Microsoft recommendation with facts here.
Conclusion
In order to understand how UDP and TCP affect the end-user experience, you need to perform tests in a diverse range of conditions, including extremely bad environments. You also need a large sample of tests to cover any behaviour and pattern in your organization. Finally, you need metrics from all your locations.
In order to read more about how Microsoft recommends you test the ability of your location to deliver good Microsoft Teams voice quality, you can read this article >>
If you want more information about how to improve the MS Teams end-user experience, please read this topic >>
At last, is UDP versus TCP a no-brainer?
We’ve seen that in a perfect environment they are identical, but as soon as you use Teams in an unstable environment, UDP clearly has an advantage.
In most companies, if UDP is not possible because of firewall configuration, it will fall automatically back on the TCP protocol. It’s important to be aware in order to understand end-user complaints.
Learn more about our Microsoft Teams performance monitoring solution and download our ‘Ensuring Teams Performance Under the Hood’ and ‘Harness The Power of Top Notch Voice Quality’ ebooks today!