VOIP should be an easy fit for wireless LANs, but mixing the two technologies today is difficult. Despite VOIP's low-bandwidth profile, even a small amount of data traffic on the same network can lead to seriously degraded audio quality and dropped calls, even with QoS features enabled.

That's the major conclusion of our first-ever assessment of VOIP capability in WLAN systems. Over the course of three months we tested WLAN switches and access points from Aruba, Chantry (now acquired by Siemens), Cisco and Colubris. in terms of audio quality, QoS enforcement, roaming capabilities, and system features. The Cisco system, is the company's own product, built around its Catalyst switches, rather than the switch system it is buying in, with vendor Airespace.

Other vendors, including Airespace, Meru and Trapeze, declined to participate.

Among our major findings:

  • With QoS enforcement enabled, the products delivered near-toll-quality ("nearly as good as wired telephones") audio, provided there is only voice traffic on the network. Unfortunately, this situation is not likely in practice, as companies move toward converged voice-data networks.
  • When voice traffic had to contend for bandwidth (even with a little data traffic), dropped calls were common and audio quality on the remaining calls was poor in many cases - and this was with QoS enforcement enabled.
  • With data traffic present, roaming from one access point to another took anywhere from 0.5 to 10 seconds - in cases where roaming succeeded at all. These long delays and dropped calls made roaming practically impossible with some vendors' gear.

While some products struggled mightily in our tests, Aruba's A2400 and A800 switches and A61 access points were consistently strong performers. The Aruba products posted generally excellent numbers, regardless of how much voice or data traffic was thrown at them. Aruba's gear just worked, earning it the Clear Choice Award.

Two issues confounded other vendors. First, when handling voice and data traffic on the same network, vendors need to pay attention to metrics such as delay and jitter rather than forwarding rates.

Many vendors are only just beginning to tune their products for voice/data convergence, even though some have touted that capability for 18 months or more. However, it's still relatively early days for VOIP over WLANs. Test tools that accurately measure these metrics on WLANs are only just beginning to appear (the VeriWave instruments we used were developed for this test, and the company launched its Wi-Fi Traffic Generator at Wi-Fi Planet in November). This test is among the first to measure audio quality, delay and jitter in a methodical way.

Second, the emerging 802.11e standard for QoS on WLANs might bring some relief. The 802.11e specification wasn't yet ratified when we began this project, so by definition all QoS methods were nonstandard. Companies might want to wait until the new 802.11e specification and products based on it are more mature and fully tested.

How to measure qualityOur tests sought to answer a simple question: How does a VOIP over WLAN system sound?

To find out, we worked with VeriWave, a start-up that makes WLAN test and measurement equipment. VeriWave developed a new application, the VOIP over WLAN Analysis Test Suite, especially for our test.

In addition to collecting delay and jitter statistics, VeriWave's test suite and TestPoint hardware let us measure R-value, an ITU specification (G.107) for determining call quality. R-value is an objective measurement, computed directly from measurements of packet loss, jitter and delay. While R-value is objective, it has a strong correlation to the subjective Mean opinion score method in ITU standard P.80 (see R-value ratings).

R-value ratingsAn ITU specification that determines call quality, R-value measures packet loss, jitter and delay.

R-valueMean opinion scoreUser satisfaction
90 or higher 4.34 or higher All users very satisfied
80 or higher4.03 or higherAll users satisfied
70 or higher3.60 or higherSome users dissatisfied
60 or higher3.10 or higherMany users dissatisfied
50 or higher2.58 or higherNearly all users dissatisfied

We measured voice call quality with up to 14 handsets and an H.323 call server from SpectraLink, a maker of 802.11 handsets. We measured audio quality with up to seven concurrent calls, and in some events configured the VeriWave TestPoint boxes to offer background data. For each system tested, we checked call quality with QoS disabled, then enabled.

Results in BriefOver the next three days, we will feature detailed commentary on quality of service, roaming and architecture issues. Here is a brief summary of the results for the four systems we tested.

Aruba NetworksA2400, A800 switches, A61 access point . Overall rating 4.58Cost: $8,780 as tested.
Pros: Outstanding voice prioritisation capabilities; rich set of QoS and radio frequency management features.
Con: Some call drops in most stressful test case.

CiscoWLSM Overall rating 3.53Cost:$51,978 as tested.
Pros: Highly scalable, rich set of routing and switching functions.
Cons: Doesn’t protect voice traffic under most stressful test case; doesn’t dynamically adjust to changes in radio frequency environment; pricey.

Colubris NetworksCN1250 Overall rating 3.0Cost: $1,800 as tested; access point, $500.
Pro: Powerful and intuitive user interface.
Con: Limited prioritisation of voice traffic.

Chantry Networks (now part of Siemens)BeaconMaster Overall rating 2.4Cost: $9,180 as tested.
Pro: Supports Open Shortest Path First routing.
Cons: Dropped calls in six- and seven-call cases; poor voice quality in the presence of data; no contingency for the loss of power.

Conclusions: don't mix voice and data unless the system can handle itIt's possible that overall products may improve when more of them support the 802.11e QoS standard (read more here). All the vendors said it was early in the evolution of VOIP over wireless, and our test results show there is certainly room for improvement. For network managers looking to deploy VOIP on WLANs in the near future, there are three choices:

  • make very few calls;
  • don't ever send data;
  • or look for equipment - such as Aruba's - that handles time-sensitive traffic in a timely way.

Future articles:
Part 2: QoS matters Part 3: Roaming is an issue Part 4: Architectures are changing Part 5: How we did it