Buyers of Wi-Fi equipment and systems must be assured that all products have the performance and stability to carry mission-critical applications and data. However, testing of Wi-Fi, or 802.11, devices and systems for performance and stability is a challenge for the industry because of the complexity of the 802.11 protocol. That is compounded by the inherent mobility of the wireless devices and the prevalence of radio frequency interference.
In July 2004, the IEEE formed the IEEE 802.11T Task Group to develop a test specification document, "Recommended Practice for the Evaluation of 802.11 Wireless Performance," expected to be completed in January 2008 (the group had started to meet earlier in 2004). By forming the task group, the IEEE has acknowledged the need to provide users with an objective means of evaluating functionality and performance of 802.11 products (early in the group's career, Techworld spoke to test expert Bob Mandeville about prospects for Wi-Fi testing.
The 802.11T document defines test metrics in the context of use cases. The three principal-use cases are data, latency-sensitive and streaming media.
Data applications do not impose critical requirements on a network and include Web downloads, file transfers, file sharing, e-mail and others. Data-oriented traffic is typically transmitted using low priority. Performance test metrics important for data use include
- throughput vs. range,
- access-point capacity and
- access-point throughput per client.
Latency-sensitive applications are time-critical, such as VOIP over Wi-Fi. QoS requirements for these applications include
- limits on voice quality (latency, jitter and packet loss) vs. range,
- voice quality vs. network load,
- voice quality vs. call load and
- Basic Service Set (BSS).
BSS is a single access-point network, similar to a single cell in the cellular environment. BSS transition is the process of a mobile station roaming from one access point to another.
Streaming-media applications include real-time audio/video streaming, stored content streaming and multicast high-definition television streaming. These applications require the most stringent QoS, including bandwidth and latency guarantees. Performance metrics include video
- quality (throughput, latency, jitter) vs. range and
- video quality vs. network load.
The metrics are classified as primary and secondary. Primary metrics directly affect the user experience, such as voice quality. Secondary metrics affect the primary metrics; for example latency, jitter and packet loss affect voice quality.
In addition to the metrics previously cited, the current version of 802.11T specifies such metrics as
- throughput vs. path loss,
- fast BSS transition,
- receiver sensitivity,
- access-point capacity and
- association performance.
802.11T defines two test environments - conducted and over-the-air. Most of the tests in the draft require a conducted environment for emulation of motion and for measurement repeatability.
Conducted environments provide RF isolation and emulate controlled motion so that mobile devices and systems can be tested in a repeatable and prescribed manner. In a conducted environment, each device in the test setup is placed in a shielded chamber for isolation. RF cables connect the antenna ports of each device to other devices through programmable attenuators. The attenuators emulate distance by virtue of controlling the path loss among devices in the test setup. Shielding and filtering are employed to protect the test setup from outside interference and to achieve device-to-device isolation.
Device isolation eliminates signal paths other than through the attenuators. Isolation between any two devices in a test setup must be greater than 110 decibels, because of the wide dynamic range of 802.11. Such isolation is difficult to achieve, particularly in the 5GHz band.
Without proper testing, wireless equipment and networks cannot be relied on to deliver the required throughput, client capacity, security or fault tolerance for acceptable enterprise network performance. The goal of 802.11T is to enable testing, comparison and deployment planning of 802.11 wireless devices based on a common and accepted set of performance metrics, measurement methodologies and test conditions. 802.11T will help ensure that 802.11 products meet the challenges and demands of enterprise networks, and will help IT managers choose the fastest and most robust 802.11 products that are resilient to adverse network conditions.
Mlinarsky is founder and CTO of Azimuth Systems, which has been selling Wi-Fi test systems for two and a half years. This article first appeared in Network World.