Metrics for Characterizing BSS Transition Time Performance

1. Metrics for CharacterizingBSS Transition Time Performance Date: September 7, 2004 Authors: Charles R. Wright, Azimuth Systems, Inc., Acton, MA charles_wright@azimuthsystems.com Chris Polanec, InterOperability Lab, Durham, NH cpolanec@iol.unh.edu C. Wright, Azimuth Systems

2. Abstract TGr is currently planning a selection process for candidate fast BSS transition protocols. This presentation proposes a metric and test methodology for characterizing the performance of real equipment using such a transition protocol. It is based on repeated measurements of a refined definition of BSS transition time as defined in 11-04/805r2. It produces a statistical characterization useful to system designers as well as for comparison of candidate fast BSS transition protocols. C. Wright, Azimuth Systems

3. Outline • Background and definitions • The metrics • Test configuration • Other relevant issues • Conclusions C. Wright, Azimuth Systems

4. Background • Held three ad hoc telecons between Portland and Berlin meetings • Discussed various issues with measuring transition time • This presentation represents a synthesis of the outcomes from these discussions on the issues • Thanks to the other ad hoc attendees: • Mike Montemurro, Fanny Mlinarsky , Chris Polanec, Jeremy Spilman, Clint Chaplin C. Wright, Azimuth Systems

5. Fast BSS Transition Time Definition • From 11-04/805r2, 1.5.4, we have “Fast-BSS-Transition-Time”: • Security not enabled: • the period between the last possible point in time (tA) where STA-TE communication can pass through the Old-AP, to the first possible point in time (tB) where STA-TE communication can pass through the New-AP • Security enabled: • the period between the last possible point in time (tA) where STA-TE communication can pass through the Old-AP, to the point in time (tB) where the first MSDU can pass through the controlled port • TE = “Traffic Endpoint” C. Wright, Azimuth Systems

6. As defined, Fast BSS Transition Time is useful for analysis, but is not measurable • The “last possible point in time where STA-TE communication can pass through the old AP” is not externally observable • Also applies to “first possible point in time where STA-TE communication can pass through the new AP” • What we can observe is the last MSDU transmitted over the air with apparent success Last possible instant where frames can pass through old AP First possible instant where frames can pass through new AP “Fast BSS Transition Time” . . . . . . MSDU ACK MSDU ACK Old AP New AP C. Wright, Azimuth Systems

7. We propose new transition time definitions • “Fast BSS Transition Time” as defined in 11-04/805r2, 1.5.4 is really the “Theoretical BSS Transition Time” • What is measured using the last successful MSDU on the first channel and the first successful MSDU on the new channel is: “Observed BSS Transition Time” Last possible instant where frames can pass through old AP First possible instant where frames can pass through new AP Theoretical BSS Transition Time . . . . . . MSDU ACK MSDU ACK Observed BSS Transition Time C. Wright, Azimuth Systems

8. BSS Transition Time Metrics • Two forms of this metric, depending on the capabilities of the client • Minimum BSS Transition Time • Is a single value • Can be measured if client can generate arbitrary traffic rates • Observed BSS Transition Time Distribution • Produces a cumulative distribution for a specified frame rate • Can be measured with any client C. Wright, Azimuth Systems

9. Measuring Minimum BSS Transition Time • Send periodic traffic and measure the transition time • Traffic interval should be greater than the expected transition time • Measure transition time “many times”; note minimum value • Decrease the frame interval and do this again • Repeat until observed transition time no longer decreases with decreased frame interval • Minimum value of transition time for this frame interval is estimate of Theoretical BSS Transition Time Frame Interval Transition Period . . . . . . MSDU ACK MSDU ACK Old AP New AP C. Wright, Azimuth Systems

10. Fixed frame rate traffic is very common • Wireless voice handsets are likely the “killer app” for TGr, at least right now • Voice traffic is periodic and fairly low rate (10 to 30 ms frame interval) • Voice handsets have no real need to vary frame rate • Minimum transition time might vary with traffic load • Under high loads, may be difficult for client to scan other channels • Need a metric that is meaningful for fixed frame rate traffic • Means metric can be targeted for a known and specific application C. Wright, Azimuth Systems

11. Observed BSS Transition Time metric is inherently statistical • Transition may involve processes that take variable time • STA/AP may introduce jitter in the target frame interval • STA may buffer frames during transition and not emit traffic at a periodic interval • STA may decide to roam at any time instant between transmitted frames Observed BSS Transition Time . . . Old AP Transition Period New AP Observed BSS Transition Time . . . Old AP Transition Period New AP • Need to measure transition time repeatedly to characterize performance C. Wright, Azimuth Systems

12. Observed BSS Transition Time Distribution • Choose or record the frame rate (frame interval) • Measure and record transition time for many trials • Display results as a cumulative distribution • Read result as (e.g.) • “64% of the time, the transition time for Proposal 2 is less than 40 ms”, or • “36% of the time, the transition time for Proposal 2 is more than 40 ms” Frame Interval: 20 ms *** Completely fictitious data! *** C. Wright, Azimuth Systems

13. Why a Cumulative Distribution of the Transition Time? • Considered counting packets lost during transition, but • Requires intimate knowledge of the protocol to count lost packets • On uplinks, STAs might decide to buffer traffic queued during a transition, so frame(s) might not be lost, only delayed • What really matters depends on the application, anyway • Some codecs can cover frame loss better than others • Some devices may be smart about buffering during transitions • CDF of Observed BSS Transition Time allows system designers to use this measure of network interruption time and full knowledge of their own devices to calculate outage rates C. Wright, Azimuth Systems

14. Cumulative distribution is useful for system engineering • For example, assume G.711 (20 ms) traffic and a 1 packet-deep jitter buffer • If transition takes longer than time it takes to receive 2 packets, then a codec underrun occurs • Cumulative distribution can answer these questions: • What is the probability that transition causes the jitter buffer to empty and we underrun for one frame? • What is the probability we underrun for 2 frames? Etc. C. Wright, Azimuth Systems

15. Example from Current Technology 40 Transitions Mix of 11a/b/g clients with various APs C. Wright, Azimuth Systems

16. Test Configuration C. Wright, Azimuth Systems

17. AP1 AP2 Test Configuration Ethernet Switch Ethernet Sniffer; Traffic Endpoint Sniffer Sniffer Atten2a Atten1a Combiner Combiner Atten1b Atten2b Placement of wireless sniffers in the RF network allows reception of AP and client no matter what the attenuator settings are Combiner Test head with roaming client STA C. Wright, Azimuth Systems

18. Some Requirements • Design test so that is possible using passive observation (airlink and ethernet) • Test run with under optimal channel conditions and no background load • Sniffers must be time synchronized • Various ways to do this • No more than one Ethernet switch on the wired backbone to minimize any switching delays C. Wright, Azimuth Systems

19. Dwell Interval AP2 max Atten AP1 min Attenuator Sweep Time Client Association: AP1 AP2 Attenuator Sweep Profile • Attenuators start off such that client associates with AP1 • Atten1a = Atten1b = minimum; Atten2a = Atten2b = maximum; • After dwell interval, attenuators sweep to make AP2 more favorable for client • Atten1a, Atten1b both increase at same rate; Atten2a, Atten2b both decrease at same rate • Eventually, client transitions to AP2 • After attenuators reach max/min and another dwell interval, process reverses to “push” client back to AP1 Atten2a+Atten2b Atten1a+Atten1b C. Wright, Azimuth Systems

20. Attenuator Considerations • Total of four programmable attenuators are recommended • Idea is to make sure airlink sniffer can always hear both the roaming client and the AP • Can probably get by with only two… • Replace Atten1a and Atten2a with a fixed attenuator, or Atten1b and Atten2b • … but using fixed attenuators can make it more difficult to cause the roaming client to go completely out of range of the AP • Designer of test system has to “add up the dBs” to make sure it will work out! C. Wright, Azimuth Systems

21. Other Issues • Uplink, downlink or bidirectional test traffic? • Speed of attenuator variation? • Power save • Issues with QoS and Security features • Architecture of network of APs C. Wright, Azimuth Systems

22. Uplink, Downlink or Bidirectional? • Should transition time metric be measured separately for each of the above traffic orientations? • Because of wireless voice handsets, bidirectional performance is most interesting • Observed BSS Transition time should be measured simultaneously for up and downlink frames • Results are reported separately for up and down link . . . . . . Up Up Up Up . . . . . . Down Down Down Down . . . . . . Up Down Up Down Up Down Up Down C. Wright, Azimuth Systems

23. Speed of Attenuator Variation • Want to measure only the BSS transition time • Do not want measurement influenced by scanning, pre-auth, neighborhood lists, etc. • These factors are out of scope for TGr • Decrease in signal level only serves to stimulate BSS transition • There may be other reasons to transition, but again, we’re interested only in the observed BSS transition time • Leads to question for test: precisely how long to dwell with one AP before stimulating the transition? • With current 802.11 equipment, about 10 sec seems to work, assuming also the attenuators change at ½ dB per second C. Wright, Azimuth Systems

24. What about Power Save modes? • Power save mechanisms • Legacy 802.11 • 802.11e Scheduled APSD • 802.11e Triggered APSD • In the ad hoc telecons, we could determine no issues where power save would meaningfully affect the transition time metric • Fast transition matters for stations sending time-critical traffic • Immediately roaming after coming out of dormancy does not seem to be time critical C. Wright, Azimuth Systems

25. Architecture of APs • Thin AP architectures may interfere with verification of network connectivity C. Wright, Azimuth Systems

26. Issues with QoS and Security Features • 802.11e NoAck policy will cause problems with knowing whether there is actual network connectivity • Uplink traffic work-around: observe data frames on the wired network • Downlink traffic work-around: none – we can only assume the STA received the frame if it was transmitted by the AP • We don’t know how popular the NoAck policy will become • For 802.11i, we must assume that after a transition, the PAE on a receiving device will open a controlled port for data frames • Frames are ACK’d but may not pass through if the PAE blocks them • Uplink traffic work-around: must confirm successful transmission by observing Ethernet frames • Downlink traffic work-around: none – same issue as with NoAck • We expect security will be very popular C. Wright, Azimuth Systems

27. Conclusions • Defined the test setup and methodology • Defined two new terms for BSS transition times • Theoretical BSS Transition Time • Observed BSS Transition Time • Defined two new metrics • Minimum BSS Transition Time (estimate of Theoretical) • Observed BSS Transition Time Distribution • Other airlink events can be measured with the test setup, but are not part of the metrics • tretry, tscan, tassociate, tdata (11-04/748r1) C. Wright, Azimuth Systems

28. References • 11-04/0086r3: Measurement of 802.11 Roaming Intervals • 11-04/0748r1: Test Methodology for Measuring BSS Transition Time • 11-04/xxxr0: BSS Transition Time Test Methodology • 11-04/805r2: 802.11 TGr Minimum Requirements • RFC 2285: Benchmarking Terminology for LAN Switching Devices C. Wright, Azimuth Systems