Brkewn 3012

Download as pdf or txt
Download as pdf or txt
You are on page 1of 85

Troubleshooting Voice over Wireless LAN Deployments

BRKEWN-3012
Patrick Croak ([email protected]) Wireless TAC Escalation CCIE Wireless #34712

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Agenda
Troubleshooting Voice over Wireless LAN Deployments

Voice over WLAN 101

VoWLAN RF Design
VoWLAN Configuration Identifying the Problem

Troubleshooting Tools
WLCCA WCS/NCS

Data Analysis Debug Analysis Summary


BRKEWN-3012 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 3

Agenda
What Will Not Be Covered

Call Manager configuration

Gateway issues and solutions

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Agenda
Troubleshooting Voice over Wireless LAN Deployments

Voice over WLAN 101


VoWLAN RF Design VoWLAN Configuration Identifying the Problem Troubleshooting Tools WLCCA WCS/NCS Data Analysis

Debug Analysis
Summary

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

VoWLAN 101
Key Concepts
Voice over WLAN would be similar to any other VoIP technology with the added issues of a wireless media
Signaling: SCCP/SIP Voice transport: RTP

Wireless adds some important differences


Media is shared Physical coverage is an issue

Security concerns
Battery life Roaming

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

VoWLAN 101
Wireless as Media
WiFi is unlicensed spectrum so has to operate on lower power

Coverage is lower than other radio technologies


As the transmit media is shared we can expect:

Interference (other WiFi) Noise Capacity issues


Access points have a limited area they can cover

Power restrictions Antenna used Physical environment

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

VoWLAN 101
Wireless as Media
Voice is one of the most critical applications to have over Wireless

Users have high expectations for voice, derived from GSM, DECT, and fixed line real-life experience
The main objective on a VoWLAN project, is to provide end users with a service level as close as possible to what they expect Wireless Networks are mostly designed for data services, so it is usually not possible to just drop voice on top, and expect any positive results

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

VoWLAN 101
Wireless as Media
Voice has very strict requirements as application

Packet Error Rate (PER) <=1% As low jitter as possible, less than 100ms Retries should be < 20% This translates to coverage needs

Normally data services can tolerate loss of connectivity or high packet loss. Users will not accept a clipping voice, or unidirectional voice flow. In general it is better to prevent a call, than to place a call over a congested media

This is where Call Admission Control takes place

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

VoWLAN 101
Roaming

Roaming means that the Phone has to find new AP before the current parent quality has gone below what is needed to maintain good voice
Has to be Secure Not too aggressive, but not conservative May use multiple triggers: Beacon, retries, packet loss, RSSI, SNR, QBSS

(792X

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

10

VoWLAN 101
Secure Roaming

How to accept a new client association quickly in a secure way? Each roaming may need full reauth
Key caching mechanisms are needed: CCKM, PMKID, Sticky roaming

A key caching will remove the need to complete 802.1x, which is slow Voice requirement: max 150ms of traffic drop, 300ms at most

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

11

VoWLAN 101
Association + 802.1x
Probe Request

AP

WLC

Radius

Probe Response
Auth Request Auth Response Association Request Association Response EAP Start EAP ID Request EAP ID Response EAP Method
Between 4 and 20+ frames

EAP Success EAPoL 4 way Exchange

DATA
BRKEWN-3012 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 12

VoWLAN 101
WPA(2)-PSK
Probe Request

AP

WLC

Radius

Probe Response
Auth Request Auth Response Association Request Association Response EAPoL 4 way Exchange

DATA

Hey looks better!


BRKEWN-3012 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 13

VoWLAN 101
CCKM
Probe Request

AP

WLC

Radius

Probe Response
Auth Request Auth Response Reassociation Request Reassociation Response DATA

Much better!

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

14

VoWLAN 101
Power Save
Devices are battery operated, so they must have power saving mechanisms
U-APSD

PSP/Legacy

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

15

VoWLAN 101
Top Ten Recommendations
1. 2. 3. 4. 5. 6. 7. 8. 9. QoS and Availability on your wired network is your foundation Security requirements for voice applications are different than from data Start with user consultation and education Address VoWLAN availability requirements in planning and design Maximize your WLAN Capacity by using the 5GHz spectrum Choose the right VoWLAN handset, based on user requirement and features, and availability in Cisco Compatible Extension Program Follow the VoWLAN handset guidance in planning and design Use Radio Resource Management for deployment, monitoring, and troubleshooting your WLAN Perform a post installation site-survey to confirm you have met your VoWLAN goals

10. Plan for the future and the addition of more services such as Location

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

16

Agenda
Troubleshooting Voice over Wireless LAN Deployments
Voice over WLAN 101

VoWLAN RF Design
VoWLAN Configuration Identifying the Problem Troubleshooting Tools WLCCA WCS/NCS Data Analysis

Debug Analysis
Summary

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

17

VoWLAN RF Design
What Should Be Covered
In order to determine if VoWLAN can be deployed, the environment must be evaluated to ensure the following items meet Cisco guidelines. Many different tools and applications can be used to evaluate these items in order to certify the deployment.

Signal Channel Utilization Noise Packet Loss / Delay Retries Multipath

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

18

VoWLAN RF Design
Coverage

The cell edge should be designed to -67 dBm, where there is a 20-30% overlap of adjacent access points at that signal level. This ensures the phone always has adequate signal and can roam seamlessly.

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

19

VoWLAN RF Design
Channel Utilization and Noise
Channel Utilization levels should be kept under 50%. If using the phone, this is provided via the QoS Basic Service Set (QBSS), which equates to a value around 105. Noise levels should not exceed -92 dBm, which allows for a Signal to Noise Ratio (SNR) of 25 dB where a -67 dBm signal should be maintained.

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

20

VoWLAN RF Design
What Should or Should Not Be Done

Highly reflective environments

Multipath distortion/fade is a consideration


802.11b most prone 802.11g/a better

Things that reflect RF:


Irregular metal surfaces Large glass enclosures/walls Lots of polished stone

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

21

VoWLAN RF Design
What Should or Should Not Be Done
Multipath. Multipath should be kept minimal as this can create nulls and reduce signal levels.

Temptation is to mount on beams or ceiling rails This reflects transmitted as well as received packets Dramatic reduction in SNR due to high-strength, multipath signals

Minimize Reflections When Choosing Locations


BRKEWN-3012 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 22

VoWLAN RF Design
What Should or Should Not Be Done

User
BRKEWN-3012

Mount the box and antennas downward Please


Cisco Public 23

2012 Cisco and/or its affiliates. All rights reserved.

VoWLAN RF Design
More Examples

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

24

VoWLAN RF Design
More Examples

Ceiling mount AP up against pipe


BRKEWN-3012

A little ICE to keep the packets cool


Cisco Public 25

2012 Cisco and/or its affiliates. All rights reserved.

VoWLAN RF Design
More Examples

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

26

VoWLAN RF Design
A Few Tips

Every site is unique, do not assume two installations would be the same

Think of the AP coverage area as a reading light: you want to illuminate where the devices will be. Avoid long run AP placement
Use the appropriate equipment for the need: 1130/1140/3500i/3600i for carpeted areas, 1240/1260/3500e/3600e for specific applications, antenna orientations Avoid using internal antennas AP in vertical placements. RF planning is more difficult

Validate that the coverage is as expected after installation

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

27

VoWLAN RF Design
A Few More Tips

Use 5 GHz whenever possible (a lot of smartphones are 2.4 GHz only)

Try to isolate sources of interference, rogues, etc, as part of initial survey


More needed in 2.4: clean air would provide continuous monitoring of possible common interferers For high ceiling: do not use omni antennas on high placed APs. Either move APs closer to clients, or use patch/directional Always do the design for APs at power level 3, so there is power budget available

Always allow diversity on AP, or MIMO if 11n

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

28

VoWLAN RF Design
Coverage Areas

A huge percentage of problems come from incorrectly defined coverage areas

Coverage areas: where the voice service should be offered


Typical errors: not needed in the bathrooms, not in the elevators, not in the stairs, not in the outdoor smoking area Talk to end-users. Think what they will need and when

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

29

VoWLAN RF Design
The Transition AP

Cell overlap coverage is not always the only concern

Roaming can fail if the client device does not have enough time to properly scan for neighboring access points
Imagine turning the corner around a metal or high attenuation barrier the RF environment changes very rapidly

Challenging RF obstacles need to be considered during AP placement


A Transition AP that is placed at the intersection of hallways can alleviate some scenarios

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

30

VoWLAN RF Design
Scanning Problems At point A the phone is connected to AP 1

B C 3

At point B the phone has AP 2 in the neighbor list, AP 3 has not yet been scanned due to the RF shadow caused by the elevator bank At point C the phone needs to roam, but AP 2 is the only AP in the neighbor list
The phone then needs to rescan and connect to AP 3

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

31

VoWLAN RF Design
Transition AP Placement

1 A B 2

At point A the phone is connected to AP 1

At point B the phone has AP 2 in the neighbor list as it was able to scan it while moving down the hall
At point C the phone needs to roam and successfully selects AP 2 The phone has sufficient time to scan for AP 3 ahead of time

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

32

VoWLAN RF Design
Avoid Pico-cells When Possible

Pico-cells: signal fade too quickly, not giving time for device to do a controlled roam 792x 1.4.2 firmware helps here. CCKM timestamp of 5 may alleviate a bit.
config wlan security wpa akm cckm timestamp-tolerance 5000 <WLAN ID>

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

33

Agenda
Troubleshooting Voice over Wireless LAN Deployments
Voice over WLAN 101 VoWLAN RF Design

VoWLAN Configuration
Identifying the Problem Troubleshooting Tools WLCCA WCS/NCS Data Analysis

Debug Analysis
Summary

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

34

VoWLAN Configuration
Use Design Guide

Disabled
DHCP required P2P blocking MFP client Band select Load balancing Low data rates

Enabled
- Aironet extensions

- DTPC is enabled
- Platinum + 802.1p 6 - Long session timeout - Fast roaming (CCKM/Open/PSK) - WMM (optional/required) - DTIM 2 - AES - EDCA for Voice or mixed

- CAC

Optional
Client Exclusion
BRKEWN-3012 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 35

VoWLAN Configuration
RRM Advanced Settings

Use long DCA period: 8/12/24 hours to prevent frequent channel changes

Set the Maximum Power Level to match your clients


Set the Minimum Power Level to avoid pico-cell issues Power Threshold can be increased to increase overall power assignments, or decreased to reduce power assignments
Default value is -70, remember it is a negative number!

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

36

VoWLAN Configuration
Channel and Power Levels

UNII-1 Channels are intended for indoor use, typically have lower max transmit power of 14 dB* UNII-2 Channels require use of DFS (Dynamic Frequency Selection), typically have max transmit power of 17 dB* UNII-3 Channels do not require DFS, and have a max transmit power of 17 dB* *Maximum transmit power may vary by AP model

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

37

VoWLAN Configuration
DSCP and COS

Trust DSCP on APs

Trust COS on WLC trunks


Note that COS value requires 802.1q tag, so it will not work for the native vlan!

Set 802.1p Wired Protocol to 6 for the Platinum QoS Profile

Ensure cos-dscp map on switched network is properly defined


- COS 3 to DSCP 24 (CS3) for SCCP traffic - COS 5 to DSCP 46 (EF) for RTP traffic mls qos map cos-dscp 0 8 16 24 32 46 48 56

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

38

VoWLAN Configuration
DTPC

Mismatched transmit powers on AP and client can cause one-way audio and poor performance 792x phone has max tx power of 40 mW (16 dBm) AP 2.4 GHz can be up to 100 mW (20 dBm) AP 5.0 GHz can be up to 50 mW (17 dBm) (varies by channel) Other benefits include reduced co-channel interference radius and power saving on clients Requires CCXv2

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

39

Agenda
Troubleshooting Voice over Wireless LAN Deployments
Voice over WLAN 101 VoWLAN RF Design VoWLAN Configuration

Identifying the Problem


Troubleshooting Tools WLCCA WCS/NCS Data Analysis

Debug Analysis
Summary

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

40

Identifying the Problem


Troubleshooting 101

Avoid jumping into conclusions

Troubleshooting is a process
Do NOT change things without understanding root cause first Find patterns: where, when, how, the problem is reproducible.

Random problems are the worst to troubleshoot

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

41

Identifying the Problem


Proper Definition is Important

It is critical that the proper problem is described


One way voice Two way failure (no voice) Robotic or choppy voice quality Network Busy Out of coverage Length of failure Location when failure occurred

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

42

Identifying the Problem


Example Scenarios

One way voice


Only one side can hear the other Typical Triggers
Asymmetrical RF (different power levels for example, antenna types) Failed ARP resolution

No way voice
No voice in both directions Typical Triggers
Very bad RF Failed Roaming

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

43

Identifying the Problem


Example Scenarios

Robotic/Choppy Voice
High packet drops affecting voice quality Typical Triggers
Bad RF/Coverage: lots of retries, low signal levels, no good candidate to roam Bad Roaming decision: phone may be too conservative Interference/Noise QoS configuration

Network Busy
CAC rejected call, high QBSS Typical Triggers
High client count High energy on channel/Interference/Noise

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

44

Identifying the Problem


Example Scenarios

Out of Coverage/Leaving service area


Phone reports no wireless network Typical Triggers
Bad RF/Coverage Bugs (either phone side, AP side for radio lockups)

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

45

Identifying the Problem


Isolate the Issue

Is the problem affecting only one area? All the building?

Does it happen while walking from X to Y?


Does it occur while stationary at the desk? Happens for all phones? Specific phones? Specific users?

Any time pattern? All day long? Lunch time?


For pervasive issues, work on contained areas: one floor, one wing, etc. There are no real random problems: a pattern must exist, and that normally points to root cause

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

46

Agenda
Troubleshooting Voice over Wireless LAN Deployments
Voice over WLAN 101 VoWLAN RF Design VoWLAN Configuration Identifying the Problem

Troubleshooting Tools
WLCCA WCS/NCS Data Analysis

Debug Analysis
Summary

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

47

Troubleshooting Tools
Wireless Captures, RF Analysis, Configuration Analysis

Wireless sniffer
Omnipeek/AirPcap Mac with OS X 10.6 and above Windows 7 with Netmon 3.4 Multichannel, for roaming issues AP in Sniffer mode

L1 analysis: SpectrumExpert, 3500/3600 Ap, etc WLCCA (WLC Configuration Analyzer)

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

48

Troubleshooting Tools
Debugs, Logs, Reports

WLC debug client


Single client on 7.0, Two clients on 7.2 Other debugs may be needed on AP side for radio issues -> only with TAC support

Phone debug
Do not overuse, as it will disrupt voice

WCS/NCS
Roaming history SNR levels TSM

Time sync is very helpful!


BRKEWN-3012 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 49

Troubleshooting Tools
Data Capturing

Wireless Capture
USB logs Time sync Wireless capture done at phone side. Multichannel if roaming issue is suspected Call placed to fixed phone to isolate one wireless path only
BRKEWN-3012 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 50

Wired Capture

Troubleshooting Tools
Steps to Resolution

Get proper problem description and area isolation

Check your config: WLCCA


Check RF groups, RF neighborhood: WLCCA Check Historical data, SNR/RSSI, TSM: WCS/NCS

Check Coverage: survey data, WCS/NCS map (if correctly done)


If suspecting a RF issue, get pictures, or go onsite to see the deployment Try to know how to reproduce

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

51

Agenda
Troubleshooting Voice over Wireless LAN Deployments
Voice over WLAN 101 VoWLAN RF Design VoWLAN Configuration Identifying the Problem Troubleshooting Tools

WLCCA
WCS/NCS Data Analysis

Debug Analysis
Summary

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

52

WLCCA
Direct Troubleshooting

Basic things first

Bad!

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

53

WLCCA
RF First View

Power levels and channel distribution: looking for anomalies

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

54

WLCCA
RF First View

Power levels and channel distribution: looking for anomalies

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

55

WLCCA
RF Groups

May indicate bad coverage. Only relevant for APs on same physical area

Bad!

Good

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

56

WLCCA
RF Problem Finder

Quick isolation of suspicious areas

No hard cut number on what is bad

Why?

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

57

WLCCA
AP RF Summary

Lots of information

Neighbor relationships, co-channel interference, isolated APs

Bad!

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

58

WLCCA
RF Neighbors

In depth analysis on how the APs see each other

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

59

Agenda
Troubleshooting Voice over Wireless LAN Deployments
Voice over WLAN 101 VoWLAN RF Design VoWLAN Configuration Identifying the Problem Troubleshooting Tools WLCCA

WCS/NCS
Data Analysis

Debug Analysis
Summary

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

60

WCS/NCS
History to the Rescue

WCS/NCS can show you where, and how the client has been over time

Bad!

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

61

WCS/NCS
History to the Rescue

TSM reports can have valuable information

Bad!

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

62

Agenda
Troubleshooting Voice over Wireless LAN Deployments
Voice over WLAN 101 VoWLAN RF Design VoWLAN Configuration Identifying the Problem Troubleshooting Tools WLCCA WCS/NCS

Data Analysis
Debug Analysis
Summary

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

63

Data Analysis
Needle in a Haystack?

The problem description normally indicates what to look for

RTP analysis can help a lot


PSK can be decoded if EAPoL 4 way is captured + shared key is available

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

64

Data Analysis
Filters

Learn how to filter properly

Bad Good

Bad

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

65

Data Analysis
Filters

Filter + IO Graph=Eagle view on activity

Why?

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

66

Data Analysis
One Way Voice

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

67

Data Analysis
Robotic/Choppy Voice

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

68

Data Analysis
Slow Roaming

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

69

Data Analysis
QoS Verification

On the 792x phone, while on a call navigate to Settings > Status > Network Statistics Check that the DataRcvVO counter is incrementing

Packet Capture

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

70

Data Analysis
Packet Loss and Delay
Per voice guidelines, PER should not exceed 1% packet loss. If there is > 1% packet loss, then voice quality can be degraded significantly. All Cisco IP Phones have the ability to display receiver lost packets as well as the total # of receiver packets in the stream (call) statistics. Simply divide the receiver lost packets by the total # of receiver packets. Jitter should also be kept at a minimum (< 100 ms).

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

71

Agenda
Troubleshooting Voice over Wireless LAN Deployments
Voice over WLAN 101 VoWLAN RF Design VoWLAN Configuration Identifying the Problem Troubleshooting Tools WLCCA WCS/NCS Data Analysis

Debug Analysis
Summary

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

72

Debug Analysis
Few Pointers

Understanding debug client, doc ID: 100260 http://www.cisco.com/en/US/products/h w/wireless/ps430/products_tech_note09 186a008091b08b.shtml

On mobility scenarios (multiple controllers), always debug client + debug mobility on all WLC where client may roam to
The controller will give a view of what is going on, and can close a lot the spectrum of issues to investigate
BRKEWN-3012 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 73

Debug Analysis
WLC Client Debug

Most common tool used by TAC


debug client xx:xx:xx:xx:xx:xx

Proper roams will be Reassociations, if you see Associations then there are roaming failures that should be investigated You can trace the roaming path of the phone by looking up the AP radio mac addresses at each (re)association *apfMsConnTask_0: Apr 13 16:10:27.014: cc:08:e0:2e:10:2b Association received from mobile on AP b4:a4:e3:b5:bc:60

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

74

Debug Analysis
792x Phone Debugs

Set phone trace logs from Error to Info (or Debug, but this can impact performance of the phone)
Kernel WLAN Driver WLAN Manager

The phone log will show the neighbor list and roaming trigger for each roam
WLAN_DRVR: 3845.629757: Roam trigger = ROAMING_TRIGGER_MAX_TX_RETRIES WLAN_DRVR: 3845.643734: Candidate 0, BSSID=c4:7d:4f:3b:02:e2, RSSI =-60 WLAN_DRVR: 3845.650550: Candidate 1, BSSID=00:19:30:76:52:dc, RSSI =-63 WLAN_DRVR: 3845.657382: Candidate 2, BSSID=00:19:56:b0:79:f0, RSSI =-72 WLAN_DRVR: 3845.664203: Candidate 3, BSSID=1c:df:0f:b5:47:b2, RSSI =-74

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

75

Debug Analysis
Some Examples

Idle timeout from AP, shortly after roam, CSCto73361 fixed in 7.0.220.0
*pemReceiveTask: Nov 23 15:34:44.946: 00:15:f9:93:42:9e 192.168.32.32 Added NPU entry of type 1, dtlFlags 0x0 *spamApTask0: Nov 23 15:34:44.948: 00:15:f9:93:42:9e Received Idle-Timeout from AP 00:24:14:31:ce:70, slot 0 for STA 00:15:f9:93:42:9e *spamApTask0: Nov 23 15:34:44.948: 00:15:f9:93:42:9e apfMsDeleteByMscb Scheduling mobile for deletion with deleteReason 4, reasonCode 4

Bad PSK configuration, also shows in logs.


*dot1xMsgTask: Sep 06 18:37:11.341: 00:0b:6b:b3:d6:e7 Sending EAPOL-Key Messageto mobile 00:0b:6b:b3:d6:e7 state INITPMK (message 1), replay counter 00.00.00.00.00.00.00.00 *Dot1x_NW_MsgTask_7: Sep 06 18:37:11.378: 00:0b:6b:b3:d6:e7 Received EAPOL-Key from mobile 00:0b:6b:b3:d6:e7 *Dot1x_NW_MsgTask_7: Sep 06 18:37:11.378: 00:0b:6b:b3:d6:e7 Received EAPOL-key in PTK_START state (message 2) from mobile 00:0b:6b:b3:d6:e7 *Dot1x_NW_MsgTask_7: Sep 06 18:37:11.378: 00:0b:6b:b3:d6:e7 Received EAPOL-key M2 with invalid MIC from mobile 00:0b:6b:b3:d6:e7

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

76

Debug Analysis
Some Examples

CCKM time validation errors


*apfMsConnTask_5: *apfMsConnTask_5: *apfMsConnTask_5: *apfMsConnTask_5: *apfMsConnTask_5: Apr Apr Apr Apr Apr 04 04 04 04 04 12:55:54.402: 12:55:54.402: 12:55:54.402: 12:55:54.402: 12:55:54.402: 00:01:e3:bb:e5:23 00:01:e3:bb:e5:23 00:01:e3:bb:e5:23 00:01:e3:bb:e5:23 00:01:e3:bb:e5:23 Processing WPA IE type 221, length 22 for mobile 00:01:e3:bb:e5:23 CCKM: Mobile is using CCKM CCKM: Processing REASSOC REQ IE CCKM: Received Timestamp deviation in REASSOC REQ IE from mobile CCKM: Failed to validate REASSOC REQ IE

In logs
*apfMsConnTask_5: Mar 22 14:49:36.109: %APF-3-VALIDATE_CCKM_REASS_REQ_ELEMENT: apf_ut:2122 Could not validate the CCKM Reassociation request element.Received Timestamp deviation > 1sec in CCKM Info Element from mobile. Mobile:00:01:e3:bb:e5:23,

Either pico-cells, or roaming issues Use: config


wlan security wpa akm cckm timestamp-tolerance 5000 X

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

77

Debug Analysis
EAPoL Key Failure
*Dot1x_NW_MsgTask_2: Aug 30 16:39:06.201: 00:1e:4a:3f:af:4f Sending EAPOL-Key Message to mobile 00:1e:4a:3f:af:4f state PTKINITNEGOTIATING (message 3), replay counter 00.00.00.00.00.00.00.02

*osapiBsnTimer: Aug 30 16:39:06.801: 00:1e:4a:3f:af:4f 802.1x 'timeoutEvt' Timer expired for station 00:1e:4a:3f:af:4f and for message = M3
*dot1xMsgTask: Aug 30 16:39:06.801: 00:1e:4a:3f:af:4f Retransmit 1 of EAPOL-Key M3 (length 155) for mobile 00:1e:4a:3f:af:4f *osapiBsnTimer: Aug 30 16:39:07.201: 00:1e:4a:3f:af:4f 802.1x 'timeoutEvt' Timer expired for station 00:1e:4a:3f:af:4f and for message = M3

*dot1xMsgTask: Aug 30 16:39:07.201: 00:1e:4a:3f:af:4f Retransmit 2 of EAPOL-Key M3 (length 155) for mobile 00:1e:4a:3f:af:4f
*osapiBsnTimer: Aug 30 16:39:07.601: 00:1e:4a:3f:af:4f 802.1x 'timeoutEvt' Timer expired for station 00:1e:4a:3f:af:4f and for message = M3 *dot1xMsgTask: Aug 30 16:39:07.601: 00:1e:4a:3f:af:4f Retransmit failure for EAPOL-Key M3 to mobile 00:1e:4a:3f:af:4f, retransmit count 3, mscb deauth count 0

*dot1xMsgTask: Aug 30 16:39:07.601: 00:1e:4a:3f:af:4f Sent Deauthenticate to mobile on BSSID c4:7d:4f:3b:02:e0 slot 1(caller 1x_ptsm.c:534)

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

78

Debug Analysis
Mitigating EAPoL Failure Impact

config advanced eap eapol-key-timeout <200-5000 ms>


Default is 1 second (1000 ms) Recommendation for 792x Phones is 200 ms

config advanced eap eapol-key-retries <0-4>


Default is 2 retries Lowering this value may reduce audio loss on roaming failure

show advanced eap to verify settings

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

79

Agenda
Troubleshooting Voice over Wireless LAN Deployments
Voice over WLAN 101 VoWLAN RF Design VoWLAN Configuration Identifying the Problem Troubleshooting Tools WLCCA WCS/NCS Data Analysis

Debug Analysis

Summary

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

80

Summary
Key Take Aways

RF: survey, survey, survey

Proper problem description, area isolation


Check basic things first Data capturing on site can be difficult, and you need the proper tools

You cant just drop voice on a WiFi deployment and expect it to work

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

81

Recommended Reading

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

82

Complete Your Online Session Evaluation


Give us your feedback and you could win fabulous prizes. Winners announced daily. Receive 20 Passport points for each session evaluation you complete.

Complete your session evaluation online now (open a browser through our wireless network to access our Dont forget to activate your Cisco Live Virtual account for access to portal) or visit one of the Internet stations throughout the Convention all session material, communities, and on-demand and live activities throughout Center. the year. Activate your account at the
Cisco booth in the World of Solutions or visit www.ciscolive.com.
BRKEWN-3012 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 83

Final Thoughts
Get hands-on experience with the Walk-in Labs located in World of Solutions, booth 1042 Come see demos of many key solutions and products in the main Cisco booth 2924 Visit www.ciscoLive365.com after the event for updated PDFs, ondemand session videos, networking, and more! Follow Cisco Live! using social media:
Facebook: https://www.facebook.com/ciscoliveus

Twitter: https://twitter.com/#!/CiscoLive
LinkedIn Group: http://linkd.in/CiscoLI

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

84

BRKEWN-3012

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Public

You might also like