Brkewn 3012
Brkewn 3012
Brkewn 3012
BRKEWN-3012
Patrick Croak ([email protected]) Wireless TAC Escalation CCIE Wireless #34712
BRKEWN-3012
Cisco Public
Agenda
Troubleshooting Voice over Wireless LAN Deployments
VoWLAN RF Design
VoWLAN Configuration Identifying the Problem
Troubleshooting Tools
WLCCA WCS/NCS
Agenda
What Will Not Be Covered
BRKEWN-3012
Cisco Public
Agenda
Troubleshooting Voice over Wireless LAN Deployments
Debug Analysis
Summary
BRKEWN-3012
Cisco Public
VoWLAN 101
Key Concepts
Voice over WLAN would be similar to any other VoIP technology with the added issues of a wireless media
Signaling: SCCP/SIP Voice transport: RTP
Security concerns
Battery life Roaming
BRKEWN-3012
Cisco Public
VoWLAN 101
Wireless as Media
WiFi is unlicensed spectrum so has to operate on lower power
BRKEWN-3012
Cisco Public
VoWLAN 101
Wireless as Media
Voice is one of the most critical applications to have over Wireless
Users have high expectations for voice, derived from GSM, DECT, and fixed line real-life experience
The main objective on a VoWLAN project, is to provide end users with a service level as close as possible to what they expect Wireless Networks are mostly designed for data services, so it is usually not possible to just drop voice on top, and expect any positive results
BRKEWN-3012
Cisco Public
VoWLAN 101
Wireless as Media
Voice has very strict requirements as application
Packet Error Rate (PER) <=1% As low jitter as possible, less than 100ms Retries should be < 20% This translates to coverage needs
Normally data services can tolerate loss of connectivity or high packet loss. Users will not accept a clipping voice, or unidirectional voice flow. In general it is better to prevent a call, than to place a call over a congested media
BRKEWN-3012
Cisco Public
VoWLAN 101
Roaming
Roaming means that the Phone has to find new AP before the current parent quality has gone below what is needed to maintain good voice
Has to be Secure Not too aggressive, but not conservative May use multiple triggers: Beacon, retries, packet loss, RSSI, SNR, QBSS
(792X
BRKEWN-3012
Cisco Public
10
VoWLAN 101
Secure Roaming
How to accept a new client association quickly in a secure way? Each roaming may need full reauth
Key caching mechanisms are needed: CCKM, PMKID, Sticky roaming
A key caching will remove the need to complete 802.1x, which is slow Voice requirement: max 150ms of traffic drop, 300ms at most
BRKEWN-3012
Cisco Public
11
VoWLAN 101
Association + 802.1x
Probe Request
AP
WLC
Radius
Probe Response
Auth Request Auth Response Association Request Association Response EAP Start EAP ID Request EAP ID Response EAP Method
Between 4 and 20+ frames
DATA
BRKEWN-3012 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
VoWLAN 101
WPA(2)-PSK
Probe Request
AP
WLC
Radius
Probe Response
Auth Request Auth Response Association Request Association Response EAPoL 4 way Exchange
DATA
VoWLAN 101
CCKM
Probe Request
AP
WLC
Radius
Probe Response
Auth Request Auth Response Reassociation Request Reassociation Response DATA
Much better!
BRKEWN-3012
Cisco Public
14
VoWLAN 101
Power Save
Devices are battery operated, so they must have power saving mechanisms
U-APSD
PSP/Legacy
BRKEWN-3012
Cisco Public
15
VoWLAN 101
Top Ten Recommendations
1. 2. 3. 4. 5. 6. 7. 8. 9. QoS and Availability on your wired network is your foundation Security requirements for voice applications are different than from data Start with user consultation and education Address VoWLAN availability requirements in planning and design Maximize your WLAN Capacity by using the 5GHz spectrum Choose the right VoWLAN handset, based on user requirement and features, and availability in Cisco Compatible Extension Program Follow the VoWLAN handset guidance in planning and design Use Radio Resource Management for deployment, monitoring, and troubleshooting your WLAN Perform a post installation site-survey to confirm you have met your VoWLAN goals
10. Plan for the future and the addition of more services such as Location
BRKEWN-3012
Cisco Public
16
Agenda
Troubleshooting Voice over Wireless LAN Deployments
Voice over WLAN 101
VoWLAN RF Design
VoWLAN Configuration Identifying the Problem Troubleshooting Tools WLCCA WCS/NCS Data Analysis
Debug Analysis
Summary
BRKEWN-3012
Cisco Public
17
VoWLAN RF Design
What Should Be Covered
In order to determine if VoWLAN can be deployed, the environment must be evaluated to ensure the following items meet Cisco guidelines. Many different tools and applications can be used to evaluate these items in order to certify the deployment.
BRKEWN-3012
Cisco Public
18
VoWLAN RF Design
Coverage
The cell edge should be designed to -67 dBm, where there is a 20-30% overlap of adjacent access points at that signal level. This ensures the phone always has adequate signal and can roam seamlessly.
BRKEWN-3012
Cisco Public
19
VoWLAN RF Design
Channel Utilization and Noise
Channel Utilization levels should be kept under 50%. If using the phone, this is provided via the QoS Basic Service Set (QBSS), which equates to a value around 105. Noise levels should not exceed -92 dBm, which allows for a Signal to Noise Ratio (SNR) of 25 dB where a -67 dBm signal should be maintained.
BRKEWN-3012
Cisco Public
20
VoWLAN RF Design
What Should or Should Not Be Done
BRKEWN-3012
Cisco Public
21
VoWLAN RF Design
What Should or Should Not Be Done
Multipath. Multipath should be kept minimal as this can create nulls and reduce signal levels.
Temptation is to mount on beams or ceiling rails This reflects transmitted as well as received packets Dramatic reduction in SNR due to high-strength, multipath signals
VoWLAN RF Design
What Should or Should Not Be Done
User
BRKEWN-3012
VoWLAN RF Design
More Examples
BRKEWN-3012
Cisco Public
24
VoWLAN RF Design
More Examples
VoWLAN RF Design
More Examples
BRKEWN-3012
Cisco Public
26
VoWLAN RF Design
A Few Tips
Every site is unique, do not assume two installations would be the same
Think of the AP coverage area as a reading light: you want to illuminate where the devices will be. Avoid long run AP placement
Use the appropriate equipment for the need: 1130/1140/3500i/3600i for carpeted areas, 1240/1260/3500e/3600e for specific applications, antenna orientations Avoid using internal antennas AP in vertical placements. RF planning is more difficult
BRKEWN-3012
Cisco Public
27
VoWLAN RF Design
A Few More Tips
Use 5 GHz whenever possible (a lot of smartphones are 2.4 GHz only)
BRKEWN-3012
Cisco Public
28
VoWLAN RF Design
Coverage Areas
BRKEWN-3012
Cisco Public
29
VoWLAN RF Design
The Transition AP
Roaming can fail if the client device does not have enough time to properly scan for neighboring access points
Imagine turning the corner around a metal or high attenuation barrier the RF environment changes very rapidly
BRKEWN-3012
Cisco Public
30
VoWLAN RF Design
Scanning Problems At point A the phone is connected to AP 1
B C 3
At point B the phone has AP 2 in the neighbor list, AP 3 has not yet been scanned due to the RF shadow caused by the elevator bank At point C the phone needs to roam, but AP 2 is the only AP in the neighbor list
The phone then needs to rescan and connect to AP 3
BRKEWN-3012
Cisco Public
31
VoWLAN RF Design
Transition AP Placement
1 A B 2
At point B the phone has AP 2 in the neighbor list as it was able to scan it while moving down the hall
At point C the phone needs to roam and successfully selects AP 2 The phone has sufficient time to scan for AP 3 ahead of time
BRKEWN-3012
Cisco Public
32
VoWLAN RF Design
Avoid Pico-cells When Possible
Pico-cells: signal fade too quickly, not giving time for device to do a controlled roam 792x 1.4.2 firmware helps here. CCKM timestamp of 5 may alleviate a bit.
config wlan security wpa akm cckm timestamp-tolerance 5000 <WLAN ID>
BRKEWN-3012
Cisco Public
33
Agenda
Troubleshooting Voice over Wireless LAN Deployments
Voice over WLAN 101 VoWLAN RF Design
VoWLAN Configuration
Identifying the Problem Troubleshooting Tools WLCCA WCS/NCS Data Analysis
Debug Analysis
Summary
BRKEWN-3012
Cisco Public
34
VoWLAN Configuration
Use Design Guide
Disabled
DHCP required P2P blocking MFP client Band select Load balancing Low data rates
Enabled
- Aironet extensions
- DTPC is enabled
- Platinum + 802.1p 6 - Long session timeout - Fast roaming (CCKM/Open/PSK) - WMM (optional/required) - DTIM 2 - AES - EDCA for Voice or mixed
- CAC
Optional
Client Exclusion
BRKEWN-3012 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 35
VoWLAN Configuration
RRM Advanced Settings
Use long DCA period: 8/12/24 hours to prevent frequent channel changes
BRKEWN-3012
Cisco Public
36
VoWLAN Configuration
Channel and Power Levels
UNII-1 Channels are intended for indoor use, typically have lower max transmit power of 14 dB* UNII-2 Channels require use of DFS (Dynamic Frequency Selection), typically have max transmit power of 17 dB* UNII-3 Channels do not require DFS, and have a max transmit power of 17 dB* *Maximum transmit power may vary by AP model
BRKEWN-3012
Cisco Public
37
VoWLAN Configuration
DSCP and COS
BRKEWN-3012
Cisco Public
38
VoWLAN Configuration
DTPC
Mismatched transmit powers on AP and client can cause one-way audio and poor performance 792x phone has max tx power of 40 mW (16 dBm) AP 2.4 GHz can be up to 100 mW (20 dBm) AP 5.0 GHz can be up to 50 mW (17 dBm) (varies by channel) Other benefits include reduced co-channel interference radius and power saving on clients Requires CCXv2
BRKEWN-3012
Cisco Public
39
Agenda
Troubleshooting Voice over Wireless LAN Deployments
Voice over WLAN 101 VoWLAN RF Design VoWLAN Configuration
Debug Analysis
Summary
BRKEWN-3012
Cisco Public
40
Troubleshooting is a process
Do NOT change things without understanding root cause first Find patterns: where, when, how, the problem is reproducible.
BRKEWN-3012
Cisco Public
41
BRKEWN-3012
Cisco Public
42
No way voice
No voice in both directions Typical Triggers
Very bad RF Failed Roaming
BRKEWN-3012
Cisco Public
43
Robotic/Choppy Voice
High packet drops affecting voice quality Typical Triggers
Bad RF/Coverage: lots of retries, low signal levels, no good candidate to roam Bad Roaming decision: phone may be too conservative Interference/Noise QoS configuration
Network Busy
CAC rejected call, high QBSS Typical Triggers
High client count High energy on channel/Interference/Noise
BRKEWN-3012
Cisco Public
44
BRKEWN-3012
Cisco Public
45
BRKEWN-3012
Cisco Public
46
Agenda
Troubleshooting Voice over Wireless LAN Deployments
Voice over WLAN 101 VoWLAN RF Design VoWLAN Configuration Identifying the Problem
Troubleshooting Tools
WLCCA WCS/NCS Data Analysis
Debug Analysis
Summary
BRKEWN-3012
Cisco Public
47
Troubleshooting Tools
Wireless Captures, RF Analysis, Configuration Analysis
Wireless sniffer
Omnipeek/AirPcap Mac with OS X 10.6 and above Windows 7 with Netmon 3.4 Multichannel, for roaming issues AP in Sniffer mode
BRKEWN-3012
Cisco Public
48
Troubleshooting Tools
Debugs, Logs, Reports
Phone debug
Do not overuse, as it will disrupt voice
WCS/NCS
Roaming history SNR levels TSM
Troubleshooting Tools
Data Capturing
Wireless Capture
USB logs Time sync Wireless capture done at phone side. Multichannel if roaming issue is suspected Call placed to fixed phone to isolate one wireless path only
BRKEWN-3012 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 50
Wired Capture
Troubleshooting Tools
Steps to Resolution
BRKEWN-3012
Cisco Public
51
Agenda
Troubleshooting Voice over Wireless LAN Deployments
Voice over WLAN 101 VoWLAN RF Design VoWLAN Configuration Identifying the Problem Troubleshooting Tools
WLCCA
WCS/NCS Data Analysis
Debug Analysis
Summary
BRKEWN-3012
Cisco Public
52
WLCCA
Direct Troubleshooting
Bad!
BRKEWN-3012
Cisco Public
53
WLCCA
RF First View
BRKEWN-3012
Cisco Public
54
WLCCA
RF First View
BRKEWN-3012
Cisco Public
55
WLCCA
RF Groups
May indicate bad coverage. Only relevant for APs on same physical area
Bad!
Good
BRKEWN-3012
Cisco Public
56
WLCCA
RF Problem Finder
Why?
BRKEWN-3012
Cisco Public
57
WLCCA
AP RF Summary
Lots of information
Bad!
BRKEWN-3012
Cisco Public
58
WLCCA
RF Neighbors
BRKEWN-3012
Cisco Public
59
Agenda
Troubleshooting Voice over Wireless LAN Deployments
Voice over WLAN 101 VoWLAN RF Design VoWLAN Configuration Identifying the Problem Troubleshooting Tools WLCCA
WCS/NCS
Data Analysis
Debug Analysis
Summary
BRKEWN-3012
Cisco Public
60
WCS/NCS
History to the Rescue
WCS/NCS can show you where, and how the client has been over time
Bad!
BRKEWN-3012
Cisco Public
61
WCS/NCS
History to the Rescue
Bad!
BRKEWN-3012
Cisco Public
62
Agenda
Troubleshooting Voice over Wireless LAN Deployments
Voice over WLAN 101 VoWLAN RF Design VoWLAN Configuration Identifying the Problem Troubleshooting Tools WLCCA WCS/NCS
Data Analysis
Debug Analysis
Summary
BRKEWN-3012
Cisco Public
63
Data Analysis
Needle in a Haystack?
BRKEWN-3012
Cisco Public
64
Data Analysis
Filters
Bad Good
Bad
BRKEWN-3012
Cisco Public
65
Data Analysis
Filters
Why?
BRKEWN-3012
Cisco Public
66
Data Analysis
One Way Voice
BRKEWN-3012
Cisco Public
67
Data Analysis
Robotic/Choppy Voice
BRKEWN-3012
Cisco Public
68
Data Analysis
Slow Roaming
BRKEWN-3012
Cisco Public
69
Data Analysis
QoS Verification
On the 792x phone, while on a call navigate to Settings > Status > Network Statistics Check that the DataRcvVO counter is incrementing
Packet Capture
BRKEWN-3012
Cisco Public
70
Data Analysis
Packet Loss and Delay
Per voice guidelines, PER should not exceed 1% packet loss. If there is > 1% packet loss, then voice quality can be degraded significantly. All Cisco IP Phones have the ability to display receiver lost packets as well as the total # of receiver packets in the stream (call) statistics. Simply divide the receiver lost packets by the total # of receiver packets. Jitter should also be kept at a minimum (< 100 ms).
BRKEWN-3012
Cisco Public
71
Agenda
Troubleshooting Voice over Wireless LAN Deployments
Voice over WLAN 101 VoWLAN RF Design VoWLAN Configuration Identifying the Problem Troubleshooting Tools WLCCA WCS/NCS Data Analysis
Debug Analysis
Summary
BRKEWN-3012
Cisco Public
72
Debug Analysis
Few Pointers
On mobility scenarios (multiple controllers), always debug client + debug mobility on all WLC where client may roam to
The controller will give a view of what is going on, and can close a lot the spectrum of issues to investigate
BRKEWN-3012 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 73
Debug Analysis
WLC Client Debug
Proper roams will be Reassociations, if you see Associations then there are roaming failures that should be investigated You can trace the roaming path of the phone by looking up the AP radio mac addresses at each (re)association *apfMsConnTask_0: Apr 13 16:10:27.014: cc:08:e0:2e:10:2b Association received from mobile on AP b4:a4:e3:b5:bc:60
BRKEWN-3012
Cisco Public
74
Debug Analysis
792x Phone Debugs
Set phone trace logs from Error to Info (or Debug, but this can impact performance of the phone)
Kernel WLAN Driver WLAN Manager
The phone log will show the neighbor list and roaming trigger for each roam
WLAN_DRVR: 3845.629757: Roam trigger = ROAMING_TRIGGER_MAX_TX_RETRIES WLAN_DRVR: 3845.643734: Candidate 0, BSSID=c4:7d:4f:3b:02:e2, RSSI =-60 WLAN_DRVR: 3845.650550: Candidate 1, BSSID=00:19:30:76:52:dc, RSSI =-63 WLAN_DRVR: 3845.657382: Candidate 2, BSSID=00:19:56:b0:79:f0, RSSI =-72 WLAN_DRVR: 3845.664203: Candidate 3, BSSID=1c:df:0f:b5:47:b2, RSSI =-74
BRKEWN-3012
Cisco Public
75
Debug Analysis
Some Examples
Idle timeout from AP, shortly after roam, CSCto73361 fixed in 7.0.220.0
*pemReceiveTask: Nov 23 15:34:44.946: 00:15:f9:93:42:9e 192.168.32.32 Added NPU entry of type 1, dtlFlags 0x0 *spamApTask0: Nov 23 15:34:44.948: 00:15:f9:93:42:9e Received Idle-Timeout from AP 00:24:14:31:ce:70, slot 0 for STA 00:15:f9:93:42:9e *spamApTask0: Nov 23 15:34:44.948: 00:15:f9:93:42:9e apfMsDeleteByMscb Scheduling mobile for deletion with deleteReason 4, reasonCode 4
BRKEWN-3012
Cisco Public
76
Debug Analysis
Some Examples
In logs
*apfMsConnTask_5: Mar 22 14:49:36.109: %APF-3-VALIDATE_CCKM_REASS_REQ_ELEMENT: apf_ut:2122 Could not validate the CCKM Reassociation request element.Received Timestamp deviation > 1sec in CCKM Info Element from mobile. Mobile:00:01:e3:bb:e5:23,
BRKEWN-3012
Cisco Public
77
Debug Analysis
EAPoL Key Failure
*Dot1x_NW_MsgTask_2: Aug 30 16:39:06.201: 00:1e:4a:3f:af:4f Sending EAPOL-Key Message to mobile 00:1e:4a:3f:af:4f state PTKINITNEGOTIATING (message 3), replay counter 00.00.00.00.00.00.00.02
*osapiBsnTimer: Aug 30 16:39:06.801: 00:1e:4a:3f:af:4f 802.1x 'timeoutEvt' Timer expired for station 00:1e:4a:3f:af:4f and for message = M3
*dot1xMsgTask: Aug 30 16:39:06.801: 00:1e:4a:3f:af:4f Retransmit 1 of EAPOL-Key M3 (length 155) for mobile 00:1e:4a:3f:af:4f *osapiBsnTimer: Aug 30 16:39:07.201: 00:1e:4a:3f:af:4f 802.1x 'timeoutEvt' Timer expired for station 00:1e:4a:3f:af:4f and for message = M3
*dot1xMsgTask: Aug 30 16:39:07.201: 00:1e:4a:3f:af:4f Retransmit 2 of EAPOL-Key M3 (length 155) for mobile 00:1e:4a:3f:af:4f
*osapiBsnTimer: Aug 30 16:39:07.601: 00:1e:4a:3f:af:4f 802.1x 'timeoutEvt' Timer expired for station 00:1e:4a:3f:af:4f and for message = M3 *dot1xMsgTask: Aug 30 16:39:07.601: 00:1e:4a:3f:af:4f Retransmit failure for EAPOL-Key M3 to mobile 00:1e:4a:3f:af:4f, retransmit count 3, mscb deauth count 0
*dot1xMsgTask: Aug 30 16:39:07.601: 00:1e:4a:3f:af:4f Sent Deauthenticate to mobile on BSSID c4:7d:4f:3b:02:e0 slot 1(caller 1x_ptsm.c:534)
BRKEWN-3012
Cisco Public
78
Debug Analysis
Mitigating EAPoL Failure Impact
BRKEWN-3012
Cisco Public
79
Agenda
Troubleshooting Voice over Wireless LAN Deployments
Voice over WLAN 101 VoWLAN RF Design VoWLAN Configuration Identifying the Problem Troubleshooting Tools WLCCA WCS/NCS Data Analysis
Debug Analysis
Summary
BRKEWN-3012
Cisco Public
80
Summary
Key Take Aways
You cant just drop voice on a WiFi deployment and expect it to work
BRKEWN-3012
Cisco Public
81
Recommended Reading
BRKEWN-3012
Cisco Public
82
Complete your session evaluation online now (open a browser through our wireless network to access our Dont forget to activate your Cisco Live Virtual account for access to portal) or visit one of the Internet stations throughout the Convention all session material, communities, and on-demand and live activities throughout Center. the year. Activate your account at the
Cisco booth in the World of Solutions or visit www.ciscolive.com.
BRKEWN-3012 2012 Cisco and/or its affiliates. All rights reserved. Cisco Public 83
Final Thoughts
Get hands-on experience with the Walk-in Labs located in World of Solutions, booth 1042 Come see demos of many key solutions and products in the main Cisco booth 2924 Visit www.ciscoLive365.com after the event for updated PDFs, ondemand session videos, networking, and more! Follow Cisco Live! using social media:
Facebook: https://www.facebook.com/ciscoliveus
Twitter: https://twitter.com/#!/CiscoLive
LinkedIn Group: http://linkd.in/CiscoLI
BRKEWN-3012
Cisco Public
84
BRKEWN-3012
Cisco Public