Document Type: MOP: RNC & Other Data Elements Basic Troubleshooting
Document Type: MOP: RNC & Other Data Elements Basic Troubleshooting
Document Type: MOP: RNC & Other Data Elements Basic Troubleshooting
Prepared By
Reviewed By
Authorized By
Pravin
Kokate
Kishan S Rawat
Hiren Joshi
Release On :- 20-June-2011
Validity: Till Further Notification
Page 1 of 18
Index
1.
2.
3.
4.
5.
Page 2 of 18
*************************************************
************ flx02 -> 1xEV-DO R32.0***********
*************************************************
R32 SU2 TMP E
TERM is set to "vt100".
[flx202]-> tools
/flx/ONEXEVc/current/release/tools
[flx202]-> cd tss
/flx/ONEXEVc/current/release/tools/tss
[flx202]-> su onexev
Password:1xevadm
[flx202]-> ./show_pdsns
Check the Sessions on all TP cards as below, please ensure below points.
1. TPs should take equal session load
2. All TPs must be reachable with both the PDSNs (Mumbai &
Chennai).
Page 3 of 18
***************************************************************************************
DATE : 13 Dec 2010
TIME : 15:47:10
***************************************************************************************
+-------------------------------------------------------------------------------------+
| TP ID : 321
|
| TP IP Address : 172.12.38.9
|
| TP Status : Up
|
+-----------------+---------+------------------+-------+------------+-----------------+
| PDSN IP Address | Type | In File |
State | SPI | Time Drift | Sessions |
+-----------------+---------+------------------+-------+------------+-----------------+
| 10.239.200.128 | PDSN | yes | Reachable
| 100 |
-14 | 432 |
| 10.239.201.128 | PDSN | yes | Reachable
| 100 |
-14 |
0|
+-----------------+---------+------------------+-------+------------+-----------------+
***************************************************************************************
+-------------------------------------------------------------------------------------+
| TP ID : 322
|
| TP IP Address : 172.12.38.10
|
| TP Status : Up
|
+-----------------+---------+------------------+-------+------------+-----------------+
| PDSN IP Address | Type | In File |
State | SPI | Time Drift | Sessions |
+-----------------+---------+------------------+-------+------------+-----------------+
| 10.239.200.128 | PDSN | yes | Reachable
| 100 |
-14 | 430 |
| 10.239.201.128 | PDSN | yes | Reachable
| 100 |
-14 |
0|
+-----------------+---------+------------------+-------+------------+-----------------+
***************************************************************************************
+-------------------------------------------------------------------------------------+
| TP ID : 323
|
| TP IP Address : 172.12.38.11
|
| TP Status : Up
|
+-----------------+---------+------------------+-------+------------+-----------------+
| PDSN IP Address | Type | In File |
State | SPI | Time Drift | Sessions |
+-----------------+---------+------------------+-------+------------+-----------------+
| 10.239.200.128 | PDSN | yes | Reachable
| 100 |
-14 | 402 |
| 10.239.201.128 | PDSN | yes | Reachable
| 100 |
-14 |
0|
+-----------------+---------+------------------+-------+------------+-----------------+
***************************************************************************************
+-------------------------------------------------------------------------------------+
| TP ID : 324
|
| TP IP Address : 172.12.38.12
|
| TP Status : Up
|
+-----------------+---------+------------------+-------+------------+-----------------+
| PDSN IP Address | Type | In File |
State | SPI | Time Drift | Sessions |
+-----------------+---------+------------------+-------+------------+-----------------+
| 10.239.200.128 | PDSN | yes | Reachable
| 100 |
-14 | 407 |
| 10.239.201.128 | PDSN | yes | Reachable
| 100 |
-14 |
0|
+-----------------+---------+------------------+-------+------------+-----------------+
***************************************************************************************
+-------------------------------------------------------------------------------------+
| TP ID : 326
|
| TP IP Address : 172.12.38.14
|
| TP Status : Up
|
+-----------------+---------+------------------+-------+------------+-----------------+
| PDSN IP Address | Type | In File |
State | SPI | Time Drift | Sessions |
Page 4 of 18
+-----------------+---------+------------------+-------+------------+-----------------+
| 10.239.200.128 | PDSN | yes | Reachable
| 100 |
-14 | 415 |
| 10.239.201.128 | PDSN | yes | Reachable
| 100 |
-14 |
0|
+-----------------+---------+------------------+-------+------------+-----------------+
***************************************************************************************
+-------------------------------------------------------------------------------------+
| TP ID : 328
|
| TP IP Address : 172.12.38.16
|
| TP Status : Up
|
+-----------------+---------+------------------+-------+------------+-----------------+
| PDSN IP Address | Type | In File |
State | SPI | Time Drift | Sessions |
+-----------------+---------+------------------+-------+------------+-----------------+
| 10.239.200.128 | PDSN | yes | Reachable
| 100 |
-14 | 404 |
| 10.239.201.128 | PDSN | yes | Reachable
| 100 |
-14 |
0|
+-----------------+---------+------------------+-------+------------+-----------------+
TP/TR/TV/P/T
1 0 0Y0
1 0 0Y0
0 0 0Y0
0 0 0Y0
Flags
Link
Link Num Num Num Jumbo QOS Load
State ELSM UPS STP VLAN Proto Size profile Master
============================================================
1
Em-la---e--fMB- active - 2 1 1
1 9216 none 1 a
2
Em-la---e--fMB- active - 2 1 1
1 9216 none 1 a
3
Em-la---e--fMB- active - 2 1 1
1 9216 none 1 a
4
Em-la---e--fMB- active - 2 1 1
1 9216 none 1 a
5
Em------e--fMB- active - 19 1 1
1 9216 none
6
Em------e--fMB- active - 21 1 1
1 9216 none
7
Em------e--fMBready - 0 1 1
1 9216 none
8
Em------e--fMBready - 0 1 1
1 9216 none
9
Em------e--fMB- active - 11 1 1
1 9216 none
10
Em------e--fMB- active - 7 1 1
1 9216 none
11
Em------e--fMBready - 0 1 1
1 9216 none
12
Em------e--fMBready - 0 1 1
1 9216 none
13
Em------e--fMB- active - 48 1 1
1 9216 none
14
Em------e--fMBready - 0 1 1
1 9216 none
15
Em------e--fMBready - 0 1 1
1 9216 none
16
Em------e--fMBready - 0 1 1
1 9216 none
17
Em------e--fMB- active - 40 1 1
1 9216 none
18
Em------e--fMB- active - 47 1 1
1 9216 none
19
Em------e--fMBready - 0 1 1
1 9216 none
20
Em------e--fMBready - 0 1 1
1 9216 none
21
Em------e--fMB- active - 20 1 1
1 9216 none
22
Em------e--fMB- active - 20 1 1
1 9216 none
23
Em------e--fMBready - 0 1 1
1 9216 none
24
Em------e--fMBready - 0 1 1
1 9216 none
25
Em------e--fMB- active - 12 1 1
1 9216 none
26
Em------e--fMB- active - 8 1 1
1 9216 none
27
Em------e--fMBready - 0 1 1
1 9216 none
28
Em------e--fMBready - 0 1 1
1 9216 none
29
Em------e--fMB- active - 21 1 1
1 9216 none
30
Em------e--fMBready - 0 1 1
1 9216 none
31
Em------e--fMBready - 0 1 1
1 9216 none
32
Em------e--fMBready - 0 1 1
1 9216 none
33
Em------e--fMB- active - 18 1 1
1 9216 none
34
Em------e--fMB- active - 34 1 1
1 9216 none
35
Em------e--fMBready - 0 1 1
1 9216 none
36
Em------e--fMBready - 0 1 1
1 9216 none
37
Em-la---e--fMB- active - 2 1 1
1 9216 none 1 a
38
Em-la---e--fMB- active - 2 1 1
1 9216 none 1 a
39
Em-la---e--fMB- active - 2 1 1
1 9216 none 1 a
Page 6 of 18
40
Em-la---e--fMB- active - 2 1 1
1 9216 none 1 a
41
Em------e--fMBready - 0 0 0
0 9216 none
42
Em------e--fMBready - 0 0 0
0 9216 none
43
Em------e--fMBready - 0 0 0
0 9216 none
44
Em------e--fMBready - 0 0 0
0 9216 none
45
Em-la---e--fMB- active - 3 0 2
0 9216 none 45 a
46
Em-la---e--fMB- active - 3 0 2
0 9216 none 45 a
47
Dm------e--fMBready - 0 0 2
0 9216 none
48
Dm------e--fMBready - 0 0 2
0 9216 none
49
Em------e--fMBready - 0 0 0
0 9216 none
50
Em------e--fMBready - 0 0 0
0 9216 none
=============================================================
> indicates Port Display Name truncated past 8 characters
Flags : a - Load Sharing Algorithm address-based, D - Port Disabled,
e - Extreme Discovery Protocol Enabled, E - Port Enabled,
g - Egress TOS Enabled, j - Jumbo Frame Enabled,
l - Load Sharing Enabled, m - MACLearning Enabled,
n - Ingress TOS Enabled, o - Dot1p Replacement Enabled,
P - Software redundant port(Primary),
R - Software redundant port(Redundant),
q - Background QOS Monitoring Enabled,
s - diffserv Replacement Enabled,
v - Vman Enabled, f - Unicast Flooding Enabled,
M - Multicast Flooding Enabled, B - Broadcast Flooding Enabled
L - Extreme Link Status Monitoring Enabled
* X450a-UNC21_A.3 #
13
A
9673
21775
8991
19767
14
R
0
0
0
0
15
R
0
0
0
0
16
R
0
0
0
0
=========================================================================
> indicates Port Display Name truncated past 8 characters
Link Status : A-Active, R-Ready, NP-Port Not Present, L-Loopback
Link Utilization Averages
Mon Dec 13 17:27:48 2010
Port Link Receive
Peak Rx
Transmit
Peak Transmit
Status packets/sec packets/sec packets/sec packets/sec
=========================================================================
17
A
8724
21696
8061
20074
18
A
9089
15496
8496
16600
19
R
0
0
0
0
20
R
0
0
0
0
21
A
0
4
6
1608
22
A
0
3
6
1608
23
R
0
0
0
0
24
R
0
0
0
0
25
A
433
8182
484
4153
26
A
596
3656
618
7191
27
R
0
0
0
0
28
R
0
0
0
0
29
A
1
6777
6
6050
30
R
0
0
0
0
31
R
0
0
0
0
32
R
0
0
0
0
=========================================================================
> indicates Port Display Name truncated past 8 characters
Link Status : A-Active, R-Ready, NP-Port Not Present, L-Loopback
Link Utilization Averages
Mon Dec 13 17:27:54 2010
Port Link Receive
Peak Rx
Transmit
Peak Transmit
Status packets/sec packets/sec packets/sec packets/sec
=========================================================================
33
A
8185
20234
7594
18269
34
A
1
7625
5
6592
35
R
0
0
0
0
36
R
0
0
0
0
37
A
968
6458
556
2663
38
A
6254
16718
3510
9305
39
A
5378
13267
3405
9328
40
A
2290
7069
1117
3263
41
R
0
0
0
0
42
R
0
0
0
0
43
R
0
0
0
0
44
R
0
0
0
0
45
A
11108
25200
17477
29861
46
A
11288
26525
19405
29223
47
R
0
0
0
0
48
R
0
0
0
0
=========================================================================
> indicates Port Display Name truncated past 8 characters
Page 8 of 18
Page 9 of 18
Take extreme switch logs for analysis of faults, check for the
time stamp.
* X450a-UNC21_A.1 # show log
12/13/2010 19:14:40.27 <Info:AAA.authPass> Login passed for user root through telnet (172.12.38.1)
12/13/2010 19:10:50.29 <Info:AAA.logout> User root logout from telnet (172.12.38.3)
12/13/2010 19:10:47.07 <Info:AAA.authPass> Login passed for user root through telnet (172.12.38.3)
12/13/2010 19:05:36.65 <Info:AAA.logout> User root logout from telnet (172.12.38.3)
Page 10 of 18
12/13/2010 19:05:32.42 <Info:AAA.authPass> Login passed for user root through telnet (172.12.38.3)
12/13/2010 19:00:21.67 <Info:AAA.logout> User root logout from telnet (172.12.38.3)
12/13/2010 19:00:17.41 <Info:AAA.authPass> Login passed for user root through telnet (172.12.38.3)
12/13/2010 18:55:54.41 <Info:EPM.cpu> CPU utilization monitor: process aaa consumes 86 % CPU
12/13/2010 18:55:52.29 <Info:AAA.logout> User root logout from telnet (172.12.38.1)
12/13/2010 18:55:06.63 <Info:AAA.logout> User root logout from telnet (172.12.38.3)
12/13/2010 18:55:02.39 <Info:AAA.authPass> Login passed for user root through telnet (172.12.38.3)
12/13/2010 18:49:52.05 <Info:AAA.logout> User root logout from telnet (172.12.38.3)
12/13/2010 18:49:47.76 <Info:AAA.authPass> Login passed for user root through telnet (172.12.38.3)
12/13/2010 18:44:38.25 <Info:AAA.logout> User root logout from telnet (172.12.38.3)
12/13/2010 18:44:35.04 <Info:AAA.authPass> Login passed for user root through telnet (172.12.38.3)
12/13/2010 18:44:34.49 <Info:EPM.cpu> CPU utilization monitor: process aaa consumes 90 % CPU
12/13/2010 18:39:25.16 <Info:AAA.logout> User root logout from telnet (172.12.38.3)
12/13/2010 18:39:22.58 <Info:AAA.authPass> Login passed for user root through telnet (172.12.38.1)
12/13/2010 18:39:20.87 <Info:AAA.authPass> Login passed for user root through telnet (172.12.38.3)
12/13/2010 18:34:08.86 <Info:AAA.logout> User root logout from telnet (172.12.38.3)
12/13/2010 18:34:06.29 <Info:AAA.authPass> Login passed for user root through telnet (172.12.38.3)
12/13/2010 18:28:54.23 <Info:AAA.logout> User root logout from telnet (172.12.38.3)
12/13/2010 18:28:51.28 <Info:AAA.authPass> Login passed for user root through telnet (172.12.38.3)
12/13/2010 18:23:40.54 <Info:AAA.logout> User root logout from telnet (172.12.38.3)
Page 12 of 18
Go to Hdrfframe to check alarms on APs & TPs. Just right click on AP or TP & give
Poke command OP: ALARM. Check for all Boards. Incase of any alarm take help
from SME to clear the same.
Page 13 of 18
Page 14 of 18
omp_delhi2 rakesh> cgrep -R "LOST COMM WITH BACKUP EVDO RNC OHM" 101214.APX| more
12 REPT:CELL 177 CDM 3, CCU(EVM) 6, HEH
SUPPRESSED MSGS: 0
ERROR TYPE: ONEBTS MODULAR CELL ERROR
CLEARED: LOST COMM WITH BACKUP EVDO RNC OHM
02 03 00 00 00 00 00 00
00 00 00 00 00 00 00 00
00 00 00 00
12/14/10 13:12:26 #309384
08 REPT:CELL 161 CDM 3, CCU(EVM) 6, HEH
SUPPRESSED MSGS: 0
ERROR TYPE: ONEBTS MODULAR CELL ERROR
CLEARED: LOST COMM WITH BACKUP EVDO RNC OHM
02 03 00 00 00 00 00 00
00 00 00 00 00 00 00 00
00 00 00 00
12/14/10 15:08:14 #359809
omp_delhi2 rakesh>
Page 15 of 18
command
11062009.EVDOPCMD.gz
11062010.EVDOPCMD.gz
11062011.EVDOPCMD
00a000001fcfdc45
000000004cf23a49
00000000ff403a7b
00000000ff402942
00a000002d3d9b84
00a000002d3d9a69
00a000002d3c9ee7
00a000001a2a74cf
00a000001f9b83d6
0000000006cf4a01
00a000001fe217a3
00a000001fcfe136
0000000006d162fa
00a000001f72977f
00a000001fe23698
00a000001f773bb0
000000000551bd11
0000000002fcb8e6
000000004cf23a64
00a000001fd00511
0000000012d5c4f2
Page 16 of 18
172.12.38.16
172.12.38.11
172.12.38.12
172.12.38.12
172.12.38.14
172.12.38.12
172.12.38.10
172.12.38.14
172.12.38.16
172.12.38.14
172.12.38.14
172.12.38.12
172.12.38.9
172.12.38.10
172.12.38.12
172.12.38.14
172.12.38.12
172.12.38.11
172.12.38.9 10.239.200.128
172.12.38.11 10.239.200.128
ALUMS TEC RAN
Page 17 of 18
Status
up
up
up
up
up
up
up
up
down
down
down
down
up
up
up
up
down
down
up
up
up
up
down
down
down
up
up
up
down
up
Protocol Description
up
up
mgmt-vlan
up
ospf-vlan
up
External-RAN-vlan
up
External-EVDO_DO-AP-vlan
up
External-PDN-vlan-MCN-2
up
Inter-RNC-Hand-Off-Delhi-MCN-2-v
up
*****connected to DLHIM002WLH02*****
down UNC-5 AP
down UNC-5 AP
down UNC-5 AP
down UNC-5 AP
up
UNC-6 AP
up
UNC-6 AP
up
UNC-6 AP
up
UNC-6 AP
down *****Inter MLS Switch Connectivity*****
down *****Inter MLS Switch Connectivity*****
up
*****Inter MLS Switch Connectivity*****
up
*****Connected to DLHIM002WLI03*****
up
*****Connected to DLHIM002WLI04*****
up
*****connected to RDN via ODF*******
down *****connected to RDN via ODF*******
down *****connected to UNC-5 Extreme switch UNC-5*****
down *****connected to UNC-5 Extreme switch UNC-5*****
up
*****connected to UNC-5 Extreme switch UNC-6*****
up
*****connected to UNC-5 Extreme switch UNC-6*****
up
****Etherchannel For Inter MLS Switch Connectivity*****
down ****Etherchannel For C4503 to Extreme Switch Connectivity*****
up
****Etherchannel For C4503 to Extreme Switch Connectivity*****
----------END----------
Page 18 of 18