Cisco Support Community Expert Series Webcast

Cisco Support Community Expert
Series Webcast
Cisco Unified Computing System (UCS) B series
Upgrade & Troubleshooting
VARUN MEHTA , ANUPAM ASTHANA
CSE, SERVER VIRTUALIZATION TAC
Q&A PANEL : AVINASH SHUKLA
16th SEPTEMBER 2014
Todays featured experts is Cisco Support Engineers
Ask questions now, about Unified Computing System, Upgrade &

Troubleshooting
Varun Mehta
Anupam Asthana
Customer Support Engg.
Customer Support Engg.
2013-2014 Cisco and/or its affiliates. All rights reserved.
Event Date: 16th Sep 2014
Avinash Shukla
CSE
Todays presentation will include audience polling questions.

We encourage you to participate!
If you would like a copy of the presentation slides, click the PDF file link in
the chat box on the right or go to:
https://supportforums.cisco.com/document/12302516/cisco-ucs-b-seriesupgrade-troubleshooting-slides-live-webcast
Do you use UCS in your company infrastructure?

a.
Yes, I am using it.
b.
Not yet, but I plan to use in future.
c.
No, I am not using it.
d.
Not Applicable.
Submit Your Questions Now!

Use the Q & A panel to submit your questions and the panel of experts
will respond.
Cisco Unified Computing System (UCS) B series

Upgrade & Troubleshooting
VARUN MEHTA , ANUPAM ASTHANA,
CSE, SERVER VIRTUALIZATION TAC
Q&A PANEL : AVINASH SHUKLA
DATE : 16th SEPTEMBER 2014
Introduction to Unified Computing System, Upgrade & Troubleshooting
UCS Architecture in brief

Manual Upgrade Process
Upgrade using Auto-Install
Live Demo of the Upgrade
UCS Troubleshooting
Q&A Session
UCS Architecture
UCS Building Blocks
UCS Manager
Fabric Interconnect
IO Modules
Endpoints
CIMC
BIOS
Adapter
Storage Controller
Board Controller
10
11
Fabric Interconnect
(FI)
Presentation_ID
2010 Cisco Systems, Inc. All rights reserved.
CAE Bootcamp
12
UCS Fabric Interconnect Overview
Each UCS system has 2 independent single switch fabrics called UCS 6x00 (Fabric
Interconnects)
based on Nexus 5K switch, but running different software
In addition to NXOS runs UCS Manager (UCSM)
Server ports connect to IO Modules in the blade chassis
Network ports connect via Fibre Channel or Ethernet to the upstream SAN or LAN
One Management port connects to the customers Out-Of-Band management network
Two Clustering ports connect the UCS Manager instances in peered 6x00s (active/
passive)
RS-232 serial port ( Console )
13
UCS 6248 FUCS 6248 Fabric Interconnect

abric Interconnect
the Port Density of Previous Generation, the Latency, Unified Ports
Double+
32 x Fixed ports: 1/10 GE or
1/2/4/8 FC
Fabric
Interconnect
Cluster
Connectivity
Console
BRKCOM-1005
Expansion Module
(GEM)
Out of Band
Mgmt
10/100/10
00
Fan Module
Fan Module
N + N Redundant FANs
2014 Cisco and/or its affiliates. All rights reserved.
Power Entry
Power Entry
N + N Power Supplies
Cisco Public
14
Cisco UCS Fabric Infrastructure

Cisco UCS 6200 and 2200 with Unied Ports
UCS Fabric
Interconnects
Typical Deployments
High End Deployments
48 Port Fabric Interconnect
96 Port Fabric Interconnect
Performance for typical deployments

1TB throughput
48 ports in 1RU
High Application performance

2TB throughput
High workload density 96 ports in 2RU
Infrastructure agility with Unified Ports
Infrastructure agility with Unified Ports
2204XP I/O Module
UCS FEX
I/O Modules
Porfolio
80G per chassis

20Gb to the Blade per I/O Module
40Gb total per blade
Improved Utilisation with Port Channels
2208XP I/O Module

160G per chassis
40Gb to the Blade per I/O Module
80Gb total per blade, for burst traffic
Improved Resiliency
Improved Utilisation with Port Channels
11
UCS 6200: Unified Ports
UCS
6200: Unified Ports
Dynamic Port Allocation: Lossless
Ethernet or Fiber Channel

Dynamic Port Allocation: Lossless Ethernet or Fibre Channel
FC
Eth
Lossless Ethernet
Native Fibre Channel
1GbE or 10GbE, FCoE, iSCSI, NAS
Supported Speeds: 1 / 2 / 4 / 8 Gb FC
Use-cases
Benefits
Simplify switch purchase. Removes

ports:ratio guess work
Flexible LAN & storage convergence based on

business needs
Increases design flexibility
Remove specific protocol bandwidth

bottlenecks
Service can be adjusted based on the demand

for specific traffic
16
Cisco Public
Unified Computing System Manager
Embedded device manager for family of UCS components

Enables stateless computing via Service Profiles
Efficient scale: Same effort for 1 or N blades
17
UCSM
Single point of device management
Standard APIs for systems management
Adapters, blades, chassis, LAN and SAN connectivity

Embedded manager
GUI & CLI
XML, SMASH-CLP, WSMAN, IPMI, SNMP

SDK for commercial and custom implementations
Designed for multi-tenancy
RBAC, organizations, pools and policies
18
Chassis
Presentation_ID
CAE Bootcamp
19
UCS Blade Slots
Blades are numbered from 1-8, going left to right across rows if looking
at the front of the chassis
Can mix half-width blades with full-width blades
If a full-width blade is in the system, it will be assigned the odd number
associated with the left most slot
20
Rear View of UCS Exhaust Air

expansion
module bay 1
or 2
2 x cluster ports
2 x power entry
1 x management port
console port
8 x fan
modules
2x fabric
extenders
4 x 10GE SFP+
fabric ports
4 x power entry
21
IO Module
(IOM / FEX)
Presentation_ID
CAE Bootcamp
22
Fabric Extender (IOM)
Part of the FI: distributed line card

Extends the I/O fabric into the blade chassis
Transparent to the user managed completely by FIs
Direct 10Gb connectivity between blades and Interconnect
Contains (CMC) Chassis Management Controller

Proxy for mgmt control plane UCSM interface to blades
Key part of HW discovery process
I/O bandwidth
10Gb/port
4 or 8 ports to FI 8 or 32 ports to blades
Blade connects to both Fabric Extender slots in chassis
23
I/O Modules (Fabric Extender)
2104-XP
8x 10GE HIFs
4x 10GE NIFs
40Gbps
2208-XP
32x 10GE HIFs
8x 10GE NIFs
80Gbps
24
Soquel Cisco UCS 2208XP
25
IO Module HIF to NIF Pinning (2208XP)

Slot 1
1-4
Slot 2
5-8
Slot 3
9-12
Slot 4
13-16
Slot 5
17-20
Slot 6
21-24
Slot 7
25-28
Slot 8
29-32
FEX
Fabric
Interconnect
26

Slot 1
1-4
Slot 2
5-8
Slot 3
9-12
Slot 4
13-16
Slot 5
17-20
Slot 6
21-24
Slot 7
25-28
Slot 8
29-32
FEX
Fabric
Interconnect
27

Slot 1
1-4
Slot 2
5-8
Slot 3
9-12
Slot 4
13-16
Slot 5
17-20
Slot 6
21-24
Slot 7
25-28
Slot 8
29-32
FEX
Fabric
Interconnect
28

Slot 1
1-4
Slot 2
5-8
Slot 3
9-12
Slot 4
13-16
Slot 5
17-20
Slot 6
21-24
Slot 7
25-28
Slot 8
29-32
FEX
Fabric
Interconnect
29
IOM and Failover

What happens in a 4-link topology when you loose 1 link?
Server on that link will lose a data path
Since we only support 1, 2, 4 and 8 link topologies the UCS will fall back to 2 links with regards to blade to
fabric port mapping if re-ackd.
Since we dont support 3 links, and fall back to a 2 link topology you need to re-acknowledge the chassis
(manual intervention) in order to re-map the fabric ports.
If 1 link failed out of 4 links and you haven't re-acknowledged the chassis, no pinning is changed, so, the
remaining 3 links will still pass traffic. After re-acknowledge is done, 2-link policy will apply.
The servers VNICs mapped to the failed link will remain down unless failover has been configured
30
Blades and
Adapters
Presentation_ID
CAE Bootcamp
31
Blades
Half width blade
form factor
2 x Flexible
Flash 16GB
SD Cards
24 x DDR3
DIMMs
Dual 2 x 10Gb
optional
modular LOM
(dedicated for
VIC 1240)
2 x hot plug
SAS / SATA
HDD / SSD
Internal USB
2.0 port
2 x E5-2600
CPUs
1 x Mezzanine slot (Gen 3)

for VIC option or 3rd party
mezzanine
32
IOM
IOM
2 KR lanes go to the mLOM slot

2 KR lanes go to the Mezz Slot
B200M3
Mezz Slot
mLOM Slot
x16 Gen 2
CPU # 1
QPI
x16 Gen 2
CPU # 0
33
UCS VIC-1280 Features

PCIe x16 Gen2 Host Interface
Eight 10G DCE ports i.e. ports to IOM
When paired with a UCS 6248UP switch and a UCS 2208XP IOM:
4x10G mode
128 VIFs (116 user VIFs)
When paired with a UCS 6120XP/6140XP switch and a UCS 2104XP IOM:
10G mode
64 VIFs (58 user VIFs)
34
UCS Connectivity & Environmental
Presentation_ID
Cisco Public
35
IOM to Fabric Interconnect Connectivity
36
Connectivity From Servers Upstream
Interconnect B
Interconnect A
Servers
IO Modules
IO Modules
37
Service Profiles
&
Stateless Computing
Presentation_ID
Cisco Public
38
Traditional Server Deployment
Storage Administrator:
Configure LUN access
Masking, binding, boot LUN
Configure switch
Zoning, VSANs, QoS
Server Administrator:
Configure management LAN
Upgrade firmware versions
Chassis, BMC, BIOS, adapters
Configure BIOS settings

Configure NIC settings
Configure HBA settings
Configure boot parameters
Network Administrator:
Configure LAN access
Uplinks, VLANs
Configure policies
QoS, ACLs
Perform tasks for each server

Inhibits pay-as-you-grow incremental deployment
Needs admin coordination every time
May incur downtime during deployments
Complex server replacement, upgrade, migration process
Most of these tasks need to be performed for replacement server
39
UCS Service Profiles: Workload Centric
SAN
LAN
40
Service Profiles:
Upgrades/Maintenance
Profile Name: finance-01
UUID: 56 4d cd 3f 59 5b 61
MAC : 08:00:69:02:01:FC
WWN: 5080020000075740
Boot Order: SAN, LAN
Firmware: xx.yy.zz
Old Server
Chassis 1 Blade 1
New Server
Chassis 10 Blade 2
Disassociate service profile from old server

Associate service profile to new server
Old server can be upgraded, retired or re-purposed
41
Service Profile Creation
42
Pools, Policies, Templates

Building block before we configure or build a server by using service profiles
Pools Predefined Resources
Policies Rules to be followed
Templates Common configuration built using pools and policies that can be applied for a
specific OS, hypervisor, db, application.
43
Identity Resource Pools

Managing server identity
Used to automatically assign identities to server and interfaces
Basis for stateless computing
Incorporate Cisco OUI 00:25:b5 where appropriate
MAC pools
WWN pools
UUID suffix pools
MAC pool 1
WWNN pool 1
UUID pool 1
00:25:b5:00:00:01
20:00:25:b5:11:00:00:01
100025b5-0000-0000-1000-0a0b0c0d0e01
MAC pool n
00:25:b5:00:00:02
WWN pool n
20:00:25:b5:11:00:00:02
UUID pool n
100025b5-0000-0000-1000-0a0b0c0d0e02
00:25:b5:00:ff:01
00:25:b5:00:00:03
20:00:25:b5:ff:00:00:01
20:00:25:b5:11:00:00:03
100025b5-0000-0000-2000-0a0b0c0d0e01
100025b5-0000-0000-1000-0a0b0c0d0e03
00:25:b5:00:ff:02
00:25:b5:00:00:04
20:00:25:b5:ff:00:00:02
20:00:25:b5:11:00:00:04
100025b5-0000-0000-2000-0a0b0c0d0e02
100025b5-0000-0000-1000-0a0b0c0d0e04
00:25:b5:00:ff:03
20:00:25:b5:ff:00:00:03
100025b5-0000-0000-2000-0a0b0c0d0e03
00:25:b5:00:ff:04
20:00:25:b5:ff:00:00:04
100025b5-0000-0000-2000-0a0b0c0d0e04
44
Server Pools
Service profile
Population of server pools
Server pool
Manual administratively defined

Automatic using policies upon discovery
Assigned to a service profile
Discovered, not-associated blades can be assigned

Server pools
Blade pool 1
Chassis1 Slot1
Chassis2 Slot1
Blade pool n
Chassis3 Slot1
Chassis1 Slot3
Chassis1 Slot6
Chassis2 Slot3
Chassis3 Slot3
45
Automatic Server Pool Population

Upon server blade discovery only
Using policies
Server pool qualification policy defines qualification criteria

Server pool policy ties together qualification and pool
Server pools
Blade pool 1
Chassis1 Slot1
Chassis2 Slot1
Blade pool n
Chassis3 Slot1
Chassis1 Slot3
Chassis1 Slot6
Chassis2 Slot3
Chassis3 Slot3
Server pool qualification policy: ESX

Min 32 GB RAM
2 CPU
Server pool policy: ESX_pool
Qualification: ESX
Pool: Blade pool 1
46
Policies Overview
Policy driven management of UCS
components
Service profile
Ensures consistent computing

environment
Policy
Determines how UCS component act in
a given circumstances
Separates functions within a system
Different polices defined for network,
storage, server
Used within service profiles

Boot order
SAN
Local drive
CD-ROM
PXE
Ethernet adapter
Queue depth
Failover timeout
Performance
Blade collection
...
Intervals for collection and

reporting blades statistics
Applied
policies
47
Operational Policies
Policy Type
Policy description
Adapter
Assign/Customize adapter Tx/Rx Queues, TCO, RSS, TCP Offload
BIOS
BIOS settings - Turbo Boost, HT, SpeedStep, VT, RAS Memory, Console
Boot
Local Disk, LAN, SAN, CD/DVD
Host Firmware
Assign specific version of firmware to hardware devices, CIMC, Adapters,

Storage Controller, BIOS version
IPMI profile
Server IPMI capability and ro/rw access type
Maintenance
Reboot policy, timer or user-ack
Power Control
Power Capping attributes
Scrub
Defines if servers state to be kept during discovery
Serial over LAN
Serial over LAN server capabilities
Server Pool
Auto-assign hardware to pools based on pre-defined qualifications
48
Part Identification and RMA
Presentation_ID
Cisco Public
49
Part replacement / RMA

TAC needs to troubleshoot before sending an RMA
Follow proper ESD measures while replacing
Let the cisco Field Engineer replace the parts if possible
The SLA for the part replacement starts from the time the RMA is booked
There is a cut off period for the NBD SLA
Few parts need special considerations while replacing
50
Part replacement / RMA (Special Inst.)

Parts with special consideration
CPUs on M3 server have multiple pins. There is a special tool to replace the CPUs.
http://www.intel.com/support/motherboards/desktop/sb/CS-032579.htm
51
Which topic would you like us to cover next on UCS session?

a.
Performance Troubleshooting
b.
LAN/Network Troubleshooting
c.
SAN/Storage Troubleshooting
d.
Server troubleshooting
52
UCS Upgrades
53
Things to consider
Always consult Release Notes as they will cover gotchas and concerns in the upgrade process
Upgrades from one version back will always work
Check release notes about prior versions
If customer is really far behind it might require two upgrades to get to current code
Schedule an maintenance window
FI and IOM will reboot during upgrade

Make sure network and storage fabric are redundant
Highly recommended to backup UCSM configuration
Upgrade Guides and videos located at:
http://www.cisco.com/en/US/products/ps10281/prod_installation_guides_list.html
Upgrade process is not quick
Impossible for QA to test every combination during testing
54
Downgrading
Not always tested
Generally done to rollback in event of issues during upgrade
Can be disruptive : New features may get activated and that will stop
working if downgraded
UCSM Backup is always recommended
55
Downloading UCS Firmware
56
Bundles
Bundles are downloaded from the Download Tasks tab
Downloads can be through desktop or using ftp/scp/sftp/tftp
57
Bundles
Packages can be viewed/deleted from Packages tab
58
Upgrade Types
MANUAL UPGRADE
Manually update each component/endpoint
Manually activate each component/endpoint
Not system controlled : Each operation has to be done manually in the order
specified by the Upgrade Guide(s)
AUTO-INSTALL
This is system controlled
Components do not require to be updated/activated individually
System takes care of the order of operations
59
Upgrade Process-Manual Install
Again always consult release notes
Upgrade through GUI is easiest
General Process is
Backup UCS Config (Full & All Config)
Download code
Update components
Activate components in order of (Check RN cause order can change)
Interface cards Set Startup Only
CIMC
IOM Set Startup Only
UCSM
FI
60
Firmware Management Definition
Update
Copies the firmware to the backup partition on an endpoint
Not disruptive
Can be done to all devices at the same time
Activate
Firmware in the backup partition is set to active (startup version)
Activates the startup version on the endpoint
Disruptive
Follow specific order according to documentation
61
UCS Upgrade Live

Demo
62
Updating Components
Update means copy new code to backup location of all UCSM components
Simply stages the new code
Can update all components at once
63
Updating Components
Time to update will vary based off component
IOMs take a long time. Up to 5 minutes
If any component has issues check FSM for that component
Update process does not work on FI
Once everything is in Ready state you can move to Activate
64
Activate Blade Components
Recommended Method is to use Policies
Host Firmware Policy to apply latest BIOS, Board Controller, Adapters, etc.
The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open
the file again. If the red x still appears, you may have to delete the image and then insert it again.
The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file
again. If the red x still appears, you may have to delete the image and then insert it again.
65
Activate UCSM
Will cause UCSM to disconnect
Takes a few minutes
66
Activate Fabric Interconnect
Recommended to activate one FI at a time
A complete maintenance will not occur
All traffic via one Fabric will fail as the FI reboots to complete the Upgrade
All Network and FC traffic should failover to second Fabric
Highly recommended to have an maintenance window
Part of the process is to reboot connected IOM as well

Can take up to 10-15 minutes for FI and all IOM to come back online
If any failure during first FI upgrade STOP! Do not attempt to upgrade second FI
Activate the other FI
Important Note : Before proceeding to Activate the other FI, please check if all Ethernet and FC traffic
has come UP via the first Fabric. If that is not the case, please STOP and call TAC
67
Firmware Auto-Install
Firmware Auto-Install implements package version based upgrades for both UCS Infrastructure
components and Server components
Firmware Auto-Install can not be used to upgrade Management Extensions and Capability Catalog.
These are simple occasional updates in UCSM and hence left under user control.
It is a two step process - Install Infrastructure Firmware and Install Server Firmware.
It is recommended to run Install Infrastructure Firmware first and then Install Server Firmware
Firmware Auto-Install is available UCSM 2.1 onwards. However it can be used for 1.4/2.0 to 2.1
upgrade. User has to manually upgrade UCS Manager to 2.1 and then start using the feature. This
manual upgrade of UCS Manager is not required for upgrade from 2..1 to post-2.1 versions.
All existing firmware upgrade mechanisms are retained. For users who do not want to use Auto-Install,
they can continue to use existing documented way of doing firmware upgrades.
68
Install Infrastructure Firmware
Install Infrastructure Firmware
This is the sequence followed by Install Infrastructure Firmware
1.
Upgrade UCSM
2.
Update backup image of all IOMs
3.
Activate all IOMs with set startup option
4.
Activate secondary Fabric Interconnect
5.
Wait for User Acknowledgement***
6.
Activate primary Fabric Interconnect
69
Install Infrastructure Firmware User Acknowledgement for

primary FI
Install Infra expects an explicit permission from user to start firmware upgrade on primary Fabric
Interconnect.
This is necessary to protect the data path for servers.
As part of Install Infra, secondary FIs firmware is upgraded first.
Secondary FI reboots as part of firmware activation.
After secondary FI comes online, users are expected to check if the data path is ready for a reboot of
primary FI
When users have ensured that the data path is ready, they can
acknowledge reboot of primary FI.
Auto-Install will NOT check if paths/traffic is UP before it brings up the

option for Primary FI Reboot
70
Acknowledge Primary FI reboot
71
Install Server Firmware Screen 1
72
73
74
75
76
61XX upgrade to 62XX
Official Document
http://www.cisco.com/en/US/docs/unified_computing/ucs/sw/upgrading/
from1.3/to2.0/UpgradingCiscoUCSFrom1.3To2.0_chapter5.pdf
Port Concerns
Since a UCS 6140 has 40 ports on slot 1 and a UCS 6248 has 32 slots, any
ports currently configured on ports 1/33 to 1/40 of the UCS 6140 will have to
be moved during the upgrade process
Remember Unified Ports are hard set. Change requires reboot
Spend a lot of time considering your port options
77
UCS Troubleshooting
78
UCS Upgrade Live

Demo
79
UCS Troubleshooting
80
81
Fault properties-1
Property Name
Description
Severity
Severity of the Fault e.g Critical, Major, Minor,

Warning,Informational.
Last Transition
The day and time on which the severity for the fault last
changed.
Affected Object
Component affected
Description
ID
An Unique identifier number
Type
Type as explained before
82
Fault properties-2
Severity
Description
Cleared At
Day and time when fault created
Code
Unique fault code (Available on CCO)
Number of
Occurrences
Number of times the same fault occurred
Original Severity
Previous Severity
Highest Severity
83
Events per Component
FarNorth-A# scope server ?

WORD
<chassis-id>/<blade-id>
dynamic-uuid Dynamic UUID
FarNorth-A# scope server 1/1
FarNorth-A /chassis/server # show event
84
84
UCSM Faults - GUI
85
Unique-Id
86
Affected-component
Major Fault
Information Fault
87
Fault Status : CLI
88
Agenda
UCSM &
Fabric Interconnect
Blade Servers
IOM & Chassis
89
Contexts
UCS has three CLI Contexts
UCSM (GUI Equivalent, uses the scope command)

NXOS (not configurable read only)
Management (file management, tech support, reboot)
UCSM
NXOS
Local-Management
90
Scope
Scoping movement to different UCS configuration components
Details on hardware components
done with connect command
You want to be on the

Primary Fabric Interconnect
UCS-B# scope ?

adapter Mezzanine Adapter
chassis Chassis
eth-server Ethernet Server Domain
eth-storage Ethernet Storage
eth-traffic-mon Ether Traffic Monitoring Domain
eth-uplink Ethernet Uplink
fabric-interconnect Fabric Interconnect
fc-storage FC Storage
fc-traffic-mon FC Traffic Monitoring Domain
fc-uplink FC Uplink
fex FEX (fabric-extender) Module
firmware Firmware
host-eth-if Host Ethernet Interface
host-fc-if Host FC Interface
license License
monitoring Monitor the system
org Organizations
power-cap-mgmt Power Cap Mgmt
security security mode
server Server
service-profile Service Profile
system Systems
vhba vHBA
91
vnic vNIC
Management Commands (scope, where, up & top)

UCSM Navigation
CLI Equivalent to Nav Pane
92
Connect - Hardware Troubleshooting

Connect attaches you to hardware
and read only NXOS
FarNorth-B# connect
adapter
bmc
clp
iom
local-mgmt
nxos
Mezzanine Adapter
Baseboard Management Controller (CIMC)
Connect to DMTF CLP
IO Module
Connect to Local Management CLI
Connect to NXOS CLI
FarNorth-A# connect local-mgmt

<CR>
a
Fabric A
Defaults to primary
b
Fabric B
Most dangerous
-erase configuration
- reboot
FarNorth-A(local-mgmt)# ?
cd
Change current directory
clear
Reset functions
cluster
Cluster mode
connect
Connect to Another CLI
copy
Copy a file
cp
Copy a file
delete
Delete managed objects
dir
Show content of dir
enable
Enable
end
Go to exec mode
erase
Erase
erase-log-config Erase the mgmt logging config
file
exit
Exit from command interpreter
install-license Install a license
ls
Show content of dir
mkdir
Create a directory
move
Move a file
mv
Move a file
ping
Test network reachability
pwd
Print current directory
reboot
Reboots Fabric Interconnect
rm
Remove a file
rmdir
Remove a directory
run-script
Run a script
show
Show running system information
ssh
SSH to another system
tail-mgmt-log Tail mgmt log file
telnet
Telnet to another system
terminal
Set terminal line parameters
top
Go to the top mode
traceroute
Traceroute to destination
93
93
Connect NXOS
Used to assist in troubleshooting very familiar to IOS and Nexus users and all the show
commands
Used to run advised debugs
Show switch running config (non server config)
Clear interface counters found on the FI
Cannot be used to configure UCS (read only)
94
Connect to NXOS
FarNorth-A# connect nxos ?
<CR>
a Fabric A
b Fabric B
FarNorth-A(nxos)# ?
clear Reset functions [Only place to clear counters]
cli CLI commands
debug Debugging functions
debug-filter Enable filtering for debugging functions
ethanalyzer Configure cisco packet analyzer
interface A live capture will start on following interface
no Negate a command or set its defaults
ntp NTP configuration
show Show running system information
system System management commands
terminal Set terminal line parameters
test Test command
undebug Disable Debugging functions (See also debug)
end Go to exec mode
exit Exit from command interpreter
pop Pop mode from stack or restore from name
push Push current mode to stack or save it under name
where Shows the cli context you are in
Popular examples:
show run
show fex detail
show interface
show lacp
show trunk
show cdp
debug
show npv flogi-table
show mac-address-table
95
Connect local-mgmt
Connect attaches you to hardware
FarNorth-A(local-mgmt)# ?
cd Change current directory
clear Reset functions
cluster Cluster mode
connect Connect to Another CLI
FarNorth-B# connect ?
copy Copy a file

cp Copy a file
adapter Mezzanine Adapter
delete Delete managed objects
cimc Cisco Integrated Management Contr. dir Show content of dir
enable Enable
clp Connect to DMTF CLP
end Go to exec mode
iom IO Module
erase Erase
erase-log-config Erase the mgmt logging
local-mgmt Connect to Local Management CLI
exit Exit from command
nxos Connect to NXOS CLI
install-license Install a license
ls Show content of dir
FarNorth-A# connect local-mgmt ?
mkdir Create a directory
<CR>
move Move a file
a Fabric A [Defaults to primary]
mv Move a file
ping Test network reachability
b Fabric B
pwd Print current directory
reboot Reboots Fabric Interconnect
Dangerous Commands!!!
rm Remove a file
erase configuration
rmdir Remove a directory
96
2013-2014 Cisco and/or its affiliates.
All rights reserved.
reboot
run-script Run a script
show Show running sys info
and read only NXOS
Chassis Discovery Policy

Discovery policy only defines the minimum number of links necessary before a
chassis can be discovered and NOT how many links will be utilized
97
97
KVM
Tool to snapshot screen for support
Doing Web-ex recording best
98
98
Logs for troubleshooting

General UCS issues
UCS-A(local-mgmt)# show tech-support ucsm detail
UCS-A(local-mgmt)# show tech-support chassis # all detail
Networking Issues
Upstream_Switch# show tech-support details
SAN Issues
UCS-A(nxos)# show tech-support npv
MDS# show tech-support details
99
UCSM Access Over Firewall
Ports to be opened for Accessing UCSM Over firewall
TCP 21 if FTP will be used for image/backup transfer

TCP 22
TCP 23 if telnet is enabled (off by default)
TCP/UDP 53 DNS Resolution
UDP 123 NTP
TCP 80
UDP 161/162 if SNMP is enabled (off by default)
TCP 443 if https is enabled (off by default)
UDP 514 is syslog is enabled
UDP 623 if IPMI/SOL access is required
TCP 2068 (KVM)
100
UCSM Top 5 commands
UCS-A# show cluster extended-state

UCS-A /fabric-interconnect # show fsm status
UCS-A (local-mgmt) # show pmon state
UCS-A /monitoring/sysdebug # show cores
UCS-A (nxos) # show mgmt-ip-debug
101
Sample Cluster state
102
Sample Cluster state
103
Agenda
UCSM &
Fabric Interconnect
Blade Servers
IOM & Chassis
104
Blade servers
Blade overview CIMC and BIOS
CIMC
Monitors Temperature and
Power readings
KVM & vMedia
Blade control
BIOS
Can be configured via F2 or via
BIOS Policy
105
OBFL
Onboard Fault Log stores hardware logs on the different components, saved at time
of issue.
Alternate method to viewed by connecting to the internal component end device.
Show tech-support will capture required logs for support.
106
106
System Event Log (SEL) - Events Supported

Server BIOS events
3 Kinds of equipment end-points:

Memory Unit (DIMM) ECC errors, Address Parity, Memory Mismatch
Processor Unit Memory Mirroring, Sparing, SMI Link errors
Motherboard PCIe, QPI uncorrectable errors, Legacy PCI errors
All these errors are modeled as stats properties. The ones for which thresholds are not defined get
reported as statistics only
BMC, BIOS, OS log platform errors to CIMCs System Event Log (SEL)
Buffer
POST and Run Time errors
Used as an Effective health monitoring tool
107
107
System Event Logs

Chassis
Make sure that servers are discovered

Make sure backup destination path is valid
Can be done via CLI also
System Event Logs = Management Logs on earlier releases
Server
108
108
System Event Log (SEL) - config

Users can define rules (policies) for backing up and clearing SEL across all servers
in the UCS system, or they can manually trigger a SEL backup on individual servers.
109
109
System Event Log (SEL) - config

Users can define rules (policies) for backing up and clearing SEL across all servers
in the UCS system, or they can manually trigger a SEL backup on individual servers.
110
110
OS connectivity issue (Eth+SAN)

Few important things to check:
Is the blade running the certified OS and OS version?

Are there any special needs for that OS? E.g. VMWare OEM Image
Are the drivers at the OS level updated and current?
Answer:
UCS S/W and H/W matrix

http://www.cisco.com/en/US/products/ps10477/prod_technical_reference_list.html
111
112
What each matrix provides
113
Sample..driver versions
114
Methods for Troubleshooting Memory

Collect Tech-support files after errors produced
Investigate SEL and Fault Log
Confirm DIMM configuration supported
Follow Troubleshooting Flow (See appendix)
Utilize DIMM Error Statistics
Access Error Manager in BIOS Menu
Discover any recent changes
Use Memory Test Tools to validate DIMM ECC errors if possible
http://www.memtest86.com/
115
Memory Errors and Causes

Memory Errors
ECC Error
Parity Error
SPD (Serial Presence Detect) Error
Configuration Error
Unpaired DIMMs
Not supported DIMMs
Not supported DIMM population
Causes of Memory Error

Defected by Hardware
Defected DIMM
Broken DDR3 connectivity
Defected Hardware (CPU, Chipset)
Power
Thermal
Defected by Software
BIOS
BMC
UCSM
116
Memory Errors Viewed in UCSM

CiscoLive-A /chassis/server # show inventory memory detail
117
Viewing Memory Issues in SEL

UCS-A# show sel 1/1 | include memory
UCS-A# scope chassis 1
UCS-A /chassis # scope server 1
UCS-A /chassis/server # sh sel | include memory
CiscoLive-A /chassis/server # show sel 3/1 | include Memory
487 | 03/18/2011 00:16:49 | BIOS | Memory #0x02 | Uncorrectable ECC/other uncorrectable memory error | RUN, Rank: 0, DIMM Socket: 4,
Channel: C, Socket: 0, DIMM: C4 | Asserted
5f1 | 04/16/2011 09:53:12 | BIOS | Memory #0x02 | Uncorrectable ECC/other uncorrectable memory error | RUN, Rank: 3, DIMM Socket: 7,
Channel: A, Socket: 0, DIMM: A7 | Asserted
731 | 04/21/2011 01:59:28 | BIOS | Memory #0x02 | Correctable ECC/other correctable memory error | RUN, Rank: 1, DIMM Socket: 1,
Channel: B, Socket: 0, DIMM: B1 | Asserted
732 | 04/21/2011 10:50:55 | BIOS | Memory #0x02 | Uncorrectable ECC/other uncorrectable memory error | RUN, Rank: 2, DIMM Socket: 6,
Channel: A, Socket: 0, DIMM: A6 | Asserted
799 | 04/29/2011 02:50:31 | BIOS | Memory #0x02 | Correctable ECC/other correctable memory error | RUN, Rank: 0, DIMM Socket: 0,
79a | 04/29/2011 04:41:33 | BIOS | Memory #0x02 | Uncorrectable ECC/other uncorrectable memory error | RUN, Rank: 3, DIMM Socket: 3,
118
The image
cannot be
displayed.
Your
computer
may not
have
enough
memory to
UCS B200/B250 Memory Status Reporting
For Your Reference

UCSM
BMC
BIOS
DIMMStatus
Operability
Presence
DIMM LED
Operable
Operable
Equipped
OFF
Operable
Degraded
Degraded
Inoperable
Equipped
Equipped
Missing
OFF
Disabled
Equipped
OFF
Disabled
Inoperable
Inoperable
Equipped
Equipped
Equipped
ECC State
Comment
Installed
POST
NA
RUN
A DIMM is installed and functional.

CE
ON / OFF
Time
ON / OFF
Removed
Disabled
Configuration State
A correctable ECC DIMM error is detected during run time.

CE
Installed
POST
Not Installed
POST
Disabled
POST
Ignored
POST
Fail
POST
Fail
POST
OFF
OFF
ON
DIMM status and Operability changed due to ECC errors were

detected before host rebooted.
A DIMM is not installed or corrupted SPD data.
A DIMM may be healthy but disabled because configuration rule

could not be maintained by a failed DIMM in the same channel.
Failed to follow memory configuration rule because of missing the

first paired DIMM.
Failed to follow memory configuration rule because of missing

DIMMs.
UE
UE ECC Error was detected.
119
Blade Component Discovery
Presentation_ID
Cisco Public
120
Blade Discovery Failure Causes
No Power to Blade
POST Failure
PNOUs Not Booting
Bad or Missing Hardware
Memory
Mezz Cards
CPU
121
No Power or Bad power

Connect to CIMC and verify ability to connect
Check Led indicators for faults
If able to connect list Power & Sensors
UCS-A# connect cimc 1/1
Trying 127.5.1.1...
Connected to 127.5.1.1.
Escape character is '^]'.
CIMC Debug Firmware Utility Shell [ support ]
[ help ]# power
OP:[ status ]
Power-State:
[ on ]
VDD-Power-Good:
[ active ]
Power-On-Fail:
[ inactive ]
Power-Ctrl-Lock: [ unlocked ]
Power-System-Status: [ Good ]
OP-CCODE:[ Success ]
[ power ]#
__________________
Debug Firmware
Utility
alarms
cores
exit
help [COMMAND]
images
mctools
memory
messages
network
obfl
post
power
sensors
sel
fru
mezz1fru
mezz2fru
tasks
top
update
users
version
..
The image cannot be displayed. Your computer may not have enough memory to open the image, or the image may have been corrupted. Restart your computer, and then open the file again. If the red x
still appears, you may have to delete the image and then insert it again.
122
Power on Self Test Results Can Cause Blade Failure

View last POST test results
Change the default enabled
Quite Boot option to Disable

so to view post testing, reboot
blade and monitor with KVM
console
Errors here
123
Blade servers Common issues

Service profile association failed
Insufficient resources
Uplink connectivity issues
124
Blade Startup Utility O/S

Utility O/S must boot and complete for
blade to come up error free
Monitor the O/S in Message Status and
KVM
If Utility cannot complete Logs and
techsupport outputs need to be viewed for
issues
Utility O/S several screens
125
Utility O/S - PNOUS
126
Chassis Slot Resolution

Issue can arise when moving blades in the Chassis
127
Blade servers Top 5 commands
UCS-A /chassis/server # show inventory expand detail

UCS-A /chassis/server # show status detail
UCS-A /chassis/server # show post
UCS-A /chassis/server # show sel
UCS-A /chassis/server# show fsm status
128
Agenda
UCSM &
Fabric Interconnect
Blade Servers
IOM & Chassis
129
IOM & Chassis

Overview
1-4
Fabric links
To
Interconnect
FLASH
CMC responsibilities
Chassis Discovery
Local cluster
management
Power & Thermal
Management
DRAM
EEPROM
Chassis
Management
Controller
Control
IO
Redwood ASIC
Switch
Chassis
Signals
To Blades
130
IOM & Chassis Common issues

Chassis not discovering
UCS-A(nxos)# show run interface
Check chassis discovery policy

Server ports defined correctly
FI to IOM 1:1 relationship only
ethernet x/y
UCS-A(nxos)# show interface fex-fabric
UCS-A(nxos)# show fex <chassis#> detail
131

Fan issues
UCS-A# connect iom 1
Spinning at 100%
Temperature
Any fans missing?
CMC access to thermal sensors
Component discovery
fex-1# show platform software cmcctrl thermal status
fex-1# show platform software cmcctrl fancontrol all
fex-1# show platform software cmcctrl ohms all
132

Power issues
Power Policy
Grid, N+1 or non redundant
Power cap issues
fex-1# show platform software cmcctrl power manager
133
IOM & Chassis Top 5 commands

UCS-A(nxos)# show fex detail
fex-1# show platform software cmcctrl cmc manager
fex-1# show platform software cmcctrl thermal status
fex-1# show platform software cmcctrl obfl logs
fex-1# show platform software cmcctrl pstate
134
Ethanalyzer Tool Usage

Uses Wireshark utility to view FI control data and Management traffic
Ethanalyzer is a tool that will collect frames that are destined to, or originate
from the FI control plane. Node to FI, or FI to Network traffic can be seen with
this tool.
Need to be connected to NXOS
Capture Options
CiscoLive-A# connect nxos
<CR>
a Fabric A
b Fabric B
CiscoLive-A(nxos)# ethanalyzer local sniff-interface ?
inbound-hi Inbound(high priority) interface
inbound-low Inbound(low priority) interface
mgmt
Management interface
>
Redirect it to a file
>>
Redirect it to a file in append mode
capture-filter
Filter on ethanalyzer capture
decode-internal
Include internal system header decoding
detailed-dissection Display detailed protocol information
display-filter
Display filter on frames captured
dump-pkt
Hex/Ascii dump the packet
limit-captured-frames Maximum number of frames to be captured
limit-frame-size
Capture only a subset of a frame
write
Filename to save capture to
|
Pipe command output to filter
135
135
Ethernet Interfaces on CPU Troubleshooting Uses

Ethanalyzer terminology, internal ethernet interfaces are used:
eth3 = inbound-lo
eth4 = inbound-hi
eth3 handles Rx and Tx of low priority control pkts

IGMP, CDP
TCP/UDP/IP/ARP (for management purpose only)
eth4 handles Rx and Tx of high priority control pkts

FC (FC packets come to Switch CPU as FCoE packets) and FCoE
STP (spanning-tree) , LACP, DCBX (Data Center Bridging)
Save to file and use Wireshark tool to help diagnose issue

CiscoLive-A(nxos)# ethanalyzer local sniff-interface inbound-hi limit-captured-frames 100 write volatile:///ciscolive11
Capturing on eth4
100
CiscoLive-A(nxos)# exit
CiscoLive-A(local-mgmt)# copy volatile:///ciscolive11 tftp:
CiscoLive-A# connect local-mgmt a
Server name/IP: 10.91.42.136
CiscoLive-A(local-mgmt)# cd volatile:/
Remote username: CAE
CiscoLive-A(local-mgmt)# ls
Remote filepath: \
136
136
UCSM and Chassis show tech from GUI

Log into the UCSM GUI
Select the admin tab -> faults, Audit and event-logs section -> Tech Support
File
137
List on known bugs in the releases:

2.2(1b)
CSCug93342 FI is going to loader prompt while performing image upgrade. Fixed 2.2(1b)
2.1(3b) 2.0(5f)
CSCun00720 / CSCud91244 / CSCtz92512 / CSCue45276 / CSCun66310 FI Completely
unresponsive after upgrade
CSCum75266 Upgrade to 2.2.1b results in FI-B reload. Boots up to bash prompt
2.1(3a)
CSCuj79533 / CSCub51662 UCS Upgrade to 2.1(3a) Infra, Multiple Servers Go Into Compute
Failed Fixed 2.1(1b) 2.0(5a) 2.0(4a)

CSCug93342 FI is going to loader prompt while performing image upgrade Fixed 2.2(1b),
2.1(3b), 2.0(5f)
138
Where to find more information

Hardware Installation & Service Guides Information
http://www.cisco.com/en/US/docs/unified_computing/ucs/overview/guide/
UCS_roadmap.html#wp38892
Release Notes
http://www.cisco.com/en/US/products/ps10281/prod_release_notes_list.html
Software Upgrade & Installation Information
http://www.cisco.com/en/US/products/ps10281/prod_installation_guides_list.html
UCS Troubleshooting Guide
http://www.cisco.com/en/US/docs/unified_computing/ucs/ts/guide/UCSTroubleshooting.html
UCS Faults Reference
http://www.cisco.com/en/US/docs/unified_computing/ucs/ts/faults/reference/ErrMess.html
Cisco Support Community
https://supportforums.cisco.com/community/netpro/data-center/unified-computing
139
Do you have Nexus 1000v installed on a UCS environment?

a.
I have UCS but no Nexus 1000v on it
b.
I have Nexus 1000v but not on UCS
c.
I am planning to have Nexus 1000v on UCS
d.
I am using Nexus 1000v on UCS
140
Submit Your Questions Now!
Use the Q & A panel to submit your questions and our expert will respond
If you have additional questions, you can ask Varun

Mehta and Anupam Asthana. They will
answering from September 17 through Wednesday,
September 24th.
https://supportforums.cisco.com/discussion/12302271/ask-expertcisco-unified-computing-system-ucs-b-series-upgrade-troubleshooting
You can catch the video or read the Q&A five business days after the event at
https://supportforums.cisco.com/expert-corner/knowledge-sharing
What Does Sonic the Hedgehog and Ciscos Unified Computing System
Share in Common?
a.
Sega uses Cisco Unified Computing System as the server infrastructure for simultaneously
supporting other small-scale games. Despite being fully virtualized, the server integration ratio is
much lower than that for their number one game Phantasy Star Online 2 environment.
b.
Sega started searching for a new server foundation for POS2 back in 2010 with the assumption that
server virtualization would be introduced. Ironically, despite the long search, Cisco was not one of
Segas top choices.
c.
Gaming Company Sega, famous for their popular game Sonic the Hedgehog, uses Ciscos Unified
Computing System to maximize memory capacity for the online gaming infrastructure. This helps to
increase the number of virtual servers to keep pace with the increased number of simultaneous
connections.
143
ESA Configuration and

Troubleshooting Training
September 22, 2014
10:00 AM PST - 5:00 PM EST
Connected Analytics for

Network Deployment, with
Cisco subject matter expert,
June Zheng
October 21, 2014
10:00 AM PST - 5:00 PM EST
The ESA Content Security team is happy to announce our

upcoming Live Virtual training session for our Email Security
Appliance.
During this live event, subject matter expert

June Zheng will cover a number of advanced
topics and live demo on the Connected
Analytics for Network Deployment.
Registration for these live webcasts:

Visit the Webcast tab > https://supportforums.cisco.com/expert-corner/knowledge-sharing
144
Configuring and Troubleshooting 802.1X open

questions
Join Cisco Expert: Javier Henderson
This is an opportunity to learn and ask questions about how to configure and
troubleshoot 802.1X.
802.1X is an IEEE standard for media-level access control, offering the capability to
permit or deny network connectivity, control VLAN access, and apply traffic policy,
based on user or machine identity. During this event, Javier Henderson will answer all
your questions regarding 802.1X configuration and troubleshooting.
September 13th - September 26th, 2014
Join the discussion for these Ask The Expert Events:

https://supportforums.cisco.com/community/netpro/expert-corner#view=ask-the-experts
145
http://bit.ly/1jlI93B
146
https://supportforums.cisco.com/expert-corner/top-contributors
147
http://www.facebook.com/CiscoSupportCommunity
http://twitter.com/#!/cisco_support
http://www.youtube.com/user/ciscosupportchannel
https://plus.google.com/110418616513822966153?prsrc=3#110418616513822966153/posts
http://www.linkedin.com/groups/CSC-Cisco-Support-Community-3210019
Newsletter Subscription
https://tools.cisco.com/gdrp/coiga/showsurvey.do?
surveyCode=589&keyCode=146298_2&PHYSICAL%20FULFILLMENT%20Y/
N=NO&SUBSCRIPTION%20CENTER=YES
http://itunes.apple.com/us/app/cisco-technical-support/id398104252?mt=8
https://play.google.com/store/apps/details?id=com.cisco.swtg_android
148
Spanish https://supportforums.cisco.com/community/spanish
Portuguese https://supportforums.cisco.com/community/portuguese
Japanese https://supportforums.cisco.com/community/csc-japan
Russian https://supportforums.cisco.com/community/russian
New Chinese Community!

Chinese http://www.csc-china.com.cn/
149
Now your ratings on documents, videos, and blogs count

give points to the authors!!!
So, when you contribute and receive ratings you now get
the points in your profile.
Help us to recognize the quality content in the community
and make your searches easier. Rate content in the
community..
https://supportforums.cisco.com/blog/154746
150
More IT Training Videos

& Tech Seminars
on the Cisco Learning Network

View Upcoming Sessions Schedule
cisco.com/go/techseminars
151
What Does Sonic the Hedgehog and Ciscos Unified Computing System
Share in Common?
a.
Sega uses Cisco Unified Computing System as the server infrastructure for simultaneously
supporting other small-scale games. Despite being fully virtualized, the server integration ratio
is much lower than that for their number one game Phantasy Star Online 2 environment.
b.
Sega started searching for a new server foundation for POS2 back in 2010 with the
assumption that server virtualization would be introduced. Ironically, despite the long search,
Cisco was not one of Segas top choices.
c.
Gaming Company Sega, famous for their popular game Sonic the Hedgehog, uses Ciscos
Unified Computing System to maximize memory capacity for the online gaming infrastructure.
This helps to increase the number of virtual servers to keep pace with the increased number
of simultaneous connections.
The answer is C: Cisco Unified Computing System was eventually chosen due to its ability to run
all 400 of Segas servers.
152
Thank you for Your Time!

Please take a moment to complete the evaluation
Thank you.

Cisco Support Community Expert Series Webcast

Uploaded by

Copyright:

Available Formats

Cisco Support Community Expert Series Webcast

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Cisco Support Community Expert Series Webcast

Uploaded by

Copyright:

Available Formats

Cisco Support Community Expert

Todays featured experts is Cisco Support Engineers

Ask questions now, about Unified Computing System, Upgrade &

Customer Support Engg.

Customer Support Engg.

2013-2014 Cisco and/or its affiliates. All rights reserved.

Event Date: 16th Sep 2014

2013-2014 Cisco and/or its affiliates. All rights reserved.

Todays presentation will include audience polling questions.

2013-2014 Cisco and/or its affiliates. All rights reserved.

2013-2014 Cisco and/or its affiliates. All rights reserved.

Do you use UCS in your company infrastructure?

Yes, I am using it.

Not yet, but I plan to use in future.

No, I am not using it.

2013-2014 Cisco and/or its affiliates. All rights reserved.

Submit Your Questions Now!

Cisco Unified Computing System (UCS) B series

DATE : 16th SEPTEMBER 2014

Introduction to Unified Computing System, Upgrade & Troubleshooting

UCS Architecture in brief

2013-2014 Cisco and/or its affiliates. All rights reserved.

2013-2014 Cisco and/or its affiliates. All rights reserved.

2013-2014 Cisco and/or its affiliates. All rights reserved.

2010 Cisco Systems, Inc. All rights reserved.

UCS Fabric Interconnect Overview

In addition to NXOS runs UCS Manager (UCSM)

Server ports connect to IO Modules in the blade chassis

One Management port connects to the customers Out-Of-Band management network

RS-232 serial port ( Console )

2013-2014 Cisco and/or its affiliates. All rights reserved.

UCS 6248 FUCS 6248 Fabric Interconnect

Cisco UCS Fabric Infrastructure

High End Deployments

48 Port Fabric Interconnect

96 Port Fabric Interconnect

Performance for typical deployments

High Application performance

Infrastructure agility with Unified Ports

Infrastructure agility with Unified Ports

2204XP I/O Module

80G per chassis

2208XP I/O Module

UCS 6200: Unified Ports

Ethernet or Fiber Channel

Native Fibre Channel

1GbE or 10GbE, FCoE, iSCSI, NAS

Simplify switch purchase. Removes

Flexible LAN & storage convergence based on

Increases design flexibility

Remove specific protocol bandwidth

Service can be adjusted based on the demand

2013-2014 Cisco and/or its affiliates. All rights reserved.

2014 Cisco and/or its affiliates. All rights reserved.

Unified Computing System Manager

Embedded device manager for family of UCS components

Single point of device management

Standard APIs for systems management

Adapters, blades, chassis, LAN and SAN connectivity

XML, SMASH-CLP, WSMAN, IPMI, SNMP