EMC ® VNX ™ Procedure Generator

FCO F041211FC

Topic

Selections
VNX FCO Procedures: F041211FC for VNX5700/7500

Procedure Generated: 2022/10/10 20:57:48

IMPORTANT: To be sure you have the latest procedure generator application and content, always
choose Yes when asked if you want to update the application or repository.

REPORT PROBLEMS

If you find any errors in this procedure or have comments regarding this application, send email to
[email protected]

Copyright© 2011 EMC Corporation. All rights reserved.

Publication Date: month,_year
EMC believes the information in this publication is accurate as of its publication date. The information is subject to
change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” EMC CORPORATION MAKES NO
REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS
PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS
FOR A PARTICULAR PURPOSE.
Use, copying, and distribution of any EMC software described in this publication requires an applicable software
license.
For the most up-to-date regulatory document for your product line, go to the Technical Documentation and Advisories
section on EMC Powerlink.
For the most up-to-date listing of EMC trademarks, see the list of EMC Corporation Trademarks on EMC.com.
All other trademarks used herein are the property of their respective owners.

EMC CONFIDENTIAL version: 3.1

1 of 9
 EMC ® VNX ™ Procedure Generator

Contents
Problem Synopsis..................................................................................................................................... 2

EMC Field Service Script and Instructions

Date: October 5, 2011

EMC FCO #F041211FC

Part Numbers Affected: 110-113-418B and 110-113-724B Storage Processors
Platforms Affected: VNX5700 and VNX7500

CAUTION: You must be running 05.31.000.5.012 prior to executing this FCO or Data
Loss will result. Do not proceed with this FCO if you are running any other
version of code

FCO Action: Brief summary of action required by field

Activity Type: on-line/off-line: Preferred: Prior to installation.

Estimated Procedure Duration: Prior to Installation: This will add approximately 15
minutes to the on-site time. After Installation: plan on 1.5 hours to ensure proper failover
and fail back during and after part replacement. Travel and Admin will add additional
time.

This document is a script for use by Field Service personnel in communicating the goals
and objectives of this FCO to customers and to provide instructions on how to execute
this FCO. The document is intended for use by EMC personnel and the field staff of
EMC’s Authorized Service Providers. Use of this script will assure consistent messaging
and execution of this FCO globally. This script is an internal document and not intended
to be shown or given to customers.
Problem Synopsis
The two Storage Processors on a small number of VNX5700 and VNX7500 systems were
shipped with firmware that will prevent a successful software maintenance release upgrade.
EMC has verified that your Storage Processors have this firmware and have to be replaced prior
to a maintenance release upgrade. The two replacement Storage Processors will contain the
firmware needed to allow an upgrade to a maintenance release version. Failure to replace both
Storage Processors will mean this system will only be able to run the current software and patch

EMC CONFIDENTIAL version: 3.1

2 of 9
 EMC ® VNX ™ Procedure Generator

releases. The first maintenance release, version 05.31.000.5.502, will be released on

September 12, 2011. The Storage Processors have to be replaced before upgrading to
05.31.000.5.50x. Note: upgrading to any version <05.31.000.5.50x can be performed without
replacing the Storage Processors.

Platforms Affected
VNX5700 and VNX7500

Corrective Action Plan

Only systems on the CAL are affected. The two Storage Processors must be replaced prior to
an upgrade to Maintenance Release 05.31.000.5.50x. For the best in customer satisfaction, the
preferred solution is to replace both Storage Processors at time of Installation.
Contact Logistics and order the appropriate part:
VNX5700 Storage Processor with DIMMs: order two (2) 110-113-418B
VNX7500 Storage Processor with DIMMs: order two (2) 110-113-724B
In order to facilitate this FCO, Logistics will supply Storage Processors with DIMM memory.
Prior to powering the array, remove the two Storage Processors, inset the two new Storage
Processors, and power up the system using the standard VNX installation tools and following
standard installation documentation. For more information on replacing the Storage Processors,
refer to the VNX Procedure Generator VNX5700 and VNX7500 Hardware Replacement
Procedure entitled “SPE CPU Module Replacement.”
Replacement after Installation:
Contact Logistics and order the appropriate part:
VNX5700 Storage Processor with DIMMs: order two (2) 110-113-418B
VNX7500 Storage Processor with DIMMs: order two (2) 110-113-724B
In order to facilitate this FCO, Logistics will supply Storage Processors with DIMM memory.
Follow the VNX5700 or VNX7500 Hardware Replacement Procedure entitled “SPE CPU
Module Replacement” as published the VNX Procedure Generator, except the DIMM
replacement steps need not be followed. This procedure will be repeated for the second SP
replacement. Be sure to wait approximately 10 minutes for the first SP to be fully operational
before beginning to replace the second SP.

Field Objectives in Communicating this FCO to Customers

This customer received one of the first EMC VNX5700 or VNX7500 systems. Shortly after
shipment EMC released new BIOS 7.20 and POST 36.81 that will be needed to complete a
software maintenance release upgrade. Unfortunately, this system’s firmware version cannot be
upgraded in the field. The Storage Processors have to be changed before maintenance release
upgrades. After the new firmware is provided, future firmware changes will be able to be made
via a software upgrade.
Customers may attempt to upgrade the software on the VNX systems any time
after September 12, 2011. In order to ensure the customer can successfully upgrade after
September 12, 2011, EMC would like to replace the Storage Processors as soon as
possible.
FCO F042711FE requires applying patch VNX OE for Block patch version
05.31.000.5.007 or above to affected systems. The Storage Processors affected by this FCO do
not have to be replaced prior to applying the patch. The new SPs are only needed for upgrades
to Maintenance Release 05.31.000.5.502 and later. 05.31.000.5.502 is available for customer
download from Powerlink as of September 12.

EMC CONFIDENTIAL version: 3.1

3 of 9
 EMC ® VNX ™ Procedure Generator

Customer Statement Regarding FCO Objectives (for use in discussing with

customers)
EMC takes product quality and data availability very seriously. We believe it is part of
EMC’s value proposition to our customers to keep your systems up to date with current
quality enhancements. EMC releases FCOs to address issues we feel may affect this
goal. EMC’s commitment to the business of our existing customers is part of what
makes the company the industry leader in enterprise data storage and management.

EMC considers this FCO to be mandatory and strongly suggests that you schedule the
implementation of this FCO promptly.

Field Instructions

Storage Processor Replacement at time of initial system installation.

Contact Logistics and order the appropriate part:
VNX5700 Storage Processor with DIMMs: 110-113-418B (quantity 2 per VNX5700)
VNX7500 Storage Processor with DIMMs: 110-113-724B (quantity 2 per VNX7500)
Prior to powering the array, remove the two Storage Processors, inset the two new Storage
Processors, and power up the system using the standard VNX installation tools and following
standard installation documentation.
For more information on replacing the Storage Processors, refer to the VNX Procedure
Generator VNX5700 and VNX7500 Hardware Replacement Procedure entitled “SPE CPU
Module Replacement.”

EMC CONFIDENTIAL version: 3.1

4 of 9
 EMC ® VNX ™ Procedure Generator

Storage Processor Replacement after initial system installation:

Contact Logistics and order the appropriate part:
VNX5700 Storage Processor with DIMMs: 110-113-418B (quantity 2 per VNX5700)
VNX7500 Storage Processor with DIMMs: 110-113-724B (quantity 2 per VNX7500)
[1] Download the VNX Procedure Generator (updated after 06-Sept-2011).
[2] Generate the VNX5700 or VNX7500 “SPE CPU Module Replacement” procedure. Notice
the replacement procedure requires the VNX system to be running 05.31.000.5.012.
Where to find Patch 12:
Path: https://supportbeta.emc.com > Downloads > VNX Series > CX5-Bundle-05.31.000.5.012.pbu
Link: Patch 012 download
Alternate path to the same file: Powerlink > EMC Services Partner Web > Software >
Software Type: VNX OE Software > CX5-Bundle-05.31.000.5.012.pbu
The existing SP may be upgraded to 05.31.000.5.012 without issue. It is upgrades to
05.31.000.5.502 and greater that require the new storage processors.
[3] Replace the first SP by following the VNX5700 or VNX7500 “SPE CPU Module
Replacement” Hardware Replacement Procedure, except the DIMM replacement steps
need not be followed.
[4] Near the end of the procedure notice the requirement to generate a clean system health
check report from the “Verify Storage System” wizard. This step cannot be skipped. Do not
remove the second SP until receiving a clean heath check on the first SP that was just
replaced.
[5] Replace the second SP by following the same “SPE CPU Module Replacement” procedure
in Step 1.

Summary

 Replace both Storage Processors on the affected systems as soon as possible, ideally prior
to installation. This will likely require applying patch .012 before the hardware replacement.
 The BIOS and POST issue addressed by this FCO only affects VNX5700 and VNX7500
Storage Processors. It does not affect any other VNX product and does not affect the Data
Movers.
 Customers will expect to be able to upgrade the VNX OE software as soon as it is posted to
Powerlink. It will not be possible for the customer to perform a VNX OE Software upgrade
on this system until this FCO activity is closed.
 VNX OE for Block patch version 05.31.000.5.012 and lower can be applied without replacing
the Storage Processors.

Frequently Asked Questions:

Question: Can I apply the CDES and Dirty Cache patch without SP change?
Answer: Yes. Software patches can be applied to a VNX5700 and VNX7500 without changing
the Storage Processors. Starting on September 12, customers will be able to upgrade to the first
VNX Maintenance Release. It is the Maintenance Releases and future Major Releases that
require the new Storage Processor.
The patch that resolves the FCO F042711FE CDES issue and Dirty Cache issue is:
VNX OE for Block version 05.31.000.5.007 and greater.
The patch that resolves a potential DUDL during an SP replacement is:
VNX OE for Block version 05.31.000.5.012 and greater.

EMC CONFIDENTIAL version: 3.1

5 of 9
 EMC ® VNX ™ Procedure Generator

The first Maintenance Release will be:

File code 7.0.30.x
Block code 5.31.000.5.502 (notice the change from “0” to “5”)

Question: The VNX is running 05.31.000.5.007. I want to upgrade the system to

05.31.000.5.502 immediately after SP replacement. Do I really have to upgrade to
05.31.000.5.012 before replacing the SP?
Answer: Patch .012 addresses a DUDL issue that can be exposed by removing a working SP.
The best way to avoid this problem is to upgrade to patch .012. If that cannot be accomplished,
disabling write cache is another way to avoid the potential for DUDL. The SP replacement
procedure provides instruction for disabling write cache before the replacement. Be sure to re-
enable cache after the SPs are replaced.

Question: After replacing the first SP, why do I have to wait ~10 minutes before replacing
the second SP?
Answer: The fault tolerant design of the storage array ensures I/O stored in cache is mirrored
on both SPs. When a new SP is inserted the I/O on the peer SP will be mirrored to the first. It
will take ~10 minutes for the SP to come up, configure itself, and receive the cached data from
the peer SP. If the peer SP is removed before its cached data is mirrored to the new SP,
problems will likely result. Performing the steps in the procedure to ensure the new SP is fully
operational before the second SP is removed will avoid this problem

Question: I replaced the first SP and now between the two power supplies of the peer SP
I see an icon of a white hand with a yellow line through it. What does this icon mean?
Answer: It means, “Do not remove this storage processor.” Go back to Unisphere and verify the
new SP is fully operational before replacing the second SP. It will take approximately 10
minutes to reach this state.

Question: How long can a customer use the original SP before noticing a problem?
Answer: For as long as the customer continues to run the current code revision and related
patches, they can run their VNX5700 or VNX7500 indefinitely without issue. They will
experience a problem when they try to upgrade to a 05.31.000.5.50x, which is available as of
September 12, 2011. Until they reach that decision point, there are no downside effects to using
the current hardware.

Question: What will happen if the customer tries to upgrade to a Maintenance release
prior to replacing the SP?
Answer: The problem they will encounter is they will not be able to perform the upgrade. The
system information will report that the hardware is not ready. After the upgrade fails, they will be
able to continue running their current version. The software is customer upgradable.

Question: What are the Part and Rev numbers of the “good” parts?
Answer: For the purpose of this question a “good” part is one that allows an upgrade to
Maintenance Release 05.31.000.5.50x and greater. These revisions will contain the “good”
firmware.

VNX5700 Part Number Good Rev # Good Firmware

Storage 110-113-418B B26 or BIOS 7.20

EMC CONFIDENTIAL version: 3.1

6 of 9
 EMC ® VNX ™ Procedure Generator

Processor greater POST 36.81

Motherboard FRU 303-113-400B B25 or (and greater)
greater

VNX7500 Part Number Good Rev # Good Firmware

Storage 110-113-724B B24 or BIOS 7.20
Processor greater POST 36.81
Motherboard FRU 303-113-700B B24 or (and greater)
greater

EMC CONFIDENTIAL version: 3.1

7 of 9
 EMC ® VNX ™ Procedure Generator

Question: Where are the Part and Rev number labels located?
Answer: See the diagram below. The Storage Processor label is on the front of the SP.
Remove the bezel and look between the Power Supplies. The Motherboard and CPU label is
only seen after the SP is removed. Look on the right rear of the sheet metal. The same
information is on the motherboard. Look through the vent holes on the blue DIMM cover and
you can see the label.

Question: Can the Part and Rev Number information be seen in Unisphere and USM?
Answer: Yes.
Unisphere: System, Hardware, Storage Hardware, expand the SPs, right-click, view resume
Unisphere Service Manager: Select View System Configuration, Generate a new configuration
file and look under Hardware, Replaceable Units, SP A and SP B lines.

Question: Where can I find the BIOS and POST version information?
Answer: BIOS and POST information is available from an SPcollect.

Question: What is the difference between the “Storage Processor” and the
“Motherboard & CPU” FRU?
Answer: The Storage Processor is comprised of the motherboard & CPU assembly and the
DIMM memory installed on the motherboard. In order manage the inventory of parts for this
FCO and facilitate locating the material closer to where it is needed for the FCO, the higher level
Storage Processor assembly was temporarily made a FRU. For normal break/fix part
replacement the motherboard & CPU assembly is a FRU and the individual DIMMs are FRUs.

Question: Why replace the SP? Can I just replace the Motherboard & CPU and reuse the
DIMMs?
Answer: It is unlikely that the available spares will be at the right revision to allow this to be a
viable alternative. We have temporarily made the SP a FRU to allow us to position the
replacement parts closer to the system affected by the FCO. In parallel with the Field FCO we
are purging the spare parts from Logistics inventory, reworking, and returning to Logistics. The
Spares purge affects more parts, so it will take longer to complete. 

Question What do I do if I have a dead SP and the only available part in Logistics is a
down rev part?
Answer: Replace the SP using the available FRU from Logistics. If it is below the minimum
acceptable revision, notify the FCO administrator to open an FCO SR so the SP can be
replaced a second time as soon as the FCO part is available.

EMC CONFIDENTIAL version: 3.1

8 of 9
 EMC ® VNX ™ Procedure Generator

Question Customer has several VNX systems, but not all are on the CAL. Should they all
be on the CAL?
Answer: No. This problem only affects the Storage Processors on a small number of VNX5700
and VNX7500 systems. The affected systems shipped before April 18, 2011. The Revision
Number of the Storage Processor will be less than the acceptable rev shown earlier in the table.

Question: How quickly do parts have to be returned?

Answer: As fast as possible. The Storage Processors are in short supply. The parts returned
from the field are returned to Manufacturing, reworked, and used as spares for other customers.
The faster you return the old part, the faster another customer will receive remediation from the
same problems.

EMC CONFIDENTIAL version: 3.1

9 of 9