VVR Implementation

UNIX Essentials: VVR Implementation
http://unix-essentials.blogspot.com/2007/11/vvr-implementation_3082.html
Higit Pa
Susunod na Blog
Bumuo ng Blog
Mag-sign in
UNIX Essentials
Jaughn's official UNIX library on the web.
VVR Implementation
Friday, November 2, 2007 at 4:36 PM |
Heres one of the VVR projects I designed and implemented. I documented the whole process so I could use it as a reference for future projects.
The document has a detailed implementation (with screenshots). The changes were done on both primary and secondary sites. Downtime was not necess but the customer decided to shut down the apps and database. This implementation involves VCS,CCA and SNAPshots. I will not discuss CCA in this document.
Contents
Project Description Server Configurations Network Configurations Essential Terminology Volumes to be Replicated Storage Replication Log (SRL) Data Change Map (DCM) Disk Change Object (DCO) Technical Implementation Steps/Procedures Detailed Implementation 1. Install VVR License on all nodes for both sites 2. Install VVR Packages on all nodes for both sites 3. Create a Diskgroup and volumes in the secondary site 4. Update the .rdg file on both nodes in the secondary site 5. Stop all applications both clustered and non-clustered apps 6. Bring up the virtual IPs on all nodes on both sites 7. Create the SRL Volume in the primary site 8. Add DCM logs to the all the primary site volumes that will be replicated 9. Update the .rdg file on both nodes in the primary site 10. Ensure that VVRTypes.cf exists in /etc/VRTSvcs/conf 11. Ensure that other application types i.e. Db2udbTypes.cf exists in /etc/VRTSvcs/conf 12. Create the Primary RVG 13. Create the Secondary RVG 14. Configure VCS to integrate VVR objects Primary Site 15. Configure VCS to integrate VVR objects Secondary Site 16. Bring up VCS engine and bring up the vvr service group in the Secondary Site 17. Bring up VCS engine and bring up the vvr service group in the Primary Site 18. Check the Rlink and RVG status 20. Start the replication 21. Check the Replication status 22. Mount the Replicated Volumes 23. Prepare the Replicated Volumes for Snapshot in the secondary site 24. Create the SNAP Volumes in the secondary site 25. Prepare the SNAP Volumes 26. Run a point-in-time snapshot 27. Mount the SNAP Volumes
1 of 27
6/30/2013 1:30 PM
Project Description
As part of the Disaster recovery project, Veritas Volume Replicator (VVR) will be implemented on EMM servers to handle the replication between the Phoen servers and the DR servers in Minneapolis. VVR will run on top of Veritas Cluster System (VCS). VCS is already installed and running on the servers. Replicated volumes in the secondary site will be configured to have snapshot volumes.
Server Configurations
E3 (Clustered) SUNSRV01 (Solaris 8, SunFire V440) SUNSRV02 (Solaris 8, SunFire V440)
DR (Clustered) SUNSRV01-DR (Solaris 8, SunFire V440) SUNSRV02-DR (Solaris 8, SunFire V440)
Network Configurations
IP Address SUNSRV01 10.10.231.130 sunsrv01 Host IP 10.11.196.192 sunsrv01-DRN E3/DR Link IP SUNSRV02 10.10.231.131 sunsrv02 Host IP 10.11.196.193 sunsrv02-DRN E3/DR Link IP SUNSRV01-DR 10.10.231.130 sunsrv01-dr Host IP 10.12.94.192 sunsrv01dr-DRN E3/DR Link IP SUNSRV02-DR 10.10.231.131 sunsrv02-dr Host IP 10.12.94.193 sunsrv02dr-DRN E3/DR Link IP
VVR Config RVG db2dg_rvg SRL db2dg_srl RLINKs rlk_db2instdr-vipvr_db2dg_rvg (E3) rlk_db2inste3-vipvr_db2dg_rvg (DR)
Virtual IPs SUNSRV01/SUNSRV02 Application 10.10.231.191 E3/DR Link 10.11.196.191 SUNSRV01-DR/SUNSRV02-DR Application 10.10.231.191 E3/DR Link 10.12.94.191
2 of 27
6/30/2013 1:30 PM
VCS Heartbeats SUNSRV01 ce0/ce4 SUNSRV02 ce0/ce4 SUNSRV01-DR ce4 SUNSRV02-DR ce4
Essential Terminology
Data Change Map (DCM) - An object containing a bitmap that can be optionally associated with a data volume on the Primary RVG. The bits represent reg of data that are different between the Primary and the Secondary. DCMs are used to mark sections of volumes on the primary that have changed during extended network outages in order to minimize the amount of data that must be synchronized to the secondary site during the outage.
Disk Change Object (DCO) - DCO volumes are used by Volume Manager FastResync (FR). FR is used to quickly resynchronize mirrored volumes that ha been temporarily split and rejoined. FR works by copying only changes to the newly reattached volume using FR logging. This process reduces the time required to rejoin a split mirror, and requires less processing power than full mirror resynchronization without logging. Data Volume - Volumes that are associated with an RVG and contain application data. Primary Node - The node on which the primary RVG resides.
Replication Link (RLINK) - RLINKs represent the communication link to the couterpart of the RVG on another node. At the Primary node a replicated volu RLINK on a Primary RVG represents the communication link from the Primary RVG to a corresponding Secondary RVG, via an IP connection. Replicated Data Set (RDS) - The group of the RVG on a Primary and its corresponding Secondary hosts. Replicated Volume Group (RVG) - A component of VVR that is made up of a set of data volumes, one or more RLINKs and an SRL. An RVG is a subset RVG. The primary RVG is in use by an application, while the secondary RVG receives replication and writes to local disk. The concept of primary and the storage replicator log (SRL) and replication link (RLINK).
object has one RLINK for each of its network mirrors. On the Secondary node a replicated volume has a single RLINK object that links it to its Primary. Eac
volumes within a given VxVM disk group configured for replication to one or more secondary systems. Data is replicated from a primary RVG to a secondar
secondary is per RVG, not per system. A system can simultaneously be a primary RVG for some RVGs and secondary RVG for others. The RVG also conta
Storage Replication Log (SRL) - Writes to the Primary RVG are saved in the SRL on the Primary side. The SRL is used to aid in recovery, as well as to b another to the Primary SRL. Secondary Node - The node too which the primary RVG replicates.
writes when the system operates in asynchronous mode. Each write to a data volume in the RVG generates two write requests: one to the Secondary SRL
Volumes to be Replicated
These are the volumes that will be configured for replication. These volumes belong to the same diskgroup db2dg.
v v v v v v
bcp db dba db2 lg1 tp01
ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED
ACTIVE ACTIVE ACTIVE ACTIVE ACTIVE ACTIVE
585105408 1172343808 98566144 8388608 396361728 192937984
SELECT SELECT SELECT SELECT SELECT SELECT
fsgen fsgen fsgen fsgen fsgen fsgen
3 of 27
6/30/2013 1:30 PM
Storage Replication Log (SRL)
Storage Replication Log. All data writes destined for volumes configured for replication are first persistently queued in a log called the Storage Replicator L
VVR implements the SRL at the primary side to store all changes for transmission to the secondary(s). The SRL is a VxVM volume configured as part of an at the secondary. All writes sent to the VxVM volume layer, whether from an application such as a database writing directly to storage, or an application accessing storage via a file system are faithfully replicated in application write order to the secondary.
RVG. The SRL gives VVR the ability to associate writes to specific volumes within the replicated configuration in a specific order, maintaining write order fid
db2dg_srl
ENABLED
ACTIVE
856350720 SELECT
fsgen
Data Change Map (DCM)
Data Change Maps (DCM) are used to mark sections of volumes on the primary that have changed during extended network outages in order to minimize t amount of data that must be synchronized to the secondary site during the outage. Only one disk will be used for DCM logs at this time. In both primary and secondary sites, the same disk device name is used.
GROUP db2dg
DISK db2dg95
DEVICE EMC0_94
TAG EMC0_94
OFFSET 0
LENGTH 35681280
FLAGS -
Disk Change Object (DCO)
Disk Change Object (DCO) volumes are used by Volume Manager FastResync (FR). FR is used to quickly resynchronize mirrored volumes that have been rejoin a split mirror, and requires less processing power than full mirror resynchronization without logging.
temporarily split and rejoined. FR works by copying only changes to the newly reattached volume using FR logging. This process reduces the time required
GROUP db2dg
DISK db2dg95
DEVICE EMC0_94
TAG EMC0_94
OFFSET 0
LENGTH 35681280
FLAGS -
Technical Implementation Steps/Procedures

1. Install VVR License on all nodes for both sites 2. Install VVR Packages on all nodes for both sites 3. Create a Diskgroup and volumes in the secondary site 4. Update the .rdg file on both nodes in the secondary site 5. Stop all applications both clustered and non-clustered apps 6. Bring up the virtual IPs on all nodes on both sites 7. Create the SRL Volume in the primary site 8. Add DCM logs to the all the primary site volumes that will be replicated 9. Update the .rdg file on both nodes in the primary site 10. Ensure that VVRTypes.cf exists in /etc/VRTSvcs/conf 11. Ensure that other application types i.e. Db2udbTypes.cf exists in /etc/VRTSvcs/conf 12. Create the Primary RVG 13. Create the Secondary RVG 14. Configure VCS to integrate VVR objects Primary Site 15. Configure VCS to integrate VVR objects Secondary Site
4 of 27
6/30/2013 1:30 PM
16. Bring up VCS engine and bring up the vvr service group in the Secondary Site 17. Bring up VCS engine and bring up the vvr service group in the Primary Site 18. Check the Rlink and RVG status 20. Start the replication 21. Check the Replication status 22. Mount the Replicated Volumes 23. Prepare the Replicated Volumes for Snapshot in the secondary site 24. Create the SNAP Volumes in the secondary site 25. Prepare the SNAP Volumes 26. Run a point-in-time snapshot 27. Mount the SNAP Volumes
Detailed Implementation
Install VVR License on all nodes for both sites
# /opt/VRTSvlic/sbin/vxlicinst
Install VVR packages on all nodes for both sites

List of Required and Optional Packages for VVR
Required software packages for VVR: VRTSvlic VERITAS Licensing Utilities. VRTSvxvm VERITAS Volume Manager and Volume Replicator. VRTSob VERITAS Enterprise Administrator Service. VRTSvmpro VERITAS Volume Manager Management Services Provider. VRTSvrpro VERITAS Volume Replicator Management Services Provider. VRTSvcsvr VERITAS Cluster Server Agents for VERITAS Volume Replicator. Optional software packages for VVR: VRTSjre VERITAS JRE Redistribution. VRTSweb VERITAS Java Web Server. VRTSvrw VERITAS Volume Replicator Web Console. VRTSobgui VERITAS Enterprise Administrator. VRTSvmdoc VERITAS Volume Manager documentation. VRTSvrdoc VERITAS Volume Replicator documentation. VRTSvmman VERITAS Volume Manager manual pages. VRTSap VERITAS Action Provider. VRTStep VERITAS Task Execution Provider.
Installing the VVR Packages Using the pkgadd Command 1. Log in as root. 2. Mount from the software repository:
# mount software:/repos /mnt # cd /mnt/software/os/SunOS/middleware/Veritas4.1/volume_replicator
3. Alternatively, you may choose to run the install script from the same directory. Refer to VVR Installation Guide for more information.
5 of 27
6/30/2013 1:30 PM
# ./installvvr
4. But this document follows the manual process.
# cd /mnt/software/os/SunOS/middleware/Veritas4.1/volume_replicator
Note: Install the packages in the order specified below to ensure proper installation. 5. Use the following command to install the required software packages in the specified order. Some of these may have already been installed.
# pkgadd -d . VRTSvlic VRTSvxvm VRTSob VRTSvcsvr
6. Install the following patch:
# patchadd 115209-<latest>
7. Install the following required packages in the specified order:
# pkgadd -d . VRTSvmpro VRTSvrpro
8. Use the following command to install the optional software packages:
# pkgadd -d . VRTSobgui VRTSjre VRTSweb VRTSvrw VRTSvmdoc \ VRTSvrdoc VRTSvmman VRTSap VRTStep
The system prints out a series of status messages as the installation progresses and prompts you for any required information, such as the license key.
Create a Diskgroup and volumes in the secondary site

initialize all LUNs and assign to Diskgroup db2dg sunsrv01-dr:# vxdiskadm Create volumes with the same sizes as the primary sites volumes. sunsrv01-dr:# sunsrv01-dr:# sunsrv01-dr:# sunsrv01-dr:# sunsrv01-dr:# sunsrv01-dr:# vxassist vxassist vxassist vxassist vxassist vxassist -g -g -g -g -g -g db2dg db2dg db2dg db2dg db2dg db2dg make make make make make make bcp 585105408 db 1172343808 dba 98566144 db2 8388608 lg1 396361728 tp01 192937984 layout=concat layout=concat layout=concat layout=concat layout=concat layout=concat
Create the SRL Volume. Allocate disks that are dedicated just for SRL volume. No other volumes should use those disks.
6 of 27
6/30/2013 1:30 PM
sunsrv01-dr:# vxassist -g > db2dg71 db2dg72 db2dg73 > db2dg79 db2dg80 db2dg81 > db2dg87 db2dg88 db2dg89
db2dg make db2dg_srl 856350720 layout=concat \ db2dg74 db2dg75 db2dg76 db2dg77 db2dg78 \ db2dg82 db2dg83 db2dg84 db2dg85 db2dg86 \ db2dg90 db2dg91 db2dg92 db2dg93 db2dg94
Add DCM logs to the volumes. Use the disk thats been assigned only for logs.
sunsrv01-dr:# sunsrv01-dr:# sunsrv01-dr:# sunsrv01-dr:# sunsrv01-dr:# sunsrv01-dr:#
vxassist vxassist vxassist vxassist vxassist vxassist
-g -g -g -g -g -g
db2dg db2dg db2dg db2dg db2dg db2dg
addlog addlog addlog addlog addlog addlog
logtype=dcm logtype=dcm logtype=dcm logtype=dcm logtype=dcm logtype=dcm
nlog=1 nlog=1 nlog=1 nlog=1 nlog=1 nlog=1
db2dg95 db2dg95 db2dg95 db2dg95 db2dg95 db2dg95
Update the .rdg file on both nodes in the secondary site

The Secondary Node must be given permission to manage the disk group created on the Primary Node. To do this add the diskgroup ID into /etc/vx /vras/.rdg. The diskgroup ID is the value of dgid in the output of vxprint -l db2dg on the Primary Node.
sunsrv01:# vxprint -l db2dg | grep dgid info: dgid=1138140445.1393.sunsrv01 noautoimport sunsrv01-dr:# echo 1138140445.1393.sunsrv01 > /etc/vx/vras/.rdg sunsrv02-dr:# echo 1138140445.1393.sunsrv01 > /etc/vx/vras/.rdg
Stop all applications both clustered and non-clustered apps

Stop all clustered and non-clustered applications that are using the volumes, which will be replicated. Note: You may leave the applications running while replicating.
# su db2inst c db2stop
Unmount all filesystems whose underlying volumes will be replicated.
# umount ` mount | grep '/dev/vx/dsk/db2dg/' | awk '{ print $1 }'`
Forcibly stop VCS so it leaves all running resources online especially Volumes and the diskgroup
# hastop all -force
Bring up the virtual IPs on all nodes on both sites

These are the VVR Replication IPs (R-Link IPs). Must be unique one at the primary site and other at the secondary site.
7 of 27
6/30/2013 1:30 PM
Primary (sunsrv01 or sunsrv02):
ce3:1 - 10.11.196.191 netmask fffffe00 broadcast 10.11.197.255
Secondary (sunsrv01-dr or sunsrv02-dr):
ce5:1 - 10.12.94.191 netmask fffffe00 broadcast 10.12.95.255
Create the SRL Volume in the primary site

Allocate disks that are dedicated just for SRL volume. No other volumes should use those disks.
sunsrv01:# vxassist -g db2dg make > db2dg71 db2dg72 db2dg73 db2dg74 > db2dg79 db2dg80 db2dg81 db2dg82 > db2dg87 db2dg88 db2dg89 db2dg90
db2dg_srl 856350720 layout=concat \ db2dg75 db2dg76 db2dg77 db2dg78 \ db2dg83 db2dg84 db2dg85 db2dg86 \ db2dg91 db2dg92 db2dg93 db2dg94
Add DCM logs to all the primary site volumes that will be replicated
Use the disk that is dedicated for logs only.
sunsrv01:# sunsrv01:# sunsrv01:# sunsrv01:# sunsrv01:# sunsrv01:#
vxassist vxassist vxassist vxassist vxassist vxassist
-g -g -g -g -g -g
addlog addlog addlog addlog addlog addlog
logtype=dcm logtype=dcm logtype=dcm logtype=dcm logtype=dcm logtype=dcm
nlog=1 nlog=1 nlog=1 nlog=1 nlog=1 nlog=1
db2dg95 db2dg95 db2dg95 db2dg95 db2dg95 db2dg95
Update the .rdg file on both nodes in the primary site

The diskgroup ID is the value of dgid in the output of vxprint -l db2dg on the Second ary Node.
sunsrv02-dr:# vxprint -l db2dg | grep dgid info: dgid=1182230506.2373.sunsrv01-dr noautoimport sunsrv01:# echo 1182230506.2373.sunsrv01-dr > /etc/vx/vras/.rdg sunsrv02:# echo 1182230506.2373.sunsrv01-dr > /etc/vx/vras/.rdg
Ensure that VVRTypes.cf exists in /etc/VRTSvcs/conf

This types file must exist on all nodes in the cluster.
sunsrv01:# ls -l /etc/VRTSvcs/conf/VVRTypes.cf
8 of 27
6/30/2013 1:30 PM
-rw-rw-r--
1 root
sys
1811 Jan 20
2005 /etc/VRTSvcs/conf/VVRTypes.cf
Ensure that other application types i.e. Db2udbTypes.cf exists in /etc/VRTSvcs/conf

This types file must exist on all nodes in the cluster.
sunsrv01:# ls -l /etc/VRTSvcs/conf/Db2udbTypes.cf -rw-rw-r-1 root sys 1080 Jun 26 2006 /etc/VRTSvcs/conf/Db2udbTypes.cf
Create the Primary RVG

First, ensure that /usr/sbin/vradmind is running. If not, start it with this command:
sunsrv01:# /etc/init.d/vras-vradmind.sh start
Run this from the primary site.
sunsrv01:# vradmin g db2dg createpri db2dg_rvg bcp,db,dba,db2,lg1,tp01 \ db2dg_srl
Usage:
vradmin -g <diskgroup> createpri <RVGname> <vol1,vol2...> <SRLname>
Where: <diskgroup> <RVGname> <SRLname> is the VxVM diskgroup name is the name for the RVG, usually <diskgroupname>_rvg is the name of the SRL volume
<vol1,vol2...> is a comma separated list of all volumes in the diskgroup to be replicated.
Create the Secondary RVG
After creating the Primary RVG, go on to adding a Secondary. Use the vradmin addsec c ommand to add a Secondary R The VIP hostnames are in the /etc/hosts file.
sunsrv01:# vradmin -g db2dg addsec db2dg_rvg db2inste3-vipvr db2instdr-vipvr
Usage:
9 of 27
6/30/2013 1:30 PM
vradmin -g <diskgroup> addsec <RVGname> <primaryhost> <secondaryhost>
Where: <diskgroup> <RVGname> <primaryhost> is the VxVM diskgroup name is the name of the RVG created on the Primary Node is the hostname of the Primary Node, this could be a VCS ServiceGroup name
<secondaryhost> is the hostname of the Secondary Node, this could be a VCS ServiceGroup name
Note: The vradmin addsec command performs the following operations:
Creates and adds a Secondary RVG of the same name as the Primary RVG to the specified RDS on the Secondary host. By default, the Secondary RVG is added to the disk group with the same name as the Primary disk grou Use the option -sdg with the vradminaddsec command to specify a different disk group on the Secondary. Automatically adds DCMs to the Primary and Secondary data volumes if they do not have DCMs. Associates to the Secondary RVG, existing data volumes of the same names and sizes as the Primary data volumes; it also associates an existing volume with the same name as the Primary SRL, as the Secondary SRL Creates and associates to the Primary and Secondary RVGs respectively, the Primary an d Secondary RLINKs wi default RLINK names rlk_remotehost_rvgname.
Configure VCS to integrate VVR objects Primary Site
There would be at least two VCS service Groups. One group represents replication grou p and the other group is th DiskGroup resource. In this particular environment, VVR Service Group and Application Service Group must be online on the same node because the diskgroup is on the VVR Service Group.
application (DB) group. Application group contains RVGPrimary resource. Replication g roup contains IP, RVG and t
VVR Service group must be started first before the Application Service Group. The dep endencies are configured in the VCS configuration file. VVR Service group maintains the synchronization between the primary and secondary sit es.
VCS Service Groups:

db2inst_vvr VVR Service Group Resources: db2dgAgent (RVG) vvrnic (NIC) vvrip (VIP for VVR communications) db2dg_dg (Diskgroup)
db2inst_grp Application Service Group Resources: db2dg_rvg_primary (RVG Primary) db2inst_db (DB2 Database) db2inst_ce1 (NIC) db2inst_ip (VIP for the application) db2inst__mnt (Filesystems) adsm_db2inst (TSM Backup)
10 of 27
6/30/2013 1:30 PM
Primary Site VCS Configuration File:

include "types.cf" include "Db2udbTypes.cf" include "VVRTypes.cf" cluster e3clus177 ( UserNames = { admin = ajkCjeJgkFkkIskEjh } ClusterAddress = "10.10.231.191" Administrators = { admin } CredRenewFrequency = 0 CounterInterval = 5 ) system sunsrv01 ( ) system sunsrv02 ( ) group db2inst_grp ( SystemList = { sunsrv01 = 0, sunsrv02 = 1 } AutoStart = 0 ) Application adsm_db2inst ( Critical = 0 User = root StartProgram = "/bcp/db2inst/tsm/adsmcad.db start" StopProgram = "/bcp/db2inst/tsm/adsmcad.db stop" MonitorProcesses = { "/usr/bin/dsmcad -optfile=/bcp/db2inst/tsm/dsm.opt.db2inst" } ) Db2udb db2inst_db ( Critical = 0 DB2InstOwner = db2inst DB2InstHome = "/db2/db2inst" MonScript = "/opt/VRTSvcs/bin/Db2udb/SqlTest.pl" ) DiskGroup db2dgB_dg ( DiskGroup = db2dgB ) IP db2inst_ip ( Device = ce1 Address = "10.10.231.191" ) Mount db2dgB_bkup_mnt ( MountPoint = "/backup/db2inst" BlockDevice = "/dev/vx/dsk/db2dgB/bkp" FSType = vxfs FsckOpt = "-y" ) Mount db2inst_bcp_mnt ( MountPoint = "/bcp/db2inst" BlockDevice = "/dev/vx/dsk/db2dg/bcp" FSType = vxfs MountOpt = rw FsckOpt = "-y" ) Mount db2inst_db2_mnt ( MountPoint = "/db2/db2inst" BlockDevice = "/dev/vx/dsk/db2dg/db2" FSType = vxfs MountOpt = rw FsckOpt = "-y" )
11 of 27
6/30/2013 1:30 PM
Mount db2inst_db_mnt ( MountPoint = "/db/db2inst/PEMMP00P/NODE0000" BlockDevice = "/dev/vx/dsk/db2dg/db" FSType = vxfs MountOpt = rw FsckOpt = "-y" ) Mount db2inst_dba_mnt ( MountPoint = "/dba/db2inst" BlockDevice = "/dev/vx/dsk/db2dg/dba" FSType = vxfs MountOpt = rw FsckOpt = "-y" ) Mount db2inst_lg1_mnt ( MountPoint = "/db/db2inst/log1" BlockDevice = "/dev/vx/dsk/db2dg/lg1" FSType = vxfs MountOpt = rw FsckOpt = "-y" ) Mount db2inst_tp01_mnt ( MountPoint = "/db/db2inst/PEMMP00P/tempspace01/NODE0000" BlockDevice = "/dev/vx/dsk/db2dg/tp01" FSType = vxfs MountOpt = rw FsckOpt = "-y" ) NIC db2inst_ce1 ( Device = ce1 NetworkType = ether ) RVGPrimary db2dg_rvg_primary ( Critical = 0 RvgResourceName = db2dgAgent ) requires group db2inst_vvr online local hard adsm_db2inst requires db2inst_bcp_mnt db2dgB_bkup_mnt requires db2dgB_dg db2inst_bcp_mnt requires db2dg_rvg_primary db2inst_db requires db2inst_bcp_mnt db2inst_db requires db2inst_db2_mnt db2inst_db requires db2inst_db_mnt db2inst_db requires db2inst_dba_mnt db2inst_db requires db2inst_ip db2inst_db requires db2inst_lg1_mnt db2inst_db requires db2inst_tp01_mnt db2inst_db2_mnt requires db2dg_rvg_primary db2inst_db_mnt requires db2dg_rvg_primary db2inst_dba_mnt requires db2dg_rvg_primary db2inst_ip requires db2inst_ce1 db2inst_lg1_mnt requires db2dg_rvg_primary db2inst_tp01_mnt requires db2dg_rvg_primary
group db2inst_vvr ( SystemList = { sunsrv01 = 0, sunsrv02 = 1 } ) DiskGroup db2dg_dg ( DiskGroup = db2dg ) IP vvrip ( Device = ce3 Address = "10.11.196.191" NetMask = "255.255.254.0"
12 of 27
6/30/2013 1:30 PM
) NIC vvrnic ( Device = ce3 NetworkType = ether ) RVG db2dgAgent ( Critical = 0 RVG = db2dg_rvg DiskGroup = db2dg ) db2dgAgent requires db2dg_dg vvrip requires vvrnic
Configure VCS to integrate VVR objects Secondary Site
In this particular environment, there will be four VCS service Groups CCA Group, Replication Group, Application (DB) Group, and SNAP Group. CCA gro Group contains the resources for SNAP copy. This document will not discuss Veritas CCA. In this particular environment, VVR Service Group, Application Service Group, and SNAP Service Group must be online on the same node because the diskgroup is on the VVR Service Group. SNAP volumes are on the same diskgroup as the application/DB volumes. SNAP Configuration is discussed later in this document. VVR Service group must be started first before the Application Service Group. The dependencies are configured in the VCS configuration file. VVR Service group maintains the synchronization between the primary and secondary sites.
for remote cluster administration. Application group contains RVGPrimary resource. Replication group contains IP, RVG and the DiskGroup resource. SNAP
VCS Service Groups:

db2inst_vvr VVR Service Group Resources: db2dgAgent (RVG) vvrnic (NIC) vvrip (VIP for VVR communications) db2dg_dg (Diskgroup) db2inst_grp Application Service Group Resources: db2dg_rvg_primary (RVG Primary) db2inst_db (DB2 Database) db2inst_ce1 (NIC) db2inst_ip (VIP for the application) db2inst__mnt (Filesystems) adsm_db2inst (TSM Backup) snap_db2inst_grp Snap Copy Service Group Resources: snap_db2inst_db (DB2 Database) snap_db2inst_ce1 (NIC) snap_db2inst_ip (VIP for the application) snap_db2inst__mnt (Filesystems)
13 of 27
6/30/2013 1:30 PM
Secondary Site VCS Configuration File:

include include include include "types.cf" "ClusterMonitorConfigType.cf" "Db2udbTypes.cf" "VVRTypes.cf"
cluster e3clus177 ( UserNames = { admin = ajkCjeJgkFkkIskEjh } ClusterAddress = "10.10.231.191" Administrators = { admin } CredRenewFrequency = 0 CounterInterval = 5 ) system sunsrv01-dr ( ) system sunsrv02-dr ( ) group CCAvail ( SystemList = { sunsrv01-dr = 0, sunsrv02-dr = 1 } AutoStartList = { sunsrv01-dr, sunsrv02-dr } ) ClusterMonitorConfig CCAvail_ClusterConfig ( MSAddress = "10.11.198.53" ClusterId = 1183482234 VCSLoggingLevel = TAG_A Logging = "/opt/VRTSccacm/conf/k2_logging.properties" ClusterMonitorVersion = "4.1.2272.1" ) Process CCAvail_ClusterMonitor ( PathName = "/opt/VRTSccacm/bin/ClusterMonitor" Arguments = "-config" ) CCAvail_ClusterMonitor requires CCAvail_ClusterConfig
group db2inst_grp ( SystemList = { sunsrv01-dr = 0, sunsrv02-dr = 1 } Enabled @sunsrv01-dr = 0 Enabled @sunsrv02-dr = 0 AutoStart = 0 ) Application adsm_db2inst ( Enabled = 0 Critical = 0 User = root StartProgram = "/bcp/db2inst/tsm/adsmcad.db start" StopProgram = "/bcp/db2inst/tsm/adsmcad.db stop" MonitorProcesses = { "/usr/bin/dsmcad -optfile=/bcp/db2inst/tsm/dsm.opt.db2inst" } ) Db2udb db2inst_db ( Enabled = 0 Critical = 0 DB2InstOwner = db2inst DB2InstHome = "/db2/db2inst" MonScript = "/opt/VRTSvcs/bin/Db2udb/SqlTest.pl" ) DiskGroup db2dgB_dg ( Enabled = 0 DiskGroup = db2dgB ) IP db2inst_ip ( Enabled = 0
14 of 27
6/30/2013 1:30 PM
Device = ce1 Address = "10.10.231.191" ) Mount db2dgB_bkup_mnt ( Enabled = 0 MountPoint = "/backup/db2inst" BlockDevice = "/dev/vx/dsk/db2dgB/bkp" FSType = vxfs FsckOpt = "-y" ) Mount db2inst_bcp_mnt ( Enabled = 0 MountPoint = "/bcp/db2inst" BlockDevice = "/dev/vx/dsk/db2dg/bcp" FSType = vxfs MountOpt = rw FsckOpt = "-y" ) Mount db2inst_db2_mnt ( Enabled = 0 MountPoint = "/db2/db2inst" BlockDevice = "/dev/vx/dsk/db2dg/db2" FSType = vxfs MountOpt = rw FsckOpt = "-y" ) Mount db2inst_db_mnt ( Enabled = 0 MountPoint = "/db/db2inst/PEMMP00P/NODE0000" BlockDevice = "/dev/vx/dsk/db2dg/db" FSType = vxfs MountOpt = rw FsckOpt = "-y" ) Mount db2inst_dba_mnt ( Enabled = 0 MountPoint = "/dba/db2inst" BlockDevice = "/dev/vx/dsk/db2dg/dba" FSType = vxfs MountOpt = rw FsckOpt = "-y" ) Mount db2inst_lg1_mnt ( Enabled = 0 MountPoint = "/db/db2inst/log1" BlockDevice = "/dev/vx/dsk/db2dg/lg1" FSType = vxfs MountOpt = rw FsckOpt = "-y" ) Mount db2inst_tp01_mnt ( Enabled = 0 MountPoint = "/db/db2inst/PEMMP00P/tempspace01/NODE0000" BlockDevice = "/dev/vx/dsk/db2dg/tp01" FSType = vxfs MountOpt = rw FsckOpt = "-y" ) NIC db2inst_ce1 ( Enabled = 0 Device = ce1 NetworkType = ether ) RVGPrimary db2dg_rvg_primary ( Enabled = 0 Critical = 0
15 of 27
6/30/2013 1:30 PM
RvgResourceName = db2dgAgent ) requires group db2inst_vvr online local firm adsm_db2inst requires db2inst_bcp_mnt db2dgB_bkup_mnt requires db2dgB_dg db2inst_bcp_mnt requires db2dg_rvg_primary db2inst_db requires db2inst_bcp_mnt db2inst_db requires db2inst_db2_mnt db2inst_db requires db2inst_db_mnt db2inst_db requires db2inst_dba_mnt db2inst_db requires db2inst_ip db2inst_db requires db2inst_lg1_mnt db2inst_db requires db2inst_tp01_mnt db2inst_db2_mnt requires db2dg_rvg_primary db2inst_db_mnt requires db2dg_rvg_primary db2inst_dba_mnt requires db2dg_rvg_primary db2inst_ip requires db2inst_ce1 db2inst_lg1_mnt requires db2dg_rvg_primary db2inst_tp01_mnt requires db2dg_rvg_primary
group db2inst_vvr ( SystemList = { sunsrv01-dr = 0, sunsrv02-dr = 1 } ) DiskGroup db2dg_dg ( DiskGroup = db2dg ) IP vvrip ( Device = ce5 Address = "10.12.94.191" NetMask = "255.255.254.0" ) NIC vvrnic ( Device = ce5 NetworkType = ether ) RVG db2dgAgent ( Critical = 0 RVG = db2dg_rvg DiskGroup = db2dg ) db2dgAgent requires db2dg_dg vvrip requires vvrnic
group snap_db2inst_grp ( SystemList = { sunsrv01-dr = 0, sunsrv02-dr = 1 } AutoStart = 0 ) Db2udb snap_db2inst_db ( Critical = 0 DB2InstOwner = db2inst DB2InstHome = "/db2/db2inst" MonScript = "/opt/VRTSvcs/bin/Db2udb/SqlTest.pl" ) DiskGroup snap_db2dgB_dg ( DiskGroup = db2dgB ) IP snap_db2inst_ip ( Device = ce1 Address = "10.10.231.191" ) Mount snap_db2dgB_bkup_mnt ( MountPoint = "/backup/db2inst" BlockDevice = "/dev/vx/dsk/db2dgB/bkp"
16 of 27
6/30/2013 1:30 PM
FSType = vxfs FsckOpt = "-y" ) Mount snap_db2inst_bcp_mnt ( MountPoint = "/bcp/db2inst" BlockDevice = "/dev/vx/dsk/db2dg/bcp_snapvol" FSType = vxfs FsckOpt = "-y" ) Mount snap_db2inst_db2_mnt ( MountPoint = "/db2/db2inst" BlockDevice = "/dev/vx/dsk/db2dg/db2_snapvol" FSType = vxfs FsckOpt = "-y" ) Mount snap_db2inst_db_mnt ( MountPoint = "/db/db2inst/PEMMP00P/NODE0000" BlockDevice = "/dev/vx/dsk/db2dg/db_snapvol" FSType = vxfs FsckOpt = "-y" ) Mount snap_db2inst_dba_mnt ( MountPoint = "/dba/db2inst" BlockDevice = "/dev/vx/dsk/db2dg/dba_snapvol" FSType = vxfs FsckOpt = "-y" ) Mount snap_db2inst_lg1_mnt ( MountPoint = "/db/db2inst/log1" BlockDevice = "/dev/vx/dsk/db2dg/lg1_snapvol" FSType = vxfs FsckOpt = "-y" ) Mount snap_db2inst_tp01_mnt ( MountPoint = "/db/db2inst/PEMMP00P/tempspace01/NODE0000" BlockDevice = "/dev/vx/dsk/db2dg/tp01_snapvol" FSType = vxfs FsckOpt = "-y" ) NIC snap_db2inst_ce1 ( Device = ce1 NetworkType = ether ) requires group db2inst_vvr online local firm snap_db2dgB_bkup_mnt requires snap_db2dgB_dg snap_db2inst_db requires snap_db2inst_bcp_mnt snap_db2inst_db requires snap_db2inst_db2_mnt snap_db2inst_db requires snap_db2inst_db_mnt snap_db2inst_db requires snap_db2inst_dba_mnt snap_db2inst_db requires snap_db2inst_lg1_mnt snap_db2inst_db requires snap_db2inst_tp01_mnt snap_db2inst_ip requires snap_db2inst_ce1
Bring up VCS engine and bring up the vvr service group in the Secondary Site
Start VCS on both nodes
sunsrv01-dr:# hastart sunsrv02-dr:# hastart
17 of 27
6/30/2013 1:30 PM
Bring up the VVR Service Group on one node
sunsrv01-dr:# hagrp online db2inst_vvr sys sunsrv01-dr
Bring up VCS engine and bring up the vvr service group in the Primary Site
Start VCS on both nodes
sunsrv01:# hastart sunsrv02:# hastart
Bring up the VVR Service Group on one node
sunsrv01:# hagrp online db2inst_vvr sys sunsrv01
Bring up the Application Service Group
sunsrv01:# hagrp online db2inst_grp sys sunsrv01
Check the Rlink and RVG status
Make sure that the flags are attached and connected. If for some reason links were detached or disconnected, check to make sure that the communicatio good, you may restart VVR (see next step for restarting VVR engine).
between the primary site and the secondary site are working fine. You should be able to ping, ssh, or telnet from each site thru the VIPs. If communications
sunsrv01:# vxprint -Pl Disk group: db2dg Rlink: info: rlk_db2instdr-vipvr_db2dg_rvg timeout=500 packet_size=8400 rid=0.2007 latency_high_mark=10000 latency_low_mark=9950 bandwidth_limit=none state: state=ACTIVE synchronous=off latencyprot=off srlprot=autodcm assoc: rvg=db2dg_rvg remote_host=db2instdr-vipvr IP_addr=10.12.94.191 port=4145 remote_dg=db2dg remote_dg_dgid=1182230506.2373.sunsrv01-dr remote_rvg_version=21 remote_rlink=rlk_db2inste3-vipvr_db2dg_rvg remote_rlink_rid=0.2127 local_host=db2inste3-vipvr IP_addr=10.11.196.191 port=4145 protocol: UDP/IP flags: write enabled attached consistent cant_sync connected asynchronous autosync sunsrv01:# vradmin -l printrvg Replicated Data Set: db2dg_rvg Primary: HostName: db2inste3-vipvr RvgName: db2dg_rvg DgName: db2dg
18 of 27
6/30/2013 1:30 PM
datavol_cnt: 6 srl: db2dg_srl RLinks: name=rlk_db2instdr-vipvr_db2dg_rvg, detached=off, synchronous=off Secondary: HostName: db2instdr-vipvr RvgName: db2dg_rvg DgName: db2dg datavol_cnt: 6 srl: db2dg_srl RLinks: name=rlk_db2inste3-vipvr_db2dg_rvg, detached=off, synchronous=off
Restart VVR engine (if the link is detached or disconnected)
This is only required if you see the links between primary and secondary as detached or disconnected. If you think that the communication between the two working fine, then run the following commands in the primary site:
sunsrv01:# vxstart_vvr stop sunsrv01:# vxstart_vvr start
Then, check the status again.
sunsrv01:# vxprint -Pl
Start the replication

Initiate the command from the primary site.
sunsrv01:# vradmin -g db2dg -a startrep db2dg_rvg db2instdr-vipvr Message from Primary: VxVM VVR vxrlink WARNING V-5-1-3359 Attaching rlink to non-empty rvg. Autosync will be performed. VxVM VVR vxrlink INFO V-5-1-3614 Secondary data volumes detected with rvg db2dg_rvg a s parent: VxVM VxVM VxVM VxVM VxVM VxVM VVR VVR VVR VVR VVR VVR vxrlink vxrlink vxrlink vxrlink vxrlink vxrlink INFO INFO INFO INFO INFO INFO V-5-1-6183 V-5-1-6183 V-5-1-6183 V-5-1-6183 V-5-1-6183 V-5-1-6183 bcp: db: db2: dba: lg1: tp01: len=585105408 len=1172343808 len=8388608 len=98566144 len=396361728 len=192937984 primary_datavol=bcp primary_datavol=db primary_datavol=db2 primary_datavol=dba primary_datavol=lg1 primary_datavol=tp01
VxVM VVR vxrlink INFO V-5-1-3365 Autosync operation has started
Usage:
vradmin -g <diskgroup> -a startrep <RVGname> <secondaryhost>
Where: <diskgroup> is the VxVM diskgroup name
19 of 27
6/30/2013 1:30 PM
<RVGname>
is the name for the RVG, usually <diskgroupname>_rvg
<secondaryhost> is the hostname of the Secondary Site. Check the /etc/hosts file.
Check the Replication status

Initiate the command from the primary site. You can tell if its syncing by looking a t the number of bytes remaining. If it changes, then its working.
sunsrv01:# vxrlink -g db2dg -i 2 Fri Jun 22 00:01:37 MST 2007 VxVM VVR vxrlink INFO V-5-1-4464 VxVM VVR vxrlink INFO V-5-1-4464 VxVM VVR vxrlink INFO V-5-1-4464 VxVM VVR vxrlink INFO V-5-1-4464 VxVM VVR vxrlink INFO V-5-1-4464
status rlk_db2instdr-vipvr_db2dg_rvg Rlink Rlink Rlink Rlink Rlink rlk_db2instdr-vipvr_db2dg_rvg rlk_db2instdr-vipvr_db2dg_rvg rlk_db2instdr-vipvr_db2dg_rvg rlk_db2instdr-vipvr_db2dg_rvg rlk_db2instdr-vipvr_db2dg_rvg is is is is is in in in in in AUTOSYNC. AUTOSYNC. AUTOSYNC. AUTOSYNC. AUTOSYNC. 1226649984 1226644224 1226638464 1226632128 1226626368 Kbytes Kbytes Kbytes Kbytes Kbytes remaini remaini remaini remaini remaini
Another way to check is to use the vradmin repstatus command. The key bits of information here are Data status Replication status.
sunsrv01:# vradmin -g db2dg repstatus db2dg_rvg Replicated Data Set: db2dg_rvg Primary: Host name: db2inste3-vipvr RVG name: db2dg_rvg DG name: db2dg RVG state: enabled for I/O Data volumes: 6 SRL name: db2dg_srl SRL size: 952.84 G Total secondaries: 1 Secondary: Host name: RVG name: DG name: Data status: Replication status: Current mode: Logging to: Timestamp Information:
db2instdr-vipvr db2dg_rvg db2dg consistent, up-to-date replicating (connected) asynchronous SRL N/A
Mount the Replicated Volumes

When replication is 100% complete, you may check the volumes on the secondary site by mounting the filesystems. first, you need to check the status of replication.
sunsrv01:# vxrlink -g db2dg -i 2 status rlk_db2instdr-vipvr_db2dg_rvgThu Jun 28 08:54:42 MST 2007 VxVM VVR vxrlink INFO V-5-1-4467 Rlink rlk_db2instdr-vipvr_db2dg_rvg is up to date
Use VCS to mount the replicated Volumes
20 of 27
6/30/2013 1:30 PM
sunsrv01-dr:# hagrp online db2inst_grp sys sunsrv01-dr
Display the filesystems and compare them with the primary.
sunsrv01-dr:# df -k | grep db2dg /dev/vx/dsk/db2dg/db2 /dev/vx/dsk/db2dg/bcp /dev/vx/dsk/db2dg/dba /dev/vx/dsk/db2dg/db /dev/vx/dsk/db2dg/tp01 /dev/vx/dsk/db2dg/lg1 4194304 78059 3859030 2% /db2/db2inst 292552704 1527508 272836173 1% /bcp/db2inst 49283072 28555 46176114 1% /dba/db2inst 586171904 1733538 547911037 1% /db/db2inst/PEMMP00P/NODE0000 96468992 40176 90402086 1% /db/db2inst/PEMMP00P/tempspace 01/NODE0000 198180864 1779853 184126012 1% /db/db2inst/log1
sunsrv01:# df -k | grep db2dg /dev/vx/dsk/db2dg/db2 /dev/vx/dsk/db2dg/bcp /dev/vx/dsk/db2dg/dba /dev/vx/dsk/db2dg/db /dev/vx/dsk/db2dg/tp01 /dev/vx/dsk/db2dg/lg1 4194304 78059 3859030 2% /db2/db2inst 292552704 1527508 272836173 1% /bcp/db2inst 49283072 28555 46176114 1% /dba/db2inst 586171904 1733538 547911037 1% /db/db2inst/PEMMP00P/NODE0000 96468992 40176 90402086 1% /db/db2inst/PEMMP00P/tempspace 01/NODE0000 198180864 1779853 184126012 1% /db/db2inst/log1
This is the best time to test VCS for switching the service group.
sunsrv01-dr:# hagrp switch db2inst_grp to sunsrv02-dr
Prepare the Replicated Volumes for Snapshot in the secondary site

First, display the volume information
sunsrv02-dr:# vxprint -ht bcp Disk group: db2dg v pl sd sd sd pl sd bcp bcp-01 db2dg214-01 db2dg215-01 db2dg219-01 bcp-02 db2dg95-06 db2dg_rvg bcp bcp-03 bcp-03 bcp-03 bcp bcp-02 ENABLED ENABLED db2dg214 db2dg215 db2dg219 ENABLED db2dg95 ACTIVE ACTIVE 5376 5376 5376 ACTIVE 0 585105408 585108480 285473280 285473280 14161920 LOGONLY 512 SELECT CONCAT 0 285473280 570946560 CONCAT LOG EMC0_219 EMC0_220 EMC0_224 EMC0_154 fsgen RW ENA ENA ENA RW ENA
Prepare the volume for snapshot by adding a DCO log. The same disks is used for DCO a nd DCM logs.
sunsrv02-dr:# vxsnap -g db2dg prepare bcp ndcomirs=1 alloc=db2dg95 VxVM vxsnap INFO V-5-1-9270 Volume is under RVG, setting drl=no.
Display the volume information to see the changes. Note that the DCO volume has been added with two logs.
sunsrv02-dr:# vxprint -ht bcp Disk group: db2dg
21 of 27
6/30/2013 1:30 PM
v pl sd sd sd pl sd dc v pl sd
bcp bcp-01 db2dg214-01 db2dg215-01 db2dg219-01 bcp-02 db2dg95-06 bcp_dco bcp_dcl bcp_dcl-01 db2dg95-07
db2dg_rvg bcp bcp-03 bcp-03 bcp-03 bcp bcp-02 bcp bcp_dcl bcp_dcl-01
ENABLED ENABLED db2dg214 db2dg215 db2dg219 ENABLED db2dg95 bcp_dcl ENABLED ENABLED db2dg95
ACTIVE ACTIVE 5376 5376 5376 ACTIVE 0
585105408 585108480 285473280 285473280 14161920 LOGONLY 512
SELECT CONCAT 0 285473280 570946560 CONCAT LOG SELECT CONCAT 0
EMC0_219 EMC0_220 EMC0_224 EMC0_154
fsgen RW ENA ENA ENA RW ENA
ACTIVE 40368 ACTIVE 40368 960 40368
gen RW EMC0_154 ENA
Run the same steps on the rest of the volumes
sunsrv02-dr:# vxsnap -g db2dg prepare db ndcomirs=1 alloc=db2dg95 VxVM vxsnap INFO V-5-1-9270 Volume is under RVG, setting drl=no. sunsrv02-dr:# vxsnap -g db2dg prepare dba ndcomirs=1 alloc=db2dg95 VxVM vxsnap INFO V-5-1-9270 Volume is under RVG, setting drl=no. sunsrv02-dr:# vxsnap -g db2dg prepare db2 ndcomirs=1 alloc=db2dg95 VxVM vxsnap INFO V-5-1-9270 Volume is under RVG, setting drl=no. sunsrv02-dr:# vxsnap -g db2dg prepare lg1 ndcomirs=1 alloc=db2dg95 VxVM vxsnap INFO V-5-1-9270 Volume is under RVG, setting drl=no. sunsrv02-dr:# vxsnap -g db2dg prepare tp01 ndcomirs=1 alloc=db2dg95 VxVM vxsnap INFO V-5-1-9270 Volume is under RVG, setting drl=no.
Create the SNAP Volumes in the secondary site
sunsrv02-dr:# sunsrv02-dr:# sunsrv02-dr:# sunsrv02-dr:# sunsrv02-dr:# sunsrv02-dr:# sunsrv02-dr:#
vxassist vxassist vxassist vxassist vxassist vxassist vxassist
-g -g -g -g -g -g -g
db2dg db2dg db2dg db2dg db2dg db2dg db2dg
make make make make make make make
bcp_snapvol 585105408 db_snapvol 1172343808 dba_snapvol 98566144 db2_snapvol 8388608 db2_snapvol 8388608 lg1_snapvol 396361728 tp01_snapvol 192937984
layout=concat layout=concat layout=concat layout=concat layout=concat layout=concat layout=concat
Prepare the SNAP Volumes

Identify the region size of the main volumes. The region size must be the same for bo th the main volume and the snap volume.
sunsrv02-dr:# for DCONAME in `vxprint -tg db2dg | grep "^dc" | awk '{ print $2 }'` > do > echo "$DCONAME\t\c" > vxprint -g db2dg -F%regionsz $DCONAME > done bcp_dco 128 db_dco 128 dba_dco 128 db2_dco 128 lg1_dco 128 tp01_dco 128
22 of 27
6/30/2013 1:30 PM
Display the volume information
sunsrv02-dr:# vxprint -ht bcp_snapvol Disk group: db2dg v pl sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd bcp_snapvol fsgen ENABLED bcp_snapvol-01 bcp_snapvol ENABLED db2dg114-01 bcp_snapvol-01 ENABLED db2dg106-01 bcp_snapvol-01 ENABLED db2dg101-01 bcp_snapvol-01 ENABLED db2dg102-01 bcp_snapvol-01 ENABLED db2dg103-01 bcp_snapvol-01 ENABLED db2dg104-01 bcp_snapvol-01 ENABLED db2dg105-01 bcp_snapvol-01 ENABLED db2dg107-01 bcp_snapvol-01 ENABLED db2dg108-01 bcp_snapvol-01 ENABLED db2dg109-01 bcp_snapvol-01 ENABLED db2dg110-01 bcp_snapvol-01 ENABLED db2dg111-01 bcp_snapvol-01 ENABLED db2dg112-01 bcp_snapvol-01 ENABLED db2dg113-01 bcp_snapvol-01 ENABLED db2dg115-01 bcp_snapvol-01 ENABLED db2dg116-01 bcp_snapvol-01 ENABLED db2dg117-01 bcp_snapvol-01 ENABLED 585105408 ACTIVE 585105600 ACTIVE 35726400 0 35681280 35726400 35681280 71407680 35681280 107088960 35681280 142770240 35681280 178451520 35681280 214132800 35681280 249814080 35681280 285495360 13844160 321176640 35726400 335020800 35726400 370747200 35726400 406473600 35726400 442200000 35726400 477926400 35726400 513652800 35726400 549379200 -
Prepare the snapshot volume by adding a DCO log and the same region size as the main volume. The same disks is u for DCO and DCM logs.
sunsrv02-dr:# vxsnap -g db2dg prepare bcp_snapvol ndcomirs=1 regionsize=128 \ alloc=db2dg95
Display the volume information to see the changes. Note that the DCO volume has been added.
sunsrv02-dr:# vxprint -ht bcp_snapvol Disk group: db2dg v pl sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd sd dc v pl sd bcp_snapvol fsgen ENABLED 585105408 ACTIVE bcp_snapvol-01 bcp_snapvol ENABLED 585105600 ACTIVE db2dg114-01 bcp_snapvol-01 ENABLED 35726400 0 db2dg106-01 bcp_snapvol-01 ENABLED 35681280 35726400 db2dg101-01 bcp_snapvol-01 ENABLED 35681280 71407680 db2dg102-01 bcp_snapvol-01 ENABLED 35681280 107088960 db2dg103-01 bcp_snapvol-01 ENABLED 35681280 142770240 db2dg104-01 bcp_snapvol-01 ENABLED 35681280 178451520 db2dg105-01 bcp_snapvol-01 ENABLED 35681280 214132800 db2dg107-01 bcp_snapvol-01 ENABLED 35681280 249814080 db2dg108-01 bcp_snapvol-01 ENABLED 35681280 285495360 db2dg109-01 bcp_snapvol-01 ENABLED 13844160 321176640 db2dg110-01 bcp_snapvol-01 ENABLED 35726400 335020800 db2dg111-01 bcp_snapvol-01 ENABLED 35726400 370747200 db2dg112-01 bcp_snapvol-01 ENABLED 35726400 406473600 db2dg113-01 bcp_snapvol-01 ENABLED 35726400 442200000 db2dg115-01 bcp_snapvol-01 ENABLED 35726400 477926400 db2dg116-01 bcp_snapvol-01 ENABLED 35726400 513652800 db2dg117-01 bcp_snapvol-01 ENABLED 35726400 549379200 bcp_snapvol_dco bcp_snapvol bcp_snapvol_dcl gen ENABLED 40368 ACTIVE bcp_snapvol_dcl-01 bcp_snapvol_dcl ENABLED 40368 ACTIVE db2dg95-01 bcp_snapvol_dcl-01 ENABLED 40368 0 -
23 of 27
6/30/2013 1:30 PM
Run the same steps on the rest of the volumes
sunsrv02-dr:# vxsnap -g db2dg alloc=db2dg95 sunsrv02-dr:# vxsnap -g db2dg alloc=db2dg95 sunsrv02-dr:# vxsnap -g db2dg alloc=db2dg95 sunsrv02-dr:# vxsnap -g db2dg alloc=db2dg95 sunsrv02-dr:# vxsnap -g db2dg alloc=db2dg95
prepare db_snapvol ndcomirs=1 regionsize=128 \ prepare dba_snapvol ndcomirs=1 regionsize=128 \ prepare db2_snapvol ndcomirs=1 regionsize=128 \ prepare lg1_snapvol ndcomirs=1 regionsize=128 \ prepare tp01_snapvol ndcomirs=1 regionsize=128 \
Verify the region sizes are the same
sunsrv02-dr:# for DCONAME in `vxprint -tg db2dg | grep "^dc" | awk '{ print $2 }'` > do > echo "$DCONAME\t\c" > vxprint -g db2dg -F%regionsz $DCONAME > done bcp_dco 128 bcp_snapvol_dco 128 db_dco 128 db_snapvol_dco 128 dba_dco 128 dba_snapvol_dco 128 db2_dco 128 db2_snapvol_dco 128 lg1_dco 128 lg1_snapvol_dco 128 tp01_dco 128 tp01_snapvol_dco 128
Run a point-in-time snapshot

Verify the RLINK and RVG are active and up to date. Run this command from the primary site.
sunsrv01:# vxrlink -g db2dg -i 2 status rlk_db2instdr-vipvr_db2dg_rvg Thu Jun 28 08:54:42 MST 2007 VxVM VVR vxrlink INFO V-5-1-4467 Rlink rlk_db2instdr-vipvr_db2dg_rvg is up to date sunsrv02-dr:# vxprint -Pl Disk group: db2dg Rlink: info: rlk_db2inste3-vipvr_db2dg_rvg timeout=500 packet_size=8400 rid=0.2127 latency_high_mark=10000 latency_low_mark=9950 bandwidth_limit=none state: state=ACTIVE synchronous=off latencyprot=off srlprot=autodcm assoc: rvg=db2dg_rvg remote_host=db2inste3-vipvr IP_addr=10.11.196.191 port=4145 remote_dg=db2dg remote_dg_dgid=1138140445.1393.sunsrv01 remote_rvg_version=21 remote_rlink=rlk_db2instdr-vipvr_db2dg_rvg remote_rlink_rid=0.2007 local_host=db2instdr-vipvr IP_addr=10.12.94.191 port=4145 protocol: UDP/IP flags: write enabled attached consistent connected
24 of 27
6/30/2013 1:30 PM
Start the Snap Copy
sunsrv02-dr:# vxsnap -g db2dg make source=bcp/snapvol=bcp_snapvol \ source=db/snapvol=db_snapvol source=dba/snapvol=dba_snapvol \ source=db2/snapvol=db2_snapvol source=lg1/snapvol=lg1_snapvol \ source=tp01/snapvol=tp01_snapvol
Display the sync status
sunsrv02-dr:# vxtask list TASKID PTID TYPE/STATE 168 SNAPSYNC/R 170 SNAPSYNC/R 172 SNAPSYNC/R 174 SNAPSYNC/R 176 SNAPSYNC/R 178 SNAPSYNC/R
PCT 00.07% 00.05% 00.46% 04.30% 00.16% 00.18%
PROGRESS 0/585105408/430080 0/1172343808/567296 0/98566144/452608 0/8388608/360448 0/396361728/641024 0/192937984/346112
SNAPSYNC SNAPSYNC SNAPSYNC SNAPSYNC SNAPSYNC SNAPSYNC
bcp_snapvol db_snapvol dba_snapvol db2_snapvol lg1_snapvol tp01_snapvol
sunsrv02-dr:# vxsnap -g db2dg print NAME bcp SNAPOBJECT -bcp_snapvol_snp -db_snapvol_snp -dba_snapvol_snp -db2_snapvol_snp -lg1_snapvol_snp -tp01_snapvol_snp TYPE volume volume volume volume volume volume volume volume volume volume volume volume PARENT ------------SNAPSHOT -bcp_snapvol -db_snapvol -dba_snapvol -db2_snapvol -lg1_snapvol %DIRTY -0.00 -0.00 -0.00 -0.00 -0.00 %VALID 100.00 -100.00 -100.00 -100.00 -100.00 -100.00 --
db
dba
db2
lg1
tp01
--tp01_snapvol 0.00
bcp_snapvol db_snapvol dba_snapvol db2_snapvol lg1_snapvol tp01_snapvol
bcp_snp db_snp dba_snp db2_snp lg1_snp tp01_snp
volume volume volume volume volume volume
-------
0.00 0.00 0.00 0.00 0.00 0.00
0.11 0.02 0.77 9.13 0.09 0.10
Mount the SNAP Volumes

When the Sync is complete, bring up the snap service group in VCS to verify all chang es. But first, unmount the same mountpoints.
filesystems on replicated volumes. In this particular server, the replicated volumes and the snap volumes use th
sunsrv02-dr:# umount ` mount | grep '/dev/vx/dsk/db2dg/' | awk '{ print $1 }'`
25 of 27
6/30/2013 1:30 PM
sunsrv02-dr:# vxtask list TASKID PTID TYPE/STATE
PCT
PROGRESS
sunsrv02-dr:# hastatus -sum -- SYSTEM STATE -- System A A sunsrv01-dr sunsrv02-dr
State RUNNING RUNNING
Frozen 0 0
-- GROUP STATE -- Group B B B B B B B B CCAvail CCAvail db2inst_grp db2inst_grp db2inst_vvr db2inst_vvr snap_db2inst_grp snap_db2inst_grp
System sunsrv01-dr sunsrv02-dr sunsrv01-dr sunsrv02-dr sunsrv01-dr sunsrv02-dr sunsrv01-dr sunsrv02-dr
Probed Y Y Y Y Y Y Y Y
AutoDisabled N N N N N N N N
State ONLINE OFFLINE OFFLINE OFFLINE OFFLINE ONLINE OFFLINE OFFLINE
sunsrv02-dr:# hagrp -online snap_db2inst_grp -sys sunsrv02-dr
sunsrv02-dr:# hastatus -sum -- SYSTEM STATE -- System A A sunsrv01-dr sunsrv02-dr
State RUNNING RUNNING
Frozen 0 0
-- GROUP STATE -- Group B B B B B B B B CCAvail CCAvail db2inst_grp db2inst_grp db2inst_vvr db2inst_vvr snap_db2inst_grp snap_db2inst_grp
System sunsrv01-dr sunsrv02-dr sunsrv01-dr sunsrv02-dr sunsrv01-dr sunsrv02-dr sunsrv01-dr sunsrv02-dr
Probed Y Y Y Y Y Y Y Y
AutoDisabled N N N N N N N N
State ONLINE OFFLINE OFFLINE OFFLINE OFFLINE ONLINE OFFLINE ONLINE
Verify the filesystems and compare the sizes with the primary site.
sunsrv02-dr:# df -k | grep snapvol /dev/vx/dsk/db2dg/db2_snapvol 4194304 78059 3859030 /dev/vx/dsk/db2dg/bcp_snapvol 292552704 1527508 272836173 /dev/vx/dsk/db2dg/dba_snapvol 49283072 28555 46176114 /dev/vx/dsk/db2dg/db_snapvol 586171904 1733538 547911037 /dev/vx/dsk/db2dg/tp01_snapvol 96468992 40176 90402086 /dev/vx/dsk/db2dg/lg1_snapvol 198180864 1779853 184126012 2% 1% 1% 1% 1% 1% /db2/db2inst /bcp/db2inst /dba/db2inst /db/db2inst/PEMMP00P/NO DE0000 /db/db2inst/PEMMP00P/te mpspace01/NODE0000 /db/db2inst/log1
sunsrv01:# df -k | grep db2dg /dev/vx/dsk/db2dg/db2 /dev/vx/dsk/db2dg/bcp /dev/vx/dsk/db2dg/dba /dev/vx/dsk/db2dg/db /dev/vx/dsk/db2dg/tp01 /dev/vx/dsk/db2dg/lg1 4194304 78059 3859030 292552704 1527508 272836173 49283072 28555 46176114 586171904 1733538 547911037 96468992 40176 90402086 198180864 1779853 184126012 2% 1% 1% 1% 1% 1% /db2/db2inst /bcp/db2inst /dba/db2inst /db/db2inst/PEMMP00P/NO DE0000 /db/db2inst/PEMMP00P/te mpspace01/NODE0000 /db/db2inst/log1
26 of 27
6/30/2013 1:30 PM
Posted by JAUGHN Labels: Veritas Volume Replicator
0 comments: Post a Comment Newer Post Home
27 of 27
6/30/2013 1:30 PM

VVR Implementation

Uploaded by

Copyright:

Available Formats

VVR Implementation

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

VVR Implementation

Uploaded by

Copyright:

Available Formats

UNIX Essentials: VVR Implementation

UNIX Essentials: VVR Implementation

DR (Clustered) SUNSRV01-DR (Solaris 8, SunFire V440) SUNSRV02-DR (Solaris 8, SunFire V440)

UNIX Essentials: VVR Implementation

bcp db dba db2 lg1 tp01

ENABLED ENABLED ENABLED ENABLED ENABLED ENABLED

ACTIVE ACTIVE ACTIVE ACTIVE ACTIVE ACTIVE

585105408 1172343808 98566144 8388608 396361728 192937984

SELECT SELECT SELECT SELECT SELECT SELECT

fsgen fsgen fsgen fsgen fsgen fsgen

UNIX Essentials: VVR Implementation

Storage Replication Log (SRL)

Data Change Map (DCM)

Disk Change Object (DCO)

Technical Implementation Steps/Procedures

UNIX Essentials: VVR Implementation

Install VVR packages on all nodes for both sites

# mount software:/repos /mnt # cd /mnt/software/os/SunOS/middleware/Veritas4.1/volume_replicator

UNIX Essentials: VVR Implementation

4. But this document follows the manual process.

# pkgadd -d . VRTSvlic VRTSvxvm VRTSob VRTSvcsvr

6. Install the following patch:

7. Install the following required packages in the specified order:

# pkgadd -d . VRTSvmpro VRTSvrpro

8. Use the following command to install the optional software packages:

Create a Diskgroup and volumes in the secondary site

UNIX Essentials: VVR Implementation

sunsrv01-dr:# sunsrv01-dr:# sunsrv01-dr:# sunsrv01-dr:# sunsrv01-dr:# sunsrv01-dr:#

vxassist vxassist vxassist vxassist vxassist vxassist

db2dg db2dg db2dg db2dg db2dg db2dg

addlog addlog addlog addlog addlog addlog

bcp db dba db2 lg1 tp01

logtype=dcm logtype=dcm logtype=dcm logtype=dcm logtype=dcm logtype=dcm

nlog=1 nlog=1 nlog=1 nlog=1 nlog=1 nlog=1

db2dg95 db2dg95 db2dg95 db2dg95 db2dg95 db2dg95

Update the .rdg file on both nodes in the secondary site

Stop all applications both clustered and non-clustered apps

Unmount all filesystems whose underlying volumes will be replicated.

# umount ` mount | grep '/dev/vx/dsk/db2dg/' | awk '{ print $1 }'`

# hastop all -force

Bring up the virtual IPs on all nodes on both sites

UNIX Essentials: VVR Implementation

Primary (sunsrv01 or sunsrv02):

ce3:1 - 10.11.196.191 netmask fffffe00 broadcast 10.11.197.255

Secondary (sunsrv01-dr or sunsrv02-dr):

ce5:1 - 10.12.94.191 netmask fffffe00 broadcast 10.12.95.255

Create the SRL Volume in the primary site

sunsrv01:# sunsrv01:# sunsrv01:# sunsrv01:# sunsrv01:# sunsrv01:#

vxassist vxassist vxassist vxassist vxassist vxassist

db2dg db2dg db2dg db2dg db2dg db2dg

addlog addlog addlog addlog addlog addlog

bcp db dba db2 lg1 tp01

logtype=dcm logtype=dcm logtype=dcm logtype=dcm logtype=dcm logtype=dcm

nlog=1 nlog=1 nlog=1 nlog=1 nlog=1 nlog=1

db2dg95 db2dg95 db2dg95 db2dg95 db2dg95 db2dg95

Update the .rdg file on both nodes in the primary site

Ensure that VVRTypes.cf exists in /etc/VRTSvcs/conf

UNIX Essentials: VVR Implementation

Ensure that other application types i.e. Db2udbTypes.cf exists in /etc/VRTSvcs/conf