VERITAS Volume Manager: An Overview

Here's an excerpt from the Sun web page (www.sun.com) regarding VERITAS:
VERITAS Volume Manager software is an advanced, system-level disk and storage
array solution that alleviates downtime during system maintenance by enabling easy,
online disk administration and configuration. The product also helps ensure data
integrity and high availability by offering fast failure recovery and fault tolerant
features. VERITAS Volume Manager software provides easy-to-use, online storage
management for enterprise computing and emerging Storage Area Network (SAN)
environments. Through the support of RAID redundancy techniques, VERITAS
Volume Manager software helps protect against disk and hardware failures, while
providing the flexibility to extend the capabilities of existing hardware. By providing
a logical volume management layer, VERITAS Volume Manager overcomes the
physical restriction imposed by hardware disk devices.
So, there you have it. But let me briefly list some of the things they totally missed.

Support for: Simple, RAID0, RAID1, RAID0+1, RAID1+0, RAID5

Dynamic Multipathing (DMP): Load balancing and redundant I/O paths to
disk arrays supporting multi-controller attachment.

Online Relayout: VERITAS allows you to change the layout of a VERITAS

volume while it is live and mounted. Change a RAID0 to a RAID5 without a
second of downtime!

Snapshoting: Take a snapshot of your data, creating a "shadow" of it which

you can use for online backups.

Hot Relocation: Designate "spares" which will take the place of failed disks
on-the-fly.

Dirty Region Logging (DRL): Volume transaction logs which provide fast
recoveries after system crashes.

More...... more than I can list!

So lets start by building on all that. VERITAS Volume Manager is a product of

VERITAS Software Corporation (www.VERITAS.com), which is used by Sun and
also sold to Sun customers under a Sun part number. VERITAS is also can "Vx" or
"VRTS" (the prefix of VERITAS packages, and VERITAS' ticker symbol). VERITAS
Volume Manager is often abbreviated "VxVM". Sun, before VERITAS 3.x, had its
own marketing names for VERITAS, the names you'll see and the version numbers
are:

<= Vx 2.5: Sun Enterprise Volume Manager (SEVM)

= Vx 2.6: Sun StorEdge Volume Manager (Sun SEVM)

> Vx 3.x: VERITAS Volume Manger (VxVM)

As for OS support:

SEVM 2.5: Solaris 2.5.1 and older.

SEVM 2.6: Solaris 2.6 and Solaris 2.5.x

VxVM 3.0.2: Solaris 7, Solaris 2.6, and Solaris 2.5.x

VxVM 3.0.3: Solaris 8, Solaris 7, and Solaris 2.6

Let me add, when Sun and/or VERITAS says "OS xxx isn't supported", they don't
really mean "Ummm, we haven't really tried it yet", like most other companies in the
world. VERITAS actually installs kernel modules into the system, so if you run
SEVM 2.6 on Solaris 7, you'll be running Solaris 2.6 kernel modules in a Solaris 7
kernel. It doesn't work, I tried it! (and Sun was really pissed when they found out I
tried it )

VERITAS Volume Manager and Vx Objects

VERITAS is a logical volume manager. The operative word here is "logical". So, in
order to keep things in order we have some basic pieces which are used in different
ways for building volumes. The better you understand these guys, the better you'll
understand VERITAS. Here's the list of objects:

Disk: This is a normal physical disk with a SCSI id. (c0t0d0...)

VM Disk (dm): A disk that is put under Vx control.

Sub Disk (sd): A section of VM disk used to build plexes.

Plex (pl): A mirror.

Volume (v): A virtual disk, which can contain data.

Now, lets talk about these objects a bit. A disk is nothing new or magical. When we
want to use a disk in VERITAS we need to turn it over to VERITAS control. A disk
turned over to VERITAS control is then given a VERITAS name (like disk01). After
doing this the disk is no longer available to the system for anything outside of use
inside VERITAS. Also, when we turn disks over to VERITAS control we don't turn
over a partition (c0t0d0s0), but the whole disk itself (c0t0d0). Now that we've got a
VM Disk we then create a Sub Disk from the VM Disk. Think of a subdisk as a
VERITAS partition. We could make one big subdisk from the VM Disk and use the
whole disk, or we could create a bunch of smaller subdisks. You can divide VM Disks
into subdisks however you like. From subdisks (one or more) we create what is called
a plex. Plexes are confusing so let's talk about these in some length.
Say the following out loud: "A Plex is a Mirror. A Plex is a Mirror. A Plex is a
Mirror." Maybe you should do that a couple times. You may feel silly, but plexes are a
sort of cornerstone in VERITAS. A Volume is a container, in VERITAS. The volume
is made up of one or more plexes. See the catch? A Plex is a mirror, yes, however you
can make a volume with only one plex. Is a volume made with only one plex

mirrored? No. We'll explain this more later, but for the time being keep it in your
head. So, subdisks are grouped together into a plex. The interesting thing is that in
VERITAS the plexes do all the work, so lets say you wanted to create a Striped
volume. You would actually create a striped plex, and then attach that striped plex to a
volume. The volume doesn't care, it's just a container. See the beauty here? Let's put
all this stuff together and build an imaginary volume in VERITAS.
We're going to build a striped (RAID0) volume from 2 9G disks. We'll say that the
first disk is c1t0d0, and the second is c1t1d0. First, we need to put them in VERITAS
control, so we create VM disks. The VM disks are then named disk01, and disk02.
Next, we'll create subdisks using these two disks. Let's use the whole disks and just
create 2 subdisks, one for each VM disk. We'll call disk01's subdisk "disk01-01", and
disk02's subdisk "disk02-01". Now, it's plex time! We're going to build a striped plex
using our two subdisks, which we'll call "myplex". (Don't worry yet about how we
create the plex, just get the concepts now.) So now we've got a plex, which contains
the subdisks "disk01-01" and "disk02-01". Now, we create a volume named "myvol"
using "myplex". And bingo! We've got a striped 18G volume ready to create a file
system on! Maybe, if we used the short names mentioned earlier (with the list of
objects) and make an outline it'd look something like this:
dm
dm

disk01
disk02

c1t0d0 <-- VM Disk named "disk01" made from "c1t0d0"

c1t1d0 <-- VM named "disk02" made from "c1t1d0"

sd
disk01-01 disk01 <-- Subdisk named "disk01-01" made from
"disk01"
sd
disk02-01 disk02 <-- Subdisk named "disk02-01" made from
"disk02"
pl
sd
sd

myplex striped <-- Striped Plex named "myplex"

disk01-01
<-- Made using subdisk "disk01-01"
disk02-01
<-and subdisk "disk02-01"

v
pl
from...
sd
sd

myvol
myplex striped

<-- Volume made from....

<-- the striped plex named "myplex", made

disk01-01
disk02-01

<-- Subdisk "disk01-01", and...

<-"disk02-01"

Look OK? Because if it does, take a look at this, real output from VERITAS, from a
real volume I created on my test machine:
v
pl
sd
sd
-

myvol

fsgen

ENABLED

35356957 -

ACTIVE

myplex

myvol

ENABLED

35357021 -

ACTIVE

disk01-01

myplex

ENABLED

17678493 0

disk02-01

myplex

ENABLED

17678493 0

Does that make any sense? Any at all? I hope it does. And if it does, you're ready for
VERITAS. We're going to explain more, much more, as we roll along, but at least
understand that:
Volumes are made up of plexes.

Plexes are made up of subdisks.

Subdisks are made up of VM Disks.
VM Disks are made up of (real) Disks.
and,
Disks can be turned into VM Disks.
Vm Disks can be turned into Subdisks.
Subdisks can be grouped into Plexes.
Plexes can be grouped into a Volume.

Good. One more note about plexes before we move on. Here's the groovy thing about
plexes. Because plexes are "mirrors", we can mirror the volume we built earlier
simply by building another plex identical to the first one, using two more subdisks
(which means we need 2 more vmdisks, etc,etc). Once we build the second plex, we
attach it to the volume (myvol) and presto chango! We're mirrored! This is really
kool... mirrors are something magical, they are a part of everything. If you have only
one mirror you may see yourself, but you'll need a second mirror to see yourself
holding the mirror. Hmmm?

VERITAS Object Creation

In VERITAS there are two ways to create objects. There's the "vxmake" way and the
"vxassist" way. Here's the difference...
VxMake is an object creation tool that allows you to specifically build objects, one at
a time, by your specific specs. The methods we mentioned before (add a disk, make a
VM disk, create a subdisk, then...) would be an example of how we use vxmake.
Piece by piece, we'd build out components, and assemble them. This is kool because
there is no "guess work", meaning we are explicitly saying "build this!" and it's built.
We'll learn more about vxmake, in detail, in the next course.
VxAssist is the other way. As you've been reading this course, have you said "this sure
looks like a lot of work!"? Well, here's your answer: vxassist. When we want to build
a volume, but we don't want to specify every little detail, we could add disks to make
VM disks and then use vxassist to do the rest. So instead of using the "build this, and
only this!" philosophy of vxmake, we turn to the vxassist mind set that sounds
something like "OK, here are some disks, build me a RAID5 that can house 10G of
data... I don't care how you do it, just hurry up." Vxassist is handy for a jam, or when
you really just don't care exactly how things are set up. Again, in our following course
we'll discuss object creation at length using vxassist.
Don't get any ideas yet about which one is better. When we get to the courses
applicable to these tools and you see how they work with your own eyes (yup, you'll
see lots of examples) then you can get defensive about your favorite. But also
remember, you can use them both interchangeably in some cases, and often with use
both. Keep your eyes open for these guys though, because they are very different
ways of doing things.

Special Notes on RAID0 and RAID5: Layouts

I want to briefly discuss layouts. This is a topic will delve into more thoroughly later
when we talk about plex creation, but because the "VERITAS Getting Started Guide"
talks about them, I'll give you the highlights. This stuff may seem pretty basic now,
but the more you work will VERITAS the more you will come to appreciate layouts.
Layout can be thought of as the defining difference between the different types of
RAID. Simple RAID's don't really have a layout, you simple write linearly starting at
the beginning of the first disk until you hit the end, at this point you start writing the
second disk, and so on. Striped RAID's have special type of layout, however. Layouts
use the following terms:

stripe width (stwidth): The Amount of data to write to a disk before moving to
the next. (Generally 32k by default)

columns (ncolumn): A subdisk.

In the case of a Striped RAID, each subdisk would have a column number. A RAID0
with 4 disks, would have 4 columns. The first disk would be column 0 (remember:
everything starts with 0), the second would be column 1, the third would be column 2,
and the fourth disk would be column 3. Now, when we build a striped RAID we
specify both stripe width (which VERITAS calls "stwidth"), and the number of
columns (which VERITAS calls "ncolumns") which is the number of subdisks that are
going to have data on them. So, as mentioned earlier, we've got 4 disks, and we know
the column numbers for each subdisk. If we assigned a stripe width of 32k, and we
wrote some data to the volume, we'd write the data 32k at a time, writing each new
32k chunk to the next column, and wrapping around from the 4th column (column 3)
to the 1st column (column 0), untill all the data is written. Refer to the "RAID
Theory" course for more on this. If you've got this down, but feel foggy, just roll with
it, it'll click later as we build plexes.
Now, RAID5 is very similar because it's striped, it's just a different type of stripe.
RAID5 uses a layout called "Left Symmetric Layout", or more commonly as simply
"RAID5 layout". It would take me too much time to explain this, so check the book.
Read through the "RAID Theory" course if you need help, and then go back to the
book. You may have to look twice, but you'll get it.

Volumes and Devices

So now that you understand how a volume is built, and you understand a little about
how it works lets talk about how you use them.
As mentioned earlier, a volume is a container. A "bit bucket" is you will. The
advantage of a volume over a normal disk partition, is that volumes aren't static. You
can grow, shrink, move, rearrange, and manipulate volumes in ways you just can't
with a partition. You can also do all this in a mounted and live environment with no
reboots. When you don't have enough disk space in a filesystem you can just grow the
volume that the filesystem is on, and then grow the filesystem. Very easy.

But... you're saying to yourself, "how do I actually use a volume?" Well, that's easy
enough. You use a volume just like a disk block device. Once VERITAS is installed
and set up, you'll see a new directory in /dev. You'll see /dev/vx/, which is where all of
your volume block devices are. If you created a volume named "myvol", and wanted
to "newfs" it, you could do it like this:
# newfs /dev/vx/rdsk/myvol

or you could mount it like this:

# mount /dev/vx/dsk/myvol /myvol

Kool huh? Nothing magical. Poke around in the /dev/vx file tree and see where things
are, everything is fairly well named, and self explanatory. If you're new to device
trees, then just know that you can access your volumes from /dev/vx/dsk/ as though it
were a normal disk.
See how simple and logical VERITAS is? This is want makes VERITAS so kool.
There's no voodoo whoodoo. Everything is clearly laid out, and organized. I love
VxVM and I'm hoping that you're starting to as well.

Kernels and Daemons

One thing that separates VERITAS from other volume managers is its integration with
the host operating system. VERITAS actually utilizes kernel modules, which are the
backbone of the application. You can see some of the driver loads in /etc/system.
You'll see several driver forceloads. This is important to note because on most systems
you'll see "WARNING: Forceload of dev/XXX Failed!" messages at boot up. Don't
worry about these, it just means that VERITAS is trying to load a driver for a piece of
hardware you don't have.
I wanted to talk about VERITAS Installation, but I'm going to leave that to the book,
but be VERY careful on VERITAS and Solaris upgrades to a system using VERITAS.
Remember that because VERITAS uses kernel modules, that if you should wipe out
and/or replace the kernel you could loose VERITAS.... which means you'll get a very
intimate session alone with the books and the system while you learn how to recover
botched VERITAS installations. Please, be informed about this, I've killed a couple
machines this way, and luckily they were all "test" machines and not production
servers.
VERITAS also has several daemons running all the time to keep watch over your
volumes, and to ensure smooth operation. To look for them try typing "ps -ef | grep
vx" on a machine running VERITAS. You generally will see 3 common daemons,
which are:

vxiod : The VERITAS IO Manager, which managers reads and writes to

volumes.
vxconfigd : The Vx Object Configuration Manager.

vxrelocd : The Vx Hot Swap Manager, which reacts to disk failures with
spares.

Maybe, by now your asking "Hurry up, just tell me where the VERITAS
configuration file is!". Whelp, there isn't one! Everything we do to VERITAS is watch
by the vxconfigd. The vxconfigd then updates VERITAS's object database, and keeps
watching. As for the other daemons, they are pretty self explanatory. Vxiod makes
sure that data is written to the volume the way it's supposed to be (so if the volume is
striped, vxiod makes sure the data is written to the stripe the way we told it too, etc..).
Vxrelocd just looks for disk failures and if it finds one it puts a spare (if there is one)
in it's place

Intro to Advanced Features

As we wrap this up, lets talk about VERITAS's truly special, and more "advanced"
features. Please don't for an instant think this is the complete and definitive list.
Rather, this is a small list of features that I really think separate VxVM from the other
volume managers available. All of these topics will be covered in detail in the
"Advanced VERITAS Theory" course later on.

Dynamic Multi-Pathing: Also know as DMP, this feature allows for arrays
which can be attached to multiple controllers (such as the Sun A5000 family
of arrays) to leverage them for both load balancing and redundancy. So if your
disks are attached to both c1 and c2, and your using a 100Mb/s fiber
connection, with DMP you'll access the disks at 200Mb/s. And if c1 dies, for
any reason, you still have access to the disk through c2, which means zero
downtime for your volume. DMP can really save your butt if there is a failure.
Hot Spares: You can allocate VM Disks as spares, so that if a disk in one of
your plexes dies for some reason the spare will immediately take it's place and
again, zero downtime. You can have as many hot spares as you like, and VM
Disks allocated as spares can are independent of any particular disk group,
plex or volume, so you don't need to have different spares for different
volumes. Spares are managed by the vxrelocd as mentioned earlier.

Dirty Region Logging: Also known as DRL, this feature can be thought of as a
volumes diary. Before writes are made to a DRL'ed volume VERITAS makes
a note of the write in the DRL, marking that region "dirty". After the note is
made in the log, the write occurs. The idea here is that if the system crashes,
when the system comes back online it won't know what writes were in
progress. So, if a DRL log exists for the volume, it checks the log and then
sync's dirty regions from the other mirror(s). In this way, we don't have to
worry about corrupted data. DRL only applies to mirrored volumes, even
though DRL can be added to unmirrored volumes (it just won't do anything
useful).

Snapshotting: Users of NetApp Filers will be familiar with this. Snapshotting

is used for online backups. When you "snapshot" a volume, you are saying "I
want to create a static picture of what the volume looks like right now". Then
once the snapshot is created it appears as a new temporary volume, which can

be mounted and backed up. Once the backup is done you "release" the
snapshot. The kool thing is that while the snapshot exists the volume is still
read and writable to everyone else. Zero downtime backups!
OK, I'm going to leave the list at that for now. Don't get frustrated if your thinking
"Hey, that's kool! But your not telling me anything useful!", because all of that will
come later in the "Advanced VERITAS Theory" course, where I will talk about these
topics and more in detail. For the meantime, just know that they exists and can be
used.