Xen and Ubuntu?

Question

How does one properly approach having Ubuntu servers on a Xen hypervisor? I don't have any experience with RAID or Xen other than from a theoretical level.

Additional requirements:

• Use with mdadm Software RAID 5 (can be on separate disks) that multiple instances with access
• Paravirtualized Ubuntu Server guests instances

Possible ideas for now:

1. Ubuntu host (dom0) with ubuntu-xen-server package (this purportedly isn't supported)
   • dom0 host will "own" RAID 5 partition
   • More Ubuntu servers as guests
2. Citrix XenServer bare-metal host
   • XenServer will own RAID
3. Citrix XenServer bare-metal host
   • Ubuntu guest instance creates and owns RAID

Questions and concerns:

1. Can Ubuntu be used as a dom0 Xen host?
2. Can XenServer install packages such as mdadm and create a partition?
3. Can multiple guest access (R + W) to the same data partition (RAID)?

Note: since it may have a bearing on support, I'm referring to Ubuntu Server 10.04

Answer 1

Answers:

1. Ubuntu can be used as a dom0 host. I have used rhel5 as xen dom0 host on top of it i installed windows server 2k8 r2 and it works fine. I know rhel license is violated, but anyway i dont use rhel 5 support contract. i help myself and from the community. The steps i took to install xen on rhel with win serv 2k8 is documented in my blog in this url http://maximus2882.blogspot.com/2010/10/windows-2008-enterprise-on-rhel-55.html . Maybe that is of some use to you. It must be similar if not easier.
2. Xenserver does not install mdadm, it comes from the os package, in this ubuntu. You must do a #sudo apt-get install mdadm and use mdadm command to create RAID partition.
3. Multiple guest can share a same datastore but cannot share the same files. But can live in the same partition. the partitions and data in guest os is stored as files in host os. So they can happily share the same partition. I think there are ways to make a certain dir share between host os and guest os. I have done that in virtualbox in windows seven as host and fedora 14 as guest. Not sure how to do the same in xen , as i have limited with xen. I would suggest you go with vmware if you can. As my experience with vmware shows it is much matured and is the industry leader and has a smooth and consistent operation and interface that doesn't fail much.

Answer 2

If you use software raid on Linux, you should create a partition for your dom0 instance and a big partition with the rest of the hard drive.

Then, you use this big partition as a lvm physical drive and you create one or more lvm logical drive per vm.

This enable you to increase/decrease easily the size of a particular logicial drive.

Answer 3

1. Can Ubuntu be used as a dom0 Xen host?

Set up Ubuntu 10.04 Server PV DomU at Xen 4.0 Dom0

Set up Libvirt 0.8.1 & Xen 4.0.1 Dom0

2. Can XenServer install packages such as mdadm and create a partition?

Installing can be little bit tricky but i might work. XenServer use RHEL 5 with custom kernel from rhel box you can ldd /path/to/mdadm and copy all linked libraries to XenServer on exact same locations. I might work ;)

**3. Can multiple guest access (R + W) to the same data partition (RAID)?****

I'm sure that you knew the answer.

Answer 4

Well in my experience, I just leave the details to libvirt. It hides the virtualization details for me. Typical setup would be to install a stock ubuntu, then libvirt and md tools. Assuming you have devices for your raid make sure that they have the right system (linux raid autodetect) in fdisk. Start fdisk, choose the partition/device, the use the t command to change it's type. An example for raid 1 is shown below. Adjust the level in your setup to 5 to get raid 5

root@mail:/etc/ldap# fdisk -l

Disk /dev/sda: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1        2433    19543041   83  Linux
/dev/sda2            2434        3042     4891792+  83  Linux
/dev/sda3            3043        3286     1959930   82  Linux swap / Solaris
/dev/sda4            3287        9729    51753397+  fd  Linux raid autodetect

Disk /dev/sdb: 80.0 GB, 80032038912 bytes
255 heads, 63 sectors/track, 9730 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        2433    19543041   83  Linux
/dev/sdb2            2434        3042     4891792+  83  Linux
/dev/sdb3            3043        3286     1959930   82  Linux swap / Solaris
/dev/sdb4            3287        9729    51753397+  fd  Linux raid autodetect

**You may need to modprobe the right raid module**
raid0   raid1   raid10  raid5   raid6
root@mail:/etc/ldap# modprobe raid1

**Create your array**
root@mail:/etc/ldap#  mdadm --create /dev/md0   --level=1  --raid-devices=2 /dev/sda4 /dev/sdb4
mdadm: array /dev/md0 started.
root@mail:/etc/ldap# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdb4[1] sda4[0]
      51753280 blocks [2/2] [UU]
      [>....................]  resync =  0.9% (466176/51753280) finish=14.6min speed=58272K/sec

unused devices: <none>
root@mail:/etc/ldap# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdb4[1] sda4[0]
      51753280 blocks [2/2] [UU]
      [==>..................]  resync = 11.5% (5956160/51753280) finish=13.6min speed=55983K/sec

unused devices: <none>

You can start using you array even as it's sync'ing. So we start with lvm: Typically, you need to setup a PV, then a vg and finally your LV for individual VMs.

Create your physical volume using pvcreate, VG and LV

# pvcreate /dev/md0  
# vgcreate xen02 /dev/md0
# lvcreate -L 100G -n nosql.example.net.storage /dev/xen02

then view your handiwork

xen02:~# pvs
  PV         VG    Fmt  Attr PSize   PFree
  /dev/md0   xen02 lvm2 a-   462.49g 273.52g
xen02:~# pvdisplay
  --- Physical volume ---
  PV Name               /dev/md0
  VG Name               xen02
  PV Size               462.50 GiB / not usable 3.81 MiB
  Allocatable           yes
  PE Size               4.00 MiB
  Total PE              118398
  Free PE               70022
  Allocated PE          48376
  PV UUID               ,.........

xen02:~# vgdisplay
  --- Volume group ---
  VG Name               xen02
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  30
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                13
  Open LV               9
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               462.49 GiB
  PE Size               4.00 MiB
  Total PE              118398
  Alloc PE / Size       48376 / 188.97 GiB
  Free  PE / Size       70022 / 273.52 GiB
  VG UUID               ,,,,,......

xen02:~# vgs
  VG    #PV #LV #SN Attr   VSize   VFree
  xen02   1  13   0 wz--n- 462.49g 273.52g

xen02:~# lvs
  LV                                  VG    Attr   LSize  Origin Snap%  Move Log Copy%  Convert
  acme.example.net.root            xen02 -wi-ao 15.00g
  ca.example.net.root              xen02 -wi-ao 15.00g
  externalra.example.net.root      xen02 -wi-ao 15.00g
  box2.example.net.root          xen02 -wi-ao 15.00g

You may now use libvirt to install over such a typical LV (notice that you can change the virt-type from kvm to xen.. :) ) - You may need to remove the virtio parameter as this is KVM specific.

virt-install --name=virtio_template \
--ram 1024 \
--os-type linux --os-variant debiansqueeze \
--disk path=/dev/xen02/nosql.example.net.storage,bus=virtio \
--network network=default,model=virtio \
--vnc \
--noautoconsole \
--cdrom ~/debian-testing-amd64-netinst.iso \
--virt-type kvm \
--force yes

What I typically do is create a template VM and clone it to create worker vms (--conect might be kvm specific):

#!/bin/sh
 # invoke as ./script "machine3"
MACHINENAME="Somemachine"

lvcreate -L 15G -n  "$MACHINENAME".example.net.root  /dev/xen02


echo Cloning template to $MACHINENAME
virt-clone \
        --prompt \
        --connect qemu:///system \
        --original template_master \
        --name "$MACHINENAME" \
        --file /dev/xen02/"$MACHINENAME".example.net.root 

Now you can have fun using the virsh commands. See it's help.

virsh start $MachineName

virsh shutdown $MachineName

Happy VM'ing