Installing Ganeti on a Redhat 6 or Scientific Linux 6 system

By jbayer - Last updated: Thursday, May 12, 2011 - Save & Share - 5 Comments

Ganeti is a cluster virtual server management software tool built on top of existing virtualization technologies such as Xen or KVM and other Open Source software.  It is written by Google, and is available from the Google code site.

The instructions on the code site refer to installing on a Debian-based system.  We are a Redhat shop, using either CentOS or Scientific Linux as the base os; both are rebrandings of the Redhat RPMs.

After googling for a while, I found a japanese site, http://jfut.integ.jp/linux/ganeti,  which had source RPMs for Redhat 5 & 6, as well as written instructions in english.  These instructions, while mostly complete, still assumes that the user do some tasks without being told.

During my work, I had to install Ganeti many times, and ended up writing a script to do the entire install, starting with a minimal or base desktop install on Scientific Linux 6.  While not tested, the script should work on Redhat 6 as well.  My script incorporates the instructions from jfut.integ.jp, as well as other steps and some code of my own to setup the bridges.  The only other requirement is access to the ‘net.

All files are available here, as well as the original Japan locations:

bridge.sh

  bridge.sh (1.5 KiB, 907 hits)

calculate-address

  calculate-address (1,013 bytes, 1,045 hits)

ganeti rpm

  ganeti-2.4.2-1.el6.noarch.rpm (1.1 MiB, 499 hits)

ganeti debootstrap

  ganeti-instance-debootstrap-0.7-2.el5.noarch.rpm (19.0 KiB, 533 hits)

ganeti install script (updated 6/20/11)

  ganeti.sh (32.1 KiB, 865 hits)

Using the script

I like using both the screen program and Free-NX.  If they aren’t installed my script asks if you want them installed.

In all questions, if there are answers in parentheses, pressing <return> will enter the default answer, which will be inside the parentheses.  If there are two choices in side the parentheses, the default will be in caps.

The first two questions will be about the screen program and Free-NX, answer as you like, they are not needed for the install.

If you are using Scientific Linux, you will be asked about the Fastbug repo.  I usually answer yes; again, this is not required.

After a number of RPMs are installed the following questions are asked:

Enter primary network interface (eth0):

The following question does not have a default answer.  You must answer either M or S (case is not important):

Is this the Master server or a Slave node (M/S):

The hostname is obtained from the system, and presented here for confirmation:

Enter hostname (localhost.localdomain):
Enter cluster name:
Enter cluster address (must be unused):
Enter cluster netmask (255.255.255.0):

The cluster network is derived from the address and netmask, and presented here for confirmation:

Enter cluster network (10.50.10):
Enter domain:

The default prefix for the node name is: “node”:

Enter node name prefix (node):
Enter node starting address:
Enter number of nodes you want:

The default prefix for the instances is: “instance”:

Enter instance name prefix (instance):
Enter instance starting address:
Enter number of instances you want:

Some more modules are installed, then:

Enter device partition for the LVM partition
(ie: /dev/sda4, press <CR> to automatically find the LVM partition):

If the volume group is already created, you will get the following error:

Can't initialize physical volume "/dev/sda4" of volume group "vmvg" without -ff
Error running:  pvcreate /dev/sda4
Please fix the problem and try again
Press <CR> to retry, or C to continue with the next step:

Enter volume group name, must be identical on all nodes (vmvg):

Ganeti will be downloaded and installed now.

There is a bug/incompatibility with the certificates which Ganeti generates which causes an error at this point.  The error message will be:

Failure: command execution error:
Node daemon on dev-ha1.regcolo.com didn't answer queries within 10.0 seconds

Rc: 1
Do you need to fix the error now (Y/n):

Press <return> to have the script regenerate the certificates automatically.  Assuming you do, you will be presented with the following query, two times.  Just press <return> when asked:

Enter PEM pass phrase:

Finally, the last question will be about doing a full system update:

Would you like to do a full system update (Y/n):

At the end, the system is rebooted.

Manual Steps, if you like doing it yourself

Start off by doing a clean install of Scientific Linux 6.  Make the OS partition between 20 and 40 gig in size, swap as needed, and create an LVM partition where the virtual systems will reside.  Do not initialize the LVM partition, just create it and mark it as LVM.

Install the repos for epel, atrpms, elrepo and rpmforge.  You can use this command:

yum install epel-release atrpms-repo elrepo-release rpmforge-release

If you want to use my scripts to create the bridges, you will also need to install ksh, since one of the scripts is currently written in ksh instead of bash:

yum install ksh

Download and install the following two scripts in /usr/local/sbin:

calculate-address:

bridge.sh:

Turn off iptables:

chkconfig iptables off

and add the following to /etc/rc.local:

iptables --flush
/usr/local/sbin/bridge.sh br0 eth0

Don’t forget to make the scripts executable:

chmod +x /usr/local/sbin/bridge.sh /usr/local/sbin/calculate-address

Install drbd:

yum --enablerepo=elrepo install drbd kmod-drbd83

Make sure that a root .ssh directory.  You can do this either by making the directory, or just sshing to another system.

Add a drbd.conf file to /etc/modprobe.d as follows:

echo "options drbd minor_count=128 usermode_helper=/bin/true" >/etc/modprobe.d/drbd.conf

Install kvm:

install kvm python-virtinst

and some other packages:

yum --enablerepo=elrepo install kmod-drbd83
yum --enablerepo=epel install pyOpenSSL python-simplejson pyparsing python-inotify python-ctypes python-pycurl python-paramiko debootstrap socat
The volume group is required to be at least 20GiB.

If you haven’t configured your LVM volume group at install time you need to do it before trying to initialize the Ganeti cluster. This is done by formatting the devices/partitions you want to use for it and then adding them to the relevant volume group::

  pvcreate /dev/sda3
  vgcreate vmvg /dev/sda3

or:

  pvcreate /dev/sdb1
  pvcreate /dev/sdc1
  vgcreate vmvg /dev/sdb1 /dev/sdc1

Finally, download and install Ganeti:

wget http://jfut.integ.jp/linux/ganeti/noarch/ganeti-2.4.2-1.el6.noarch.rpm
wget http://jfut.integ.jp/linux/ganeti/noarch/ganeti-instance-debootstrap-0.7-2.el5.noarch.rpm
rpm -ivh ganeti-2.4.2-1.el6.noarch.rpm ganeti-instance-debootstrap-0.7-2.el5.noarch.rpm

At this point, Ganeti is installed.  The final step is to fix an SSL error by regenerating the certificates:

openssl req -new -x509 -days 1825 -keyout server-key.pem -out server-cert.pem
openssl rsa -in server-key.pem -out server-key-nopass.pem
cat server-key-nopass.pem server-cert.pem > /var/lib/ganeti/server.pem

openssl req -new -x509 -days 1825 -keyout rapi-key.pem -out rapi-cert.pem
openssl rsa -in rapi-key.pem -out rapi-key-nopass.pem
cat rapi-key-nopass.pem rapi-cert.pem > /var/lib/ganeti/rapi.pem

and finally start Ganeti:

service ganeti start
gnt-cluster modify --hypervisor-parameters kvm:kernel_path=
Posted in Administration, Building packages, Linux Installations, Open Source • • Top Of Page
5,644 views

5 Responses to “Installing Ganeti on a Redhat 6 or Scientific Linux 6 system”

Comment from Jun Futagawa
Time May 13, 2011 at 4:53 am

Thank you for using my RPMs and writing additional instructions.
I put new RPM for Ganeti 2.4.2.

Comment from jbayer
Time May 13, 2011 at 8:23 am

Thank you. I’ve updated everything for the new RPM

Comment from pro cabales
Time September 26, 2011 at 6:51 am

@jbayer: nice article! our hypervisors are all running CentOS so this really helped me a lot.

@Jun Futagawa: thanks a lot for the RPMs 🙂 (i tried installing using source, got it working but it was hell!)

Comment from Chandan
Time January 10, 2013 at 4:55 pm

I am getting below error
[root@gkvm01 ~]# service ganeti restart
Stopping ganeti-confd: [ OK ]
Stopping ganeti-rapi: [ OK ]
Stopping ganeti-masterd: [ OK ]
Stopping ganeti-noded: [ OK ]
Starting ganeti-noded: [ OK ]
Starting ganeti-masterd: ERROR:root:Can’t activate master IP address: Failed to start the master IP. Script return value: 1
[ OK ]
Starting ganeti-rapi: [ OK ]
Starting ganeti-confd: [ OK ]

Comment from jbayer
Time January 11, 2013 at 7:41 am

That is OK. What is happening is that Ganeti activated the master IP address when it first started, and doesn’t deactivate it when stopping. So when you restart it, it get an error when trying to activate the address again, since it is already activated.

JBB

Write a comment

asd