Virtual Servers

Virtual Server Creation, and Deployment

Introduction

There are two methods to create vserver nodes on the gentoo vserver host. Each method begins differently, but the final setup is the same. The directions contained in this note may appear saturated by the words vserver, vsrvr, vservers, and vs0n. Due to the default vserver directory naming conventions, and the vserver node naming convention (i.e. vs00, vs01, etc.) this name-clutter is unavoidable. If setup does not follow spec, please examine your syntax carefully.

Preparation

There are two distinct steps to manually-installing a vserver:

• Host system. We are using Gentoo Linux for the Host; the purpose of the Host is to run the Guest Operating Systems
• Guest system(s). These do the actual work, such as web-serving, or perhaps Samba, or maybe OpenLDAP authentication.

Sometimes, there are grey areas regarding what services should run on the Host, or the Guest.

Host Preparation

• The host environment has only one significant change over a normal host environment - it uses the vserver kernel. The profile must not be a vserver profile on the host.

Examples (current for mid-December 2007):

vsrvr_host ~ # uname -r
2.6.35-vs2.3.0.36.32-gentoo     for example

vsrvr_host ~ # eselect profile show
Current make.profile symlink:
  default/linux/amd64/10.0     of course, your architecture may differ

• The Guests will live under /vservers. So, make sure you either:
  • have enough room under your / root-filesystem for your guest(s)
  • prepare and mount another device (drive, volume) under /vservers. Check that this "sticks" properly after a reboot (no /etc/fstab errors)

Guest Preparation (NOT from a template)

We will use our familiar Gentoo init-style, so grab the Stage 3 bzipped tarball for your architecture from your favourite Gentoo mirror. In this example, the Intel Core2Quad CPU uses the amd64 stage (and profile):

vsrvr_host vservers  # wget http://distfiles.gentoo.org/releases/amd64/autobuilds/current-stage3/stage3-amd64-20130110.tar.bz2

Once the gzipped tarball is downloaded, you follow the VServer HowTo instructions to extract and set up your guest. "Magic" words in the example below are:

• myguest -> basilisk this is the name I wanted for my (file-)server, so I have applied it in two places:
  • in the first instance, this means that the base-directory for my Guest will be /vservers/basilisk. Having the Host's container-directory share the same name as the Guest hostname made sense to me.
  • the second instance will set the hostname in various (usual) places inside the Guest (/etc/conf.d/hostname for one)
• context 30 This must be unique, and not collide with any other Guest context. That's it; a random, unique number. You could use shoe-size, or IQ, but I chose to first figure out the IP-address I wanted for this (file-)server, and then use the last portion for my context number. The next Guest may use context 40 and an IP of 192.168.0.40 with my scheme; feel free to do whatever you want, but make the context values unique!!

Here is an example Guest installation, for reference:

vsrvr_host vservers  # vserver basilisk build --context 30 --hostname basilisk --interface eth0:192.168.0.30/24 --initstyle gentoo -m template -- -d gentoo -t /vservers/stage4-amd64-20070905.tar.bz2
>>> Adding shared /usr/portage to fstab ... 
>>> Checking init-style ... gentoo
>>> Unpacking template ... ok
>>> Installing special init-style magic ... 
!!!
!!! You have to install a service (e.g. syslog/cron) and add it to the
!!! default runlevel before you start the guest the first time!
!!! Otherwise the guest will die as soon as it has finished booting.
!!!
!!! Consult the Gentoo Handbook on how to chroot and install
!!! packages into the guest environment.
!!!
>>> Fixing default runlevel scripts ... 
>>> Setting hostname ... 
>>> Fixing syslog-ng.conf ... 
>>> Fixing inittab ... 
>>> Fixing fstab ... 
>>> Providing dummy net dependency ... 
vsrvr_host vservers  #

Another Installation-Command Example

vsrvr_host vservers  # vserver mediatomb build --context 40 --hostname mediatomb --interface bond0:192.168.0.140/24 --initstyle gentoo -m template -- -d gentoo -t /vservers/stage4-amd64-20070905.tar.bz2

Yet another Installation-Command Example, this time a Debian guest on our Gentoo host... we definitely need to specify the architecture, because debootstrap will obtain the host's arch as GenuineIntel (or whatever CPU), which won't be recognized properly during installation :-(

vsrvr_host vservers  # vserver eprints build -m debootstrap --context 23 --hostname eprints.iat.sfu.ca --interface bond0:209.87.56.23/24 --initstyle sysv -- -d squeeze -m  http://ftp.de.debian.org/debian -- --arch amd64

Following the instructions above, we chroot into the new vserver guest and prepare a few things:

vserver_host root # cd /vservers/new_vserver
vserver_host /vservers # mount -t proc proc /vservers/new_vserver/proc
vserver_host /vservers # chroot /vservers/new_vserver /bin/bash
new_vserver_guest / # env-update
new_vserver_guest / # source /etc/profile
new_vserver_guest / # export PS1="(chroot) $PS1"

new_vserver_guest / # ln -s /etc/init.d/net.lo /etc/init.d/net.bond0     or net.eth0 or whatever.  None of these seem to exist at this time
new_vserver_guest / # emerge --sync
new_vserver_guest / # ln -sf /usr/share/zoneinfo/America/Vancouver /etc/localtime
new_vserver_guest / # emerge sys-process/vixie-cron
new_vserver_guest / # rc-update add vixie-cron default
new_vserver_guest / # exit
vserver_host /vservers # umount /vservers/new_vserver/proc

Hit the ignition and see if it runs ;-)

vserver_host /vservers # vserver new_vserver_guest start
    OpenRC 0.11.8 is starting up Gentoo Linux (x86_64) [VSERVER]

* /proc is already mounted
* /run/openrc: creating directory
* /run/lock: creating directory
* /run/lock: correcting owner
* Caching service dependencies ...                                                                                        [ ok ]
* Creating user login records ...                                                                                         [ ok ]
* Cleaning /var/run ...                                                                                                   [ ok ]
* Wiping /tmp directory ...                                                                                               [ ok ]
* Setting hostname to owncloud.iat.sfu.ca ...                                                                             [ ok ]
* Updating /etc/mtab ...                                                                                                  [ ok ]
* setting up tmpfiles.d entries ...                                                                                       [ ok ]
* Starting vixie-cron ...                                                                                                 [ ok ]
* Starting local                                                                                                                                              [ ok ]

syslog-ng won't run? That's because there is no access to klogd from inside a vserver-guest :-O So, stop syslog-ng from trying to read /proc/kmsg by commenting out the line shown below:

source src {
    unix-stream("/dev/log" max-connections(256));
    internal();
#    file("/proc/kmsg");
};

May as well also filter out the annoying/repetitive cron-timestamps:

destination messages { file("/var/log/messages"); };
# We will take out the annoying cron notifications
filter nocron { not facility(cron); };

log { source(src); filter(nocron); destination(messages); };

Make this Guest start each time you reboot the Host:

vsrvr_host vservers  # echo "default" > /etc/vservers/basilisk/apps/init/mark     of course, you have to use your own hostname, not my choice of basilisk!
vsrvr_host vservers  # rc-update add vservers.default default

Create a Vserver Template

vsrvr / # vserver vs0n enter

vs0n / # shutdown -h now

vsrvr / # cd /vservers/vs0n

vsrvr vs0n # tar cvf vs.template*.tar bin boot etc fastboot home lib mnt root sbin sys tmp usr var

• DO NOT INCLUDE /DEV OR /PROC

vsrvr vs0n # cp /vservers/vs0n/vs.template*.tar /vserver.template/

• Be careful to examine the /vserver.template/ directory for pre-existing vs.template*.tar files before naming any new tar files. vs.template.tar is the stock template, so name new templates vs.template.00.tar, etc. Using non-stock templates will have an impact on the *.tar name used in point 4 in the next section of this note.

• GOTO "Use the Vserver Stock Template"

Use the Vserver Stock Template

vsrvr / # vserver vs0n build -m skeleton --hostname vs0n --context 563n --interface vs0n=eth0:209.87.56.3n/24

• Make sure to check /vservers directory for pre-existing vs0n nodes matching the desired vnode name
• Make sure to correct/match every vs0n reference in the vnode creation line; there are 3 references.
• Make sure to correct/match the IP statement eth0:209.87.56.3n/24 with the IP stated in the context.
• Make sure the context matches the values of the last two octets of the target vnode IP

vsrvr / # cd /vservers/vs0n

vsrvr vs0n # tar -xvf /vserver.template/vs.template*.tar

Post Vnode Creation Setup

• Several directories will need to be customized, post vnode creation, to make the vnode unique.
• Changes must be made in the /etc/vservers/vs0n and the /vservers/vs0n before the new vnode will be ready to start.

Changes to /etc/vservers/vs0n

• To allow vnode to boot, we need to copy over init instructions.

vsrvr / # cp /vserver.template/init/* /etc/vservers/vs0n/apps/init/

• To allow for proper service initialization, we need to add eth:lo and renumber interface 0 and 1.

vsrvr / # mv /etc/vservers/vs0n/interfaces/0 /etc/vservers/vs0n/interfaces/1

vsrvr / # cp -r /vserver.template/interfaces/0 /etc/vservers/vs0n/interfaces/

vsrvr / # cp -r /vserver.template/fstab /etc/vservers/vs0n
cp: overwrite `/etc/vservers/vs0n/fstab'? yes

Changes to /vservers/vs0n

• To allow for proper service initialization, we need to change hostname and IP addresses in the following files.

vsrvr / # vi /vservers/vs0n/etc/ssh/sshd_config

vsrvr / # vi /vservers/vs0n/etc/conf.d/hostname

vsrvr / # vi /vservers/vs0n/etc/hosts

vsrvr / # vi /vservers/vs0n/etc/conf.d/net

vsrvr / # vi /vservers/vs0n/etc/conf.d/domainname

vsrvr / # vi /vservers/vs0n/etc/mysql/my.cnf

vsrvr / # vi /vservers/vs0n/etc/apache2/vhosts.d/00_default_vhost.conf

vsrvr / # vi /vservers/vs0n/etc/nullmailer/me

vsrvr / # vi /vservers/vs0n/etc/apache2/vhosts.d/00_default_ssl_vhost.conf

vsrvr / # vi /vservers/vs0n/usr/share/logwatch/custom_header

vsrvr / # vi /vservers/vs0n/etc/tripwire/twpol.txt

vsrvr / # mv /vservers/vs0n/var/lib/tripwire/vs0n.twd /vservers/vs0n/var/lib/tripwire/vs0n.twd

vsrvr / # mv /vservers/vs0n/etc/tripwire/vs0n-local.key /vservers/vs0n/etc/tripwire/vs0n-local.key

vsrvr / # vi  /vservers/vs0n/etc/awstats/awstats.awstats.conf

Start your Vserver

vsrvr / # vserver vs0n start
vsrvr / # vserver vs0n enter

Password

• Don't forget to set a new root password.

Post Creation tripwire Configuration

vsrvr / # twadmin --create-polfile -S /etc/tripwire/site.key /etc/tripwire/twpol.txt

vsrvr / # tripwire --init

vsrvr / # tripwire --check

Delete a Vserver

vsrvr / # rm -r -I /vservers/vs0n

vsrvr / # rm -r -I /etc/vservers/vs0n

Notes

When working with virtual servers, make sure you enter the virtual server before you do any work:

vsrvr / # vservers $servername enter

When you are done, just use the 'exit' command to return to the host.

Virtual Server Stock Software List

Here is a list of the significant software installed on our vserver guests:

Nagios monitoring: http, load, ntp, snmp, ssh, processes, uptime, users

Apache

Mysql

PHP

phpmyadmin

awstats

amanda

vsftp

logwatch

tripwire

Miscellaneous Virtual Server Commands

vnamespace -e 5654 mount -t tmpfs -o remount,size=256m,mode=1777 /etc/vservers/.defaults/vdirbase/sr-skimmer03/tmp{,}