Virtual Servers: Difference between revisions

From Research
Jump to navigation Jump to search
 
(24 intermediate revisions by 3 users not shown)
Line 1: Line 1:
==Virtual Server Creation, and Deployment==
==Virtual Server Creation, and Deployment==
===Introduction===
===Introduction===
*Gentoo Linux, kernel 2.6.17-vs2.0.2.1-gentoo on host server
*Gentoo Linux, kernel 2.6.17-vs2.0.2.1-gentoo on virtual nodes


There are two methods to create vserver nodes on the vs2.0.2.1-gentoo vserver.  Each method begins differently, but the final setup is the same. The directions contained in this note may appear saturated by the words vserver, vsrvr, vservers, and vs0n. Due to the default vserver directory naming conventions, and the vserver node naming convention (i.e. vs00, vs01, etc.) this name-clutter is unavoidable. If setup does not follow spec, please examine your syntax carefully.
There are two methods to create vserver nodes on the gentoo vserver host.  Each method begins differently, but the final setup is the same. The directions contained in this note may appear saturated by the words vserver, vsrvr, vservers, and vs0n. Due to the default vserver directory naming conventions, and the vserver node naming convention (i.e. vs00, vs01, etc.) this name-clutter is unavoidable. If setup does not follow spec, please examine your syntax carefully.


===Preparation===
===Preparation===
Line 17: Line 15:
Examples (current for mid-December 2007):
Examples (current for mid-December 2007):
  <font color=red>vsrvr_host</font> <font color=blue>~ # </font>'''uname -r'''
  <font color=red>vsrvr_host</font> <font color=blue>~ # </font>'''uname -r'''
  2.6.20-vs2.2.0-gentoo
  2.6.35-vs2.3.0.36.32-gentoo     ''for example''


  <font color=red>vsrvr_host</font> <font color=blue>~ # </font>'''eselect profile show'''
  <font color=red>vsrvr_host</font> <font color=blue>~ # </font>'''eselect profile show'''
  <font color=lime>'''Current make.profile symlink:'''</font>
  <font color=lime>'''Current make.profile symlink:'''</font>
   '''/usr/portage/profiles/default-linux/amd64/2007.0'''    ''of course, your architecture may differ''
   '''default/linux/amd64/10.0'''    ''of course, your architecture may differ''
*The Guests will live under '''/vservers'''.  So, make sure you either:
*The Guests will live under '''/vservers'''.  So, make sure you either:
**have enough room under your '''/''' root-filesystem for your guest(s)
**have enough room under your '''/''' root-filesystem for your guest(s)
Line 27: Line 25:


====Guest Preparation ('''NOT''' from a template)====
====Guest Preparation ('''NOT''' from a template)====
We will use our familiar Gentoo init-style, so grab the Stage 4 bzipped tarball for '''your''' architecture from [http://people.linux-vserver.org/~hollow/stages/ here].  In this example, the Intel Core2Quad CPU used the amd64 stage (and profile), simply for the 64-bit goodness.
We will use our familiar Gentoo init-style, so grab the Stage 3 bzipped tarball for '''your''' architecture from your favourite Gentoo mirror.  In this example, the Intel Core2Quad CPU uses the amd64 stage (and profile):
  <font color=red>vsrvr_host</font> <font color=blue>vservers </font> <font color=blue># </font>'''wget http://people.linux-vserver.org/~hollow/stages/stage4-amd64-20070905.tar.bz2'''
  <font color=red>vsrvr_host</font> <font color=blue>vservers </font> <font color=blue># </font>'''wget http://distfiles.gentoo.org/releases/amd64/autobuilds/current-stage3/stage3-amd64-20130110.tar.bz2'''
--08:19:32--  http://people.linux-vserver.org/~hollow/stages/stage4-amd64-20070905.tar.bz2
            => `stage4-amd64-20070905.tar.bz2'
Resolving people.linux-vserver.org... 78.47.240.170
Connecting to people.linux-vserver.org|78.47.240.170|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 176,132,669 (168M) [application/x-tar]
7% [========>


Once the gzipped tarball is downloaded, you follow the [http://www.gentoo.org/proj/en/vps/vserver-howto.xml VServer HowTo] instructions to extract and set up your guest.  "Magic" words in the example below are:
Once the gzipped tarball is downloaded, you follow the [http://www.gentoo.org/proj/en/vps/vserver-howto.xml VServer HowTo] instructions to extract and set up your guest.  "Magic" words in the example below are:
Line 65: Line 55:
  >>> Providing dummy net dependency ...  
  >>> Providing dummy net dependency ...  
  <font color=red>vsrvr_host</font> <font color=blue>vservers </font> <font color=blue>#</font>
  <font color=red>vsrvr_host</font> <font color=blue>vservers </font> <font color=blue>#</font>
Another Installation-Command Example
<font color=red>vsrvr_host</font> <font color=blue>vservers </font> <font color=blue># </font>'''vserver mediatomb build --context 40 --hostname mediatomb --interface bond0:192.168.0.140/24 --initstyle gentoo -m template -- -d gentoo -t /vservers/stage4-amd64-20070905.tar.bz2'''
Yet another Installation-Command Example, this time a Debian guest on our Gentoo host... we definitely need to specify the architecture, because debootstrap will obtain the host's arch as GenuineIntel (or whatever CPU), which won't be recognized properly during installation :-(
<font color=red>vsrvr_host</font> <font color=blue>vservers </font> <font color=blue># </font>'''vserver eprints build -m debootstrap --context 23 --hostname eprints.iat.sfu.ca --interface bond0:209.87.56.23/24 --initstyle sysv -- -d squeeze -m  http://ftp.de.debian.org/debian -- --arch amd64'''
Following the instructions above, we chroot into the new vserver guest and prepare a few things:
<font color=red>vsrvr_host</font> <font color=blue>root #</font> '''cd /vservers/new_vserver'''
<font color=red>vsrvr_host</font> <font color=blue>vservers #</font> '''mount -t proc proc /vservers/new_vserver/proc'''
<font color=red>vsrvr_host</font> <font color=blue>vservers #</font> '''chroot /vservers/new_vserver /bin/bash'''
<font color=red>new_vsrvr_guest</font> <font color=blue>/ #</font> '''env-update'''
<font color=red>new_vsrvr_guest</font> <font color=blue>/ #</font> '''source /etc/profile'''
<font color=red>new_vsrvr_guest</font> <font color=blue>/ #</font> '''export PS1="(chroot) $PS1"'''
<font color=red>new_vsrvr_guest</font> <font color=blue>/ #</font> '''ln -s /etc/init.d/net.lo /etc/init.d/net.bond0'''    ''or net.eth0 or whatever.  None of these seem to exist at this time''
<font color=red>new_vsrvr_guest</font> <font color=blue>/ #</font> '''emerge --sync'''
<font color=red>new_vsrvr_guest</font> <font color=blue>/ #</font> '''ln -sf /usr/share/zoneinfo/America/Vancouver /etc/localtime'''
<font color=red>new_vsrvr_guest</font> <font color=blue>/ #</font> '''emerge sys-process/vixie-cron'''
<font color=red>new_vsrvr_guest</font> <font color=blue>/ #</font> '''rc-update add vixie-cron default'''
<font color=red>new_vsrvr_guest</font> <font color=blue>/ #</font> '''exit'''
<font color=red>vsrvr_host</font> <font color=blue>vservers #</font> '''umount /vservers/new_vserver/proc'''
Hit the ignition and see if it runs ;-)
<font color=red>vsrvr_host</font> <font color=blue>vservers #</font> '''vserver new_vserver_guest start'''
    <font color=lime>OpenRC</font> <font color=cyan>0.11.8</font> is starting up <font color=blue>Gentoo Linux (x86_64) [VSERVER]</font>
<font color=lime>*</font> /proc is already mounted
<font color=lime>*</font> /run/openrc: creating directory
<font color=lime>*</font> /run/lock: creating directory
<font color=lime>*</font> /run/lock: correcting owner
<font color=lime>*</font> Caching service dependencies ...                                                                                        [ ok ]
<font color=lime>*</font> Creating user login records ...                                                                                        [ ok ]
<font color=lime>*</font> Cleaning /var/run ...                                                                                                  [ ok ]
<font color=lime>*</font> Wiping /tmp directory ...                                                                                              [ ok ]
<font color=lime>*</font> Setting hostname to owncloud.iat.sfu.ca ...                                                                            [ ok ]
<font color=lime>*</font> Updating /etc/mtab ...                                                                                                  [ ok ]
<font color=lime>*</font> setting up tmpfiles.d entries ...                                                                                      [ ok ]
<font color=lime>*</font> Starting vixie-cron ...                                                                                                [ ok ]
<font color=lime>*</font> Starting local                                                                                                                                              [ ok ]
syslog-ng won't run?  That's because there is no access to klogd from inside a vserver-guest :-O  So, stop syslog-ng from trying to read /proc/kmsg by commenting out the line shown below:
source src {
    unix-stream("/dev/log" max-connections(256));
    internal();
'''#    file("/proc/kmsg");'''
};
May as well also filter out the annoying/repetitive cron-timestamps:
destination messages { file("/var/log/messages"); };
# We will take out the annoying cron notifications
'''filter nocron { not facility(cron); };'''
log { source(src); '''filter(nocron);''' destination(messages); };


Make this Guest start each time you reboot the Host:
Make this Guest start each time you reboot the Host:
Line 174: Line 219:


  <font color=red>vsrvr</font> <font color=blue>/ #</font> '''rm -r -I /etc/vservers/vs0n'''
  <font color=red>vsrvr</font> <font color=blue>/ #</font> '''rm -r -I /etc/vservers/vs0n'''
===Notes===
When working with virtual servers, make sure you enter the virtual server before you do any work:
<font color=red>vsrvr</font> <font color=blue>/ #</font> '''vservers $servername enter'''
When you are done, just use the 'exit' command to return to the host.


=='''Virtual Server Stock Software List'''==
=='''Virtual Server Stock Software List'''==
Line 179: Line 231:
Here is a list of the significant software installed on our vserver guests:
Here is a list of the significant software installed on our vserver guests:


Nagios monitoring: http, load, ntp, snmp, ssh, processes, uptime, users
nrpe, nagios-plugins    ''to support Nagios monitoring: http, load, ntp, snmp, ssh, processes, uptime, users''


Apache
Apache
Line 187: Line 239:
PHP
PHP


phpmyadmin
<s>phpmyadmin</s>


awstats
awstats
Line 193: Line 245:
amanda
amanda


vsftp
<s>vsftp</s>


logwatch
logwatch


tripwire
tripwire
=='''Miscellaneous Virtual Server Commands'''==
vnamespace -e  5654 mount -t tmpfs -o remount,size=256m,mode=1777  /etc/vservers/.defaults/vdirbase/sr-skimmer03/tmp{,}

Latest revision as of 17:14, 25 January 2013

Virtual Server Creation, and Deployment

Introduction

There are two methods to create vserver nodes on the gentoo vserver host. Each method begins differently, but the final setup is the same. The directions contained in this note may appear saturated by the words vserver, vsrvr, vservers, and vs0n. Due to the default vserver directory naming conventions, and the vserver node naming convention (i.e. vs00, vs01, etc.) this name-clutter is unavoidable. If setup does not follow spec, please examine your syntax carefully.

Preparation

There are two distinct steps to manually-installing a vserver:

  • Host system. We are using Gentoo Linux for the Host; the purpose of the Host is to run the Guest Operating Systems
  • Guest system(s). These do the actual work, such as web-serving, or perhaps Samba, or maybe OpenLDAP authentication.

Sometimes, there are grey areas regarding what services should run on the Host, or the Guest.

Host Preparation

  • The host environment has only one significant change over a normal host environment - it uses the vserver kernel. The profile must not be a vserver profile on the host.

Examples (current for mid-December 2007):

vsrvr_host ~ # uname -r
2.6.35-vs2.3.0.36.32-gentoo     for example
vsrvr_host ~ # eselect profile show
Current make.profile symlink:
  default/linux/amd64/10.0     of course, your architecture may differ
  • The Guests will live under /vservers. So, make sure you either:
    • have enough room under your / root-filesystem for your guest(s)
    • prepare and mount another device (drive, volume) under /vservers. Check that this "sticks" properly after a reboot (no /etc/fstab errors)

Guest Preparation (NOT from a template)

We will use our familiar Gentoo init-style, so grab the Stage 3 bzipped tarball for your architecture from your favourite Gentoo mirror. In this example, the Intel Core2Quad CPU uses the amd64 stage (and profile):

vsrvr_host vservers  # wget http://distfiles.gentoo.org/releases/amd64/autobuilds/current-stage3/stage3-amd64-20130110.tar.bz2

Once the gzipped tarball is downloaded, you follow the VServer HowTo instructions to extract and set up your guest. "Magic" words in the example below are:

  • myguest -> basilisk this is the name I wanted for my (file-)server, so I have applied it in two places:
    • in the first instance, this means that the base-directory for my Guest will be /vservers/basilisk. Having the Host's container-directory share the same name as the Guest hostname made sense to me.
    • the second instance will set the hostname in various (usual) places inside the Guest (/etc/conf.d/hostname for one)
  • context 30 This must be unique, and not collide with any other Guest context. That's it; a random, unique number. You could use shoe-size, or IQ, but I chose to first figure out the IP-address I wanted for this (file-)server, and then use the last portion for my context number. The next Guest may use context 40 and an IP of 192.168.0.40 with my scheme; feel free to do whatever you want, but make the context values unique!!

Here is an example Guest installation, for reference:

vsrvr_host vservers  # vserver basilisk build --context 30 --hostname basilisk --interface eth0:192.168.0.30/24 --initstyle gentoo -m template -- -d gentoo -t /vservers/stage4-amd64-20070905.tar.bz2
>>> Adding shared /usr/portage to fstab ... 
>>> Checking init-style ... gentoo
>>> Unpacking template ... ok
>>> Installing special init-style magic ... 
!!!
!!! You have to install a service (e.g. syslog/cron) and add it to the
!!! default runlevel before you start the guest the first time!
!!! Otherwise the guest will die as soon as it has finished booting.
!!!
!!! Consult the Gentoo Handbook on how to chroot and install
!!! packages into the guest environment.
!!!
>>> Fixing default runlevel scripts ... 
>>> Setting hostname ... 
>>> Fixing syslog-ng.conf ... 
>>> Fixing inittab ... 
>>> Fixing fstab ... 
>>> Providing dummy net dependency ... 
vsrvr_host vservers  #

Another Installation-Command Example

vsrvr_host vservers  # vserver mediatomb build --context 40 --hostname mediatomb --interface bond0:192.168.0.140/24 --initstyle gentoo -m template -- -d gentoo -t /vservers/stage4-amd64-20070905.tar.bz2

Yet another Installation-Command Example, this time a Debian guest on our Gentoo host... we definitely need to specify the architecture, because debootstrap will obtain the host's arch as GenuineIntel (or whatever CPU), which won't be recognized properly during installation :-(

vsrvr_host vservers  # vserver eprints build -m debootstrap --context 23 --hostname eprints.iat.sfu.ca --interface bond0:209.87.56.23/24 --initstyle sysv -- -d squeeze -m  http://ftp.de.debian.org/debian -- --arch amd64

Following the instructions above, we chroot into the new vserver guest and prepare a few things:

vsrvr_host root # cd /vservers/new_vserver
vsrvr_host vservers # mount -t proc proc /vservers/new_vserver/proc
vsrvr_host vservers # chroot /vservers/new_vserver /bin/bash
new_vsrvr_guest / # env-update
new_vsrvr_guest / # source /etc/profile
new_vsrvr_guest / # export PS1="(chroot) $PS1"
new_vsrvr_guest / # ln -s /etc/init.d/net.lo /etc/init.d/net.bond0     or net.eth0 or whatever.  None of these seem to exist at this time
new_vsrvr_guest / # emerge --sync
new_vsrvr_guest / # ln -sf /usr/share/zoneinfo/America/Vancouver /etc/localtime
new_vsrvr_guest / # emerge sys-process/vixie-cron
new_vsrvr_guest / # rc-update add vixie-cron default
new_vsrvr_guest / # exit
vsrvr_host vservers # umount /vservers/new_vserver/proc

Hit the ignition and see if it runs ;-)

vsrvr_host vservers # vserver new_vserver_guest start
    OpenRC 0.11.8 is starting up Gentoo Linux (x86_64) [VSERVER]

* /proc is already mounted
* /run/openrc: creating directory
* /run/lock: creating directory
* /run/lock: correcting owner
* Caching service dependencies ...                                                                                        [ ok ]
* Creating user login records ...                                                                                         [ ok ]
* Cleaning /var/run ...                                                                                                   [ ok ]
* Wiping /tmp directory ...                                                                                               [ ok ]
* Setting hostname to owncloud.iat.sfu.ca ...                                                                             [ ok ]
* Updating /etc/mtab ...                                                                                                  [ ok ]
* setting up tmpfiles.d entries ...                                                                                       [ ok ]
* Starting vixie-cron ...                                                                                                 [ ok ]
* Starting local                                                                                                                                              [ ok ]

syslog-ng won't run? That's because there is no access to klogd from inside a vserver-guest :-O So, stop syslog-ng from trying to read /proc/kmsg by commenting out the line shown below:

source src {
    unix-stream("/dev/log" max-connections(256));
    internal();
#    file("/proc/kmsg");
};

May as well also filter out the annoying/repetitive cron-timestamps:

destination messages { file("/var/log/messages"); };
# We will take out the annoying cron notifications
filter nocron { not facility(cron); };

log { source(src); filter(nocron); destination(messages); };

Make this Guest start each time you reboot the Host:

vsrvr_host vservers  # echo "default" > /etc/vservers/basilisk/apps/init/mark     of course, you have to use your own hostname, not my choice of basilisk!
vsrvr_host vservers  # rc-update add vservers.default default

Create a Vserver Template

vsrvr / # vserver vs0n enter
vs0n / # shutdown -h now
vsrvr / # cd /vservers/vs0n
vsrvr vs0n # tar cvf vs.template*.tar bin boot etc fastboot home lib mnt root sbin sys tmp usr var
  • DO NOT INCLUDE /DEV OR /PROC
vsrvr vs0n # cp /vservers/vs0n/vs.template*.tar /vserver.template/
  • Be careful to examine the /vserver.template/ directory for pre-existing vs.template*.tar files before naming any new tar files. vs.template.tar is the stock template, so name new templates vs.template.00.tar, etc. Using non-stock templates will have an impact on the *.tar name used in point 4 in the next section of this note.
  • GOTO "Use the Vserver Stock Template"


Use the Vserver Stock Template

vsrvr / # vserver vs0n build -m skeleton --hostname vs0n --context 563n --interface vs0n=eth0:209.87.56.3n/24
  • Make sure to check /vservers directory for pre-existing vs0n nodes matching the desired vnode name
  • Make sure to correct/match every vs0n reference in the vnode creation line; there are 3 references.
  • Make sure to correct/match the IP statement eth0:209.87.56.3n/24 with the IP stated in the context.
  • Make sure the context matches the values of the last two octets of the target vnode IP
vsrvr / # cd /vservers/vs0n
vsrvr vs0n # tar -xvf /vserver.template/vs.template*.tar

Post Vnode Creation Setup

  • Several directories will need to be customized, post vnode creation, to make the vnode unique.
  • Changes must be made in the /etc/vservers/vs0n and the /vservers/vs0n before the new vnode will be ready to start.

Changes to /etc/vservers/vs0n

  • To allow vnode to boot, we need to copy over init instructions.
vsrvr / # cp /vserver.template/init/* /etc/vservers/vs0n/apps/init/
  • To allow for proper service initialization, we need to add eth:lo and renumber interface 0 and 1.
vsrvr / # mv /etc/vservers/vs0n/interfaces/0 /etc/vservers/vs0n/interfaces/1
vsrvr / # cp -r /vserver.template/interfaces/0 /etc/vservers/vs0n/interfaces/
vsrvr / # cp -r /vserver.template/fstab /etc/vservers/vs0n
cp: overwrite `/etc/vservers/vs0n/fstab'? yes


Changes to /vservers/vs0n

  • To allow for proper service initialization, we need to change hostname and IP addresses in the following files.
vsrvr / # vi /vservers/vs0n/etc/ssh/sshd_config
vsrvr / # vi /vservers/vs0n/etc/conf.d/hostname
vsrvr / # vi /vservers/vs0n/etc/hosts
vsrvr / # vi /vservers/vs0n/etc/conf.d/net
vsrvr / # vi /vservers/vs0n/etc/conf.d/domainname
vsrvr / # vi /vservers/vs0n/etc/mysql/my.cnf
vsrvr / # vi /vservers/vs0n/etc/apache2/vhosts.d/00_default_vhost.conf
vsrvr / # vi /vservers/vs0n/etc/nullmailer/me
vsrvr / # vi /vservers/vs0n/etc/apache2/vhosts.d/00_default_ssl_vhost.conf
vsrvr / # vi /vservers/vs0n/usr/share/logwatch/custom_header
vsrvr / # vi /vservers/vs0n/etc/tripwire/twpol.txt
vsrvr / # mv /vservers/vs0n/var/lib/tripwire/vs0n.twd /vservers/vs0n/var/lib/tripwire/vs0n.twd
vsrvr / # mv /vservers/vs0n/etc/tripwire/vs0n-local.key /vservers/vs0n/etc/tripwire/vs0n-local.key
vsrvr / # vi  /vservers/vs0n/etc/awstats/awstats.awstats.conf

Start your Vserver

vsrvr / # vserver vs0n start
vsrvr / # vserver vs0n enter

Password

  • Don't forget to set a new root password.


Post Creation tripwire Configuration

vsrvr / # twadmin --create-polfile -S /etc/tripwire/site.key /etc/tripwire/twpol.txt
vsrvr / # tripwire --init
vsrvr / # tripwire --check

Delete a Vserver

vsrvr / # rm -r -I /vservers/vs0n
vsrvr / # rm -r -I /etc/vservers/vs0n

Notes

When working with virtual servers, make sure you enter the virtual server before you do any work:

vsrvr / # vservers $servername enter

When you are done, just use the 'exit' command to return to the host.

Virtual Server Stock Software List

Here is a list of the significant software installed on our vserver guests:

nrpe, nagios-plugins to support Nagios monitoring: http, load, ntp, snmp, ssh, processes, uptime, users

Apache

Mysql

PHP

phpmyadmin

awstats

amanda

vsftp

logwatch

tripwire

Miscellaneous Virtual Server Commands

vnamespace -e 5654 mount -t tmpfs -o remount,size=256m,mode=1777 /etc/vservers/.defaults/vdirbase/sr-skimmer03/tmp{,}