Backups with AMANDA: Difference between revisions

From Research
Jump to navigation Jump to search
Line 70: Line 70:
  Estimated time to write 2 * 102400 Mbyte: 15600 sec = 4 h 20 min
  Estimated time to write 2 * 102400 Mbyte: 15600 sec = 4 h 20 min


'''NOTE:''' We've tried enabling hardware-compression within the tape-drive, and also using software-compression on the server.  While both approaches have their strengths and weaknesses, we've found that software-compression on the server consumes too much CPU :-(  This is particularly a problem if amanda's size-estimation is used - an already-slow compression process '''doubles''' in time :-(
'''NOTE:''' We've tried enabling hardware-compression within the tape-drive, and also using software-compression on the server.  While both approaches have their strengths and weaknesses, we've found that software-compression on the server consumes too much CPU :-(  This is particularly a problem if amanda's size-estimation is usedan already-slow compression process '''doubles''' in time :-( Generally, enabling the tape-drive's (auto-)compression and omitting software-compression seems reasonable.
<br>


== Amanda Client Configuration ==
== Amanda Client Configuration ==

Revision as of 14:48, 16 April 2007

Restore Files From Amanda

Background


We have two main file-servers (musashi, and yamato), which each perform their own backups Monday through Friday (typically). Musashi has a SCSI-attached HP LTO-1 Ultrium 230 (Aug. 2003) tape-drive; yamato has a replacement Ultrium 232 (Nov. 2006) tape-drive.
Yamato only backs up it's SCSI-connected RAID drives. Musashi doesn't back up one of it's own SCSI-connected RAID drives (too much non-essential data), but musashi does "reach out" and backup other machines' files. What gets backed up is listed/controlled in the /etc/amanda/Dailyxxx/disklist. Both yamato and musashi have amanda configured as a server and client, while the remote machines captured by musashi have amanda configured as a client-only.

Amanda Server Configuration

First, you've got to install amanda, and it's dependencies; Gentoo shown:

hostname / # emerge -pv amanda

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N    ] sys-block/mtx-1.2.18  164 kB 
[ebuild  N    ] sci-visualization/gnuplot-4.0-r1  USE="emacs gd png readline -X -doc -ggi -pdf -plotutils (-svga) -xemacs" 2,111 kB 
[ebuild  N    ] app-arch/mt-st-0.7-r1  35 kB 
[ebuild  N    ] sys-apps/xinetd-2.3.14  USE="perl tcpd" 295 kB 
[ebuild  N    ] app-arch/dump-0.4.41  USE="readline -ermt -static" 277 kB 
[ebuild  N    ] app-backup/amanda-2.4.5  USE="-berkdb -debug -gdbm -samba -xfs" 1,506 kB 

Total: 6 packages (6 new), Size of downloads: 4,386 kB

NOTE: for on-campus backup-servers, unless you know differently, we will require the xfs USE-flag enabled (because that's what our RAID-arrays use).
Determine the tapetype entry, in /etc/amanda/Dailyxxx/amanda.conf; here are the results from two different drives, on two separate systems for our LTO-1 tapes:

Ultrium 230

define tapetype Ultrium230 {
    comment "HP Ultrium 230, produced by amtapetype"
    length 101376 mbytes
    filemark 0 kbytes
    speed 13871 kps
}

Ultrium 232

amanda@trainwreck ~ $ amtapetype -f /dev/nst0 -e 100G -o
Writing 1024 Mbyte   compresseable data:  37 sec
Writing 1024 Mbyte uncompresseable data:  72 sec
WARNING: Tape drive has hardware compression enabled
Estimated time to write 2 * 102400 Mbyte: 14400 sec = 4 h 0 min
wrote 3080192 32Kb blocks in 94 files in 7025 seconds (short write)
wrote 3096576 32Kb blocks in 189 files in 7511 seconds (short write)
define tapetype unknown-tapetype {
    comment "just produced by tapetype prog (hardware compression on)"
    length 96512 mbytes
    filemark 0 kbytes
    speed 13611 kps
}

Tape Drive Compression


Compression can be turned on or off using the mt tool noted below, then initiating the tapetype check; kill the tapetype check with ctrl-C when you've got enough info (takes quite a few minutes)

trainwreck ~ # mt -f /dev/nst0 defcompression 1   turns internal-hardware compression on
trainwreck ~ # su - amanda
amanda@trainwreck ~ $ amtapetype -f /dev/nst0 -e 100G -o
Writing 1024 Mbyte   compresseable data:  42 sec
Writing 1024 Mbyte uncompresseable data:  72 sec
WARNING: Tape drive has hardware compression enabled
Estimated time to write 2 * 102400 Mbyte: 14400 sec = 4 h 0 min

amanda@trainwreck ~ $ logout
trainwreck ~ # mt -f /dev/nst0 defcompression 0   turns internal-hardware compression off
trainwreck ~ #  su - amanda
amanda@trainwreck ~ $ amtapetype -f /dev/nst0 -e 100G -o
Writing 512 Mbyte   compresseable data:  38 sec
Writing 512 Mbyte uncompresseable data:  39 sec
Estimated time to write 2 * 102400 Mbyte: 15600 sec = 4 h 20 min

NOTE: We've tried enabling hardware-compression within the tape-drive, and also using software-compression on the server. While both approaches have their strengths and weaknesses, we've found that software-compression on the server consumes too much CPU :-( This is particularly a problem if amanda's size-estimation is used: an already-slow compression process doubles in time :-( Generally, enabling the tape-drive's (auto-)compression and omitting software-compression seems reasonable.

Amanda Client Configuration


Unfortunately, with the Gentoo packaging, you get more than you need for just a client (as seen above).
NOTE the currently-unstable amanda-2.5.1_p3-r4 contains the minimal USE flag. This definitely helps cut down on extraneous stuff, when all you need is amanda-client.
Once you've got amanda or amanda-minimal installed, there are a couple of separate steps required for amanda-client to work - configuring the xinetd meta-server, and configuring amanda. Most aggravations seem to stem from permissions, whether it's the server-access through xinetd, or the amanda-permitted-user in .amandahosts.

inferno / # emacs -nw /etc/xinetd.d/amanda

service amanda
{
        socket_type     = dgram
        protocol        = udp
        wait            = yes
        user            = amanda
        group           = amanda
        groups          = yes
        server          = /usr/libexec/amandad
        # You need to ensure this points to your Amanda server!
        # Don't just remove it!
        only_from       = 192.168.0.192 (clearly, this should point to your server)
        disable         = no
}


If this is a brand-new installation of xinetd, don't forget to start it! And, add it to your startup-scripts (Gentoo shown):

inferno / # /etc/init.d/xinetd start
 * Caching service dependencies ...                                         [ ok ]
 * Starting xinetd ...                                                      [ ok ]

And:

inferno / # rc-update add xinetd default
 * xinetd added to runlevel default
inferno / # emacs -nw /etc/amanda/amandahosts

# Machine-name for those hosts which are permitted to back up (fully-qualified
# is best).  An entry in /etc/hosts adds fail-safe against DNS death :-)
# |
# |                                     permitted user-name
# |                                     |

trainwreck.shaw.ca                      amanda      (again, use your server)


That's really all the client needs, unless you're actively using tcpwrappers, in which case /etc/hosts.allow and /etc/hosts.deny may require changes. Similarly, if iptables is running, you'll need to add firewall rules to allow amanda through.
To now bring this client into the backup routine, some entries are needed on the backup-server, in the disklist file (example only):

trainwreck ~ # emacs -nw /etc/amanda/Daily/disklist

inferno         /etc                    nocomp-user     2       le0
inferno         /srv/www                nocomp-user     2       le0



Now, run a check from your backup-server, to ensure things look good:

amanda@trainwreck ~ $ amcheck Daily
Amanda Tape Server Host Check
-----------------------------
Holding disk /holding: 16814560 kB disk space available, that's plenty
NOTE: skipping tape-writable test
Tape Daily07 label ok
NOTE: info dir /usr/local/var/amanda/Daily/curinfo/inferno: does not exist
NOTE: it will be created on the next run.
Server check took 0.245 seconds

Amanda Backup Client Hosts Check
--------------------------------
Client check: 2 hosts checked in 10.069 seconds, 0 problems found

(brought to you by Amanda 2.4.5)


In Case of Difficulty


  • for backing up /var/lib/mysql, you may have to add amanda to the mysql group
  • for other "Permission Denied" issues, you may have to add amanda to the disk group