RAID Documentation - Revision history

Gordp: /* RAID Maintenance */

2015-06-16T23:13:12Z

RAID Maintenance

← Older revision		Revision as of 23:13, 16 June 2015
Line 109:		Line 109:


	Update the firmware using the command-line - first download (and unzip if necessary) the firmware file:		Update the firmware using the command-line - first download (and unzip if necessary) the firmware file. This goes pretty quickly (less than a minute) and will '''not''' take effect until after reboot:
	<font color=red>spitfire</font> <font color=blue>~ #</font> '''tw_cli /c1 update fw=~/prom0006.img'''		<font color=red>spitfire</font> <font color=blue>~ #</font> '''tw_cli /c1 update fw=~/prom0006.img'''

Gordp: /* RAID Maintenance */

2015-06-16T23:08:41Z

RAID Maintenance

← Older revision		Revision as of 23:08, 16 June 2015
Line 103:		Line 103:
	* '''tw_cli''' doesn't need to be typed on a line by itself. It can be combined with parameters such as: '''tw_cli /c1 show'''		* '''tw_cli''' doesn't need to be typed on a line by itself. It can be combined with parameters such as: '''tw_cli /c1 show'''
	* The '''/c1''' argument specifies the RAID controller. On some servers, such as spitfire, c1 is the correct controller identifier. But other servers might use c0, or c2, etc. So, when using the '''/c1 show''' command, don't be surprised if you get an error, saying that the controller isn't found. You can go ahead and try a different identifier, such as c0.		* The '''/c1''' argument specifies the RAID controller. On some servers, such as spitfire, c1 is the correct controller identifier. But other servers might use c0, or c2, etc. So, when using the '''/c1 show''' command, don't be surprised if you get an error, saying that the controller isn't found. You can go ahead and try a different identifier, such as c0.


			Change stripe size on an existing RAID-5 "live" system:
			<font color=red>spitfire</font> <font color=blue>~ #</font> '''tw_cli /c1/u0 migrate type=raid5 stripe=256'''


			Update the firmware using the command-line - first download (and unzip if necessary) the firmware file:
			<font color=red>spitfire</font> <font color=blue>~ #</font> '''tw_cli /c1 update fw=~/prom0006.img'''

Lars: /* RAID Maintenance */

2013-07-04T17:57:20Z

RAID Maintenance

← Older revision		Revision as of 17:57, 4 July 2013
Line 90:		Line 90:

	That would produce a list of statuses for each drive in the array and clearly identify the bad drive. Once you know the name of the bad drive (eg: P15), you remove it (P15) from the array, and replace it with a new drive. It will automatically commence a RAID-rebuild. You can use '''/c1 show''' later on to check on the array status.		That would produce a list of statuses for each drive in the array and clearly identify the bad drive. Once you know the name of the bad drive (eg: P15), you remove it (P15) from the array, and replace it with a new drive. It will automatically commence a RAID-rebuild. You can use '''/c1 show''' later on to check on the array status.


	Lately, however, the replaced drive is showing up as a new Unit, '''u?'''. Not helpful :-( but here's what to do:		Lately, however, the replaced drive is showing up as a new Unit, '''u?'''. Not helpful :-( but here's what to do:
Line 97:		Line 98:
	//spitfire> '''/c1/u2 del''' ''do not use '''remove'''; '''del''' will keep the drive, but un-assign it''		//spitfire> '''/c1/u2 del''' ''do not use '''remove'''; '''del''' will keep the drive, but un-assign it''
	//spitfire> '''maint rebuild c1 u0 p15''' ''example, to add replaced drive p15 into Unit u0 on Controller c1''		//spitfire> '''maint rebuild c1 u0 p15''' ''example, to add replaced drive p15 into Unit u0 on Controller c1''


			Some tips:
			* '''tw_cli''' doesn't need to be typed on a line by itself. It can be combined with parameters such as: '''tw_cli /c1 show'''
			* The '''/c1''' argument specifies the RAID controller. On some servers, such as spitfire, c1 is the correct controller identifier. But other servers might use c0, or c2, etc. So, when using the '''/c1 show''' command, don't be surprised if you get an error, saying that the controller isn't found. You can go ahead and try a different identifier, such as c0.

Lars: /* RAID Maintenance */

2013-07-02T14:59:19Z

RAID Maintenance

← Older revision		Revision as of 14:59, 2 July 2013
Line 84:		Line 84:

	== RAID Maintenance ==		== RAID Maintenance ==
	We have drive-failures periodically - reported both through Nagios and via daily logwatch-emails. These must be replaced promptly, to avoid data-loss! The Chenbro chassis supports drive-hot-swap, and it ~~used to be that~~ the ~~3Ware controller would then~~ automatically commence a RAID-rebuild. Lately, however, the replaced drive is showing up as a new Unit, '''u?'''. Not helpful :-( but here's what to do:		We have drive-failures periodically - reported both through Nagios and via daily logwatch-emails. These must be replaced promptly, to avoid data-loss! The Chenbro chassis supports drive-hot-swap. Typically, to replace a bad hard drive, you would log onto the afflicted server and type the following:

			<font color=red>spitfire</font> <font color=blue>~ #</font> '''tw_cli'''
			//spitfire> '''/c1 show'''

			That would produce a list of statuses for each drive in the array and clearly identify the bad drive. Once you know the name of the bad drive (eg: P15), you remove it (P15) from the array, and replace it with a new drive. It will automatically commence a RAID-rebuild. You can use '''/c1 show''' later on to check on the array status.

			Lately, however, the replaced drive is showing up as a new Unit, '''u?'''. Not helpful :-( but here's what to do:

	<font color=red>spitfire</font> <font color=blue>~ #</font> '''tw_cli'''		<font color=red>spitfire</font> <font color=blue>~ #</font> '''tw_cli'''

Gordp: /* Raid Usage */

2012-06-20T17:56:43Z

Raid Usage

← Older revision		Revision as of 17:56, 20 June 2012
Line 1:		Line 1:
	== Raid Usage ==		== Raid Usage ==
	In the Early Days <c> we used Arena EX3 external SCSI-attached RAID arrays. These external units featured dual-redundant power-supplies, and held six IDE drives. Later, we found we could use SATA drives, with a low-profile SATA-to-PATA (IDE) interface, and a little bit of connector trimming. This boosted the capacity for these external SCSI devices, but only to a point - the limitations of the internal ~~32-bit~~ controller seemed to cap the capacity at 2TB.		In the Early Days <c> we used Arena EX3 external SCSI-attached RAID arrays. These external units featured dual-redundant power-supplies, and held six IDE drives. Later, we found we could use SATA drives, with a low-profile SATA-to-PATA (IDE) interface, and a little bit of connector trimming. This boosted the capacity for these external SCSI devices, but only to a point - the limitations of the internal controller seemed to cap the capacity at 2TB.

	So, we phased out these Arena EX3 RAID arrays, in favour of big Chenbro rack-mounted chassis, each capable of holding 16 drives. A 3Ware controller with 16 ports addressed these drives, which are hot-pluggable in the Chenbro chassis + drive-bay-backplane.		So, we phased out these Arena EX3 RAID arrays, in favour of big Chenbro rack-mounted chassis, each capable of holding 16 drives. A 3Ware controller with 16 ports addressed these drives, which are hot-pluggable in the Chenbro chassis + drive-bay-backplane.

Gordp: /* Musashi */

2012-06-20T17:48:30Z

Musashi

← Older revision		Revision as of 17:48, 20 June 2012
Line 75:		Line 75:
	=== Musashi ===		=== Musashi ===
	Still using the older external Arena EX3 RAID-array.		Still using the older external Arena EX3 RAID-array.
	:*shows up as a single-partition '''/dev/sdc1''' mounted on '''/mnt/raid4~~/mirror~~''', type '''XFS'''		:*shows up as a single-partition '''/dev/sdc1''' mounted on '''/mnt/raid4''', type '''XFS'''
	:*bind-mounted to '''/export/mirror''' where it is NFS-exported		:*'''/mnt/raid4/mirror''' is bind-mounted to '''/export/mirror''' where it is NFS-exported
	:*holds our Gentoo GNU/Linux Mirror, for local Portage/distfile sync		:*holds our Gentoo GNU/Linux Mirror, for local Portage/distfile sync
	<br>		<br>

	=== Hercules ===		=== Hercules ===
	:*to be added		:*to be added

Gordp: /* Musashi */

2012-06-20T17:46:48Z

Musashi

← Older revision		Revision as of 17:46, 20 June 2012
Line 78:		Line 78:
	:*bind-mounted to '''/export/mirror''' where it is NFS-exported		:*bind-mounted to '''/export/mirror''' where it is NFS-exported
	:*holds our Gentoo GNU/Linux Mirror, for local Portage/distfile sync		:*holds our Gentoo GNU/Linux Mirror, for local Portage/distfile sync
			<br>
			=== Hercules ===
			:*to be added

	== RAID Maintenance ==		== RAID Maintenance ==

Gordp: /* Musashi */

2012-06-20T17:46:02Z

Musashi

← Older revision		Revision as of 17:46, 20 June 2012
Line 74:		Line 74:

	=== Musashi ===		=== Musashi ===
	:'''/mnt/raid4'''		Still using the older external Arena EX3 RAID-array.
	:* Gentoo GNU/Linux Mirror		:*shows up as a single-partition '''/dev/sdc1''' mounted on '''/mnt/raid4/mirror''', type '''XFS'''
			:*bind-mounted to '''/export/mirror''' where it is NFS-exported
			:*holds our Gentoo GNU/Linux Mirror, for local Portage/distfile sync

	== RAID Maintenance ==		== RAID Maintenance ==

Gordp at 17:39, 20 June 2012

2012-06-20T17:39:19Z

← Older revision		Revision as of 17:39, 20 June 2012
Line 84:		Line 84:
	//spitfire> '''/c1/u2 del''' ''do not use '''remove'''; '''del''' will keep the drive, but un-assign it''		//spitfire> '''/c1/u2 del''' ''do not use '''remove'''; '''del''' will keep the drive, but un-assign it''
	//spitfire> '''maint rebuild c1 u0 p15''' ''example, to add replaced drive p15 into Unit u0 on Controller c1''		//spitfire> '''maint rebuild c1 u0 p15''' ''example, to add replaced drive p15 into Unit u0 on Controller c1''

	~~== Redistributing users across the Raids ==~~

	~~To move a user from one raid to another:~~
	~~# Make sure the user is not logged in anywhere~~
	~~# Move the files from one raid to another in the appropriate directory for example~~
	~~#* mv /mnt/raid0/home/m/mdeepwel /mnt/raid1/home/m~~
	~~# Update LDAP with the new root location~~
	~~#*In ou=AutoFS,ou=home.users,cn=username,automountInformation, and update to new location. (using phpldapadmin is fine)~~
	~~# Restart '''autofs''' on any computer or service the user could be using, otherwise they will be able to login, but won't have a home directory.~~
	~~#* If they use the cluster, you must restart autofs on each node, or restart the whole cluster.~~
	~~#* Restart teleport, so they can use ssh/ftp to get their files from off site.~~
	~~# Update amanda to make sure the new user directory is backed up.~~

Gordp: /* RAID Maintenance */

2012-06-20T17:37:48Z

RAID Maintenance

← Older revision		Revision as of 17:37, 20 June 2012
Line 81:		Line 81:

	<font color=red>spitfire</font> <font color=blue>~ #</font> '''tw_cli'''		<font color=red>spitfire</font> <font color=blue>~ #</font> '''tw_cli'''
	~~'''~~//spitfire> /c1 rescan''' ''will find the replaced drive, assign it to a new Unit u2''		//spitfire> '''/c1 rescan''' ''will find the replaced drive, assign it to a new Unit u2''
	~~'''~~//spitfire> /c1/u2 del''' ''do not use '''remove'''; '''del''' will keep the drive, but un-assign it''		//spitfire> '''/c1/u2 del''' ''do not use '''remove'''; '''del''' will keep the drive, but un-assign it''
	~~'''~~//spitfire> maint rebuild c1 u0 p15''' ''example, to add replaced drive p15 into Unit u0 on Controller c1''		//spitfire> '''maint rebuild c1 u0 p15''' ''example, to add replaced drive p15 into Unit u0 on Controller c1''

	== Redistributing users across the Raids ==		== Redistributing users across the Raids ==