HOWTO Setup S.M.A.R.T. hard-drive Monitoring
Typically, we use a 3Ware controller, with anywhere from 2 - 16 individual hard-drives attached. Although we use 3Ware's tw_cli tool in both a daily cron-job and also in a Nagios monitor, it's also a good idea to get a daily logwatch-email stanza with drive-statistics and alerts. Belts and suspenders, you know.
In addition to (passive) monitoring, we actively invoke Long Self-Tests on each drive. These tests are scheduled to try to hit low-usage times, and also avoid our tape-backup times.
Install the (Gentoo) package:
hostname # emerge -av smartmontools
Start the smartd monitoring daemon automatically in the default runlevel:
hostname # rc-update add smartd default
Representative configuration file:
hostname # emacs -nw /etc/smartd.conf # This file: /etc/smartd.conf # created April 14, 2010, Gordon Pritchard # Monitor 16 SATA disks connected to a 3ware 9650 controller which # uses the (built-into-kernel) 3w-9xxx driver. # The first two drives are WD 150GB Raptor, configured as RAID-1 # long selftest takes 72min # The remaining 14 drives are Seagate 500GB Barracuda, configured as RAID-5 # long selftest takes 120min /dev/twa0 \ -s L/../../0/01 \ # Long Self-tests Sundays between 1-2:30am -d 3ware,0 \ # First physical drive, behind 3Ware controller -I 194 \ # Ignore temperature -I 9 \ # Ignore power-on hours -a # Report on: # SMART health status (-H) # usage failures (-f) # changes in Prefailure, Usage (-t) # increases in error (-l error) # increases in self-test errors (-l selftest) /dev/twa0 \ -s L/../../0/03 \ # Long Self-tests Sunday between 3-4:30am -d 3ware,1 \ # Second physical drive, behind 3Ware controller -I 194 \ # Ignore temperature -I 9 \ # Ignore power-on hours -a # Report on: # SMART health status (-H) # usage failures (-f) # changes in Prefailure, Usage (-t) # increases in error (-l error) # increases in self-test errors (-l selftest) /dev/twa0 \ -s L/../../0/22 \ # Long Self-tests Sundays between 10pm-midnight -d 3ware,2 \ # Third physical drive, behind 3Ware controller -I 194 \ # Ignore temperature -I 190 \ # Ignore Air Temperature -I 9 \ # Ignore power-on hours -a # Report on: # SMART health status (-H) # usage failures (-f) # changes in Prefailure, Usage (-t) # increases in error (-l error) # increases in self-test errors (-l selftest) /dev/twa0 \ -s L/../../1/01 \ # Long Self-tests Mondays between 1-3am -d 3ware,3 \ # Fourth physical drive, behind 3Ware controller -I 194 \ # Ignore temperature -I 190 \ # Ignore Air Temperature -I 9 \ # Ignore power-on hours -a # Report on: # SMART health status (-H) # usage failures (-f) # changes in Prefailure, Usage (-t) # increases in error (-l error) # increases in self-test errors (-l selftest) /dev/twa0 \ -s L/../../1/22 \ # Long Self-tests Mondays between 10pm-midnight -d 3ware,4 \ # Fifth physical drive, behind 3Ware controller -I 194 \ # Ignore temperature -I 190 \ # Ignore Air Temperature -I 9 \ # Ignore power-on hours -a # Report on: # SMART health status (-H) # usage failures (-f) # changes in Prefailure, Usage (-t) # increases in error (-l error) # increases in self-test errors (-l selftest) /dev/twa0 \ -s L/../../2/01 \ # Long Self-tests Tuesdays between 1-3am -d 3ware,5 \ # Sixth physical drive, behind 3Ware controller -I 194 \ # Ignore temperature -I 190 \ # Ignore Air Temperature -I 9 \ # Ignore power-on hours -a # Report on: # SMART health status (-H) # usage failures (-f) # changes in Prefailure, Usage (-t) # increases in error (-l error) # increases in self-test errors (-l selftest) /dev/twa0 \ -s L/../../2/22 \ # Long Self-tests Tuesdays between 10pm-midnight -d 3ware,6 \ # Seventh physical drive, behind 3Ware controller -I 194 \ # Ignore temperature -I 190 \ # Ignore Air Temperature -I 9 \ # Ignore power-on hours -a # Report on: # SMART health status (-H) # usage failures (-f) # changes in Prefailure, Usage (-t) # increases in error (-l error) # increases in self-test errors (-l selftest) /dev/twa0 \ -s L/../../3/01 \ # Long Self-tests Wednesdays between 1-3am -d 3ware,7 \ # Eighth physical drive, behind 3Ware controller -I 194 \ # Ignore temperature -I 190 \ # Ignore Air Temperature -I 9 \ # Ignore power-on hours -a # Report on: # SMART health status # increases in error (-l error) # increases in self-test errors (-l selftest) /dev/twa0 \ -s L/../../3/22 \ # Long Self-tests Wednesdays between 10pm-midnight -d 3ware,8 \ # Ninth physical drive, behind 3Ware controller -I 194 \ # Ignore temperature -I 190 \ # Ignore Air Temperature -I 9 \ # Ignore power-on hours -a # Report on: # SMART health status (-H) # usage failures (-f) # changes in Prefailure, Usage (-t) # increases in error (-l error) # increases in self-test errors (-l selftest) /dev/twa0 \ -s L/../../4/01 \ # Long Self-tests Thursdays between 1-3am -d 3ware,9 \ # Tenth physical drive, behind 3Ware controller -I 194 \ # Ignore temperature -I 190 \ # Ignore Air Temperature -I 9 \ # Ignore power-on hours -a # Report on: # SMART health status (-H) # usage failures (-f) # changes in Prefailure, Usage (-t) # increases in error (-l error) # increases in self-test errors (-l selftest) /dev/twa0 \ -s L/../../4/22 \ # Long Self-tests Thursdays between 10pm-midnight -d 3ware,10 \ # Eleventh physical drive, behind 3Ware controller -I 194 \ # Ignore temperature -I 190 \ # Ignore Air Temperature -I 9 \ # Ignore power-on hours -a # Report on: # SMART health status (-H) # usage failures (-f) # changes in Prefailure, Usage (-t) # increases in error (-l error) # increases in self-test errors (-l selftest) /dev/twa0 \ -s L/../../5/01 \ # Long Self-tests Fridays between 1-3am -d 3ware,11 \ # Twelfth physical drive, behind 3Ware controller -I 194 \ # Ignore temperature -I 190 \ # Ignore Air Temperature -I 9 \ # Ignore power-on hours -a # Report on: # SMART health status (-H) # usage failures (-f) # changes in Prefailure, Usage (-t) # increases in error (-l error) # increases in self-test errors (-l selftest) /dev/twa0 \ -s L/../../5/22 \ # Long Self-tests Fridays between 10pm-midnight -d 3ware,12 \ # Thirteenth physical drive, behind 3Ware controller -I 194 \ # Ignore temperature -I 190 \ # Ignore Air Temperature -I 9 \ # Ignore power-on hours -a # Report on: # SMART health status (-H) # usage failures (-f) # changes in Prefailure, Usage (-t) # increases in error (-l error) # increases in self-test errors (-l selftest) /dev/twa0 \ -s L/../../6/01 \ # Long Self-tests Saturdays between 1-3am -d 3ware,13 \ # Fourteenth physical drive, behind 3Ware controller -I 194 \ # Ignore temperature -I 190 \ # Ignore Air Temperature -I 9 \ # Ignore power-on hours -a # Report on: # SMART health status (-H) # usage failures (-f) # changes in Prefailure, Usage (-t) # increases in error (-l error) # increases in self-test errors (-l selftest) /dev/twa0 \ -s L/../../6/03 \ # Long Self-tests Saturdays between 3-5am -d 3ware,14 \ # Fifteenth physical drive, behind 3Ware controller -I 194 \ # Ignore temperature -I 190 \ # Ignore Air Temperature -I 9 \ # Ignore power-on hours -a # Report on: # SMART health status (-H) # usage failures (-f) # changes in Prefailure, Usage (-t) # increases in error (-l error) # increases in self-test errors (-l selftest) /dev/twa0 \ -s L/../../6/22 \ # Long Self-tests Saturdays between 10pm-midnight -d 3ware,15 \ # Sixteenth physical drive, behind 3Ware controller -I 194 \ # Ignore temperature -I 190 \ # Ignore Air Temperature -I 9 \ # Ignore power-on hours -a # Report on: # SMART health status (-H) # usage failures (-f) # changes in Prefailure, Usage (-t) # increases in error (-l error) # increases in self-test errors (-l selftest)
Available SMART parameters
In order to find what parameters are available for monitoring or ignoring, issue the smartctl command:
hostname # smartctl --all /dev/twa0 -d 3ware,0 (of course, change the ,0 to any other 3Ware-connected drive you're interested in)
How Long Does a (Long) Selftest Take
To learn how long you should allow for a Long SelfTest (typical values are 72min for a 150GB WD Raptor, and 150min for a 500GB WD RE2):
hostname # smartctl -c /dev/twa0 -d 3ware,0 (of course, change the ,0 to any other 3Ware-connected drive you're interested in)