From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 18837 invoked by alias); 9 Apr 2008 03:43:56 -0000 Received: (qmail 18770 invoked by uid 9702); 9 Apr 2008 03:43:55 -0000 Date: Wed, 09 Apr 2008 03:43:00 -0000 Message-ID: <20080409034355.18754.qmail@sourceware.org> From: fabbione@sourceware.org To: cluster-cvs@sources.redhat.com, cluster-devel@redhat.com Subject: Cluster Project branch, STABLE2, updated. cluster-2.02.00-57-g3467c2e X-Git-Refname: refs/heads/STABLE2 X-Git-Reftype: branch X-Git-Oldrev: 4a827a5defcc8c5a843b12442e1411beffe712f5 X-Git-Newrev: 3467c2e4c5133a1bc9b9bead66a211f0124ccf4f Mailing-List: contact cluster-cvs-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Post: List-Help: , Sender: cluster-cvs-owner@sourceware.org X-SW-Source: 2008-q2/txt/msg00037.txt.bz2 This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Cluster Project". http://sources.redhat.com/git/gitweb.cgi?p=cluster.git;a=commitdiff;h=3467c2e4c5133a1bc9b9bead66a211f0124ccf4f The branch, STABLE2 has been updated via 3467c2e4c5133a1bc9b9bead66a211f0124ccf4f (commit) via 970cba7cd07d69de25393e616c3daa0c1a9c3024 (commit) via 771f0c4dbe87acb758840efd13ef72c613379e0d (commit) via c38c8e4d437dcdac7b20692ce4c560c1a553c4f6 (commit) via 5efd73d766df4affbfca07ff92ce38b0de43f87e (commit) via 673896d5ea1aebd62a4ec011c049a99bd7e7c991 (commit) via 76d27b41a014efd9d9df974ac6ab640099b49c85 (commit) via 12f9632c7964891ca5a75ccad1cb29eec718dd20 (commit) via aa9b413135d2488c52a110dcb2d389080b81cc33 (commit) via b9eb7cfb68a26248702ef00b8b1c6d33c86fd549 (commit) via 0a6c6b19a6fe5c688c34d578542df8cce781f687 (commit) via b3cce8a1a61ff8e7b3d5ef9888e391abed672c91 (commit) via c828cabab9cc82b38b0f039b0d5f72682c4f836e (commit) via a17c7093b22fcbfc363f1808dc57b06aa47eb09d (commit) via 6c7aa8b977cb44a9c642778b3654af8f79da0d97 (commit) via a46629aff677ce45cacc4aa6dab26061fca004d3 (commit) via 3fb2ed252790c8c68f92b4a61abaa472a36a69bc (commit) via a14739496c021b6f950d46c38be04a1f44ae8bda (commit) via 13aeff6b6e6b4d439a0de0f1e006b8feceaeb9d1 (commit) from 4a827a5defcc8c5a843b12442e1411beffe712f5 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 3467c2e4c5133a1bc9b9bead66a211f0124ccf4f Author: Ryan O'Hara Date: Mon Apr 7 11:16:55 2008 -0500 BZ 441323 : Redirect stderr to /dev/null when getting list of devices. commit 970cba7cd07d69de25393e616c3daa0c1a9c3024 Author: Chris Feist Date: Thu Jan 24 22:20:23 2008 +0000 Added back in change to description line to make chkconfig work properly. commit 771f0c4dbe87acb758840efd13ef72c613379e0d Author: Ryan O'Hara Date: Fri Jan 18 23:02:58 2008 +0000 BZ: 373491, 373511, 373531, 373541, 373571, 429033 BZ: 373491, 373511, 373531, 373541, 373571, 429033 - Prevent "reservation conflict" messageswhen scsi_reserve starts. - Leave the fence domain if scsi_reserve fails to register with any device. - Improve logging in scsi_reserve script. - Use "locking_type = 0" for all lvm commands (ie. vgs). - Fix SCSI reservations scripts to handle LVM mirrors and stripes. - Not an error if fence_scsi attempts to remove a non-existent key from a device. commit c38c8e4d437dcdac7b20692ce4c560c1a553c4f6 Author: Ryan O'Hara Date: Thu Nov 8 18:48:08 2007 +0000 BZ 248715 - Use cluster ID and node ID for key rather than IP address. commit 5efd73d766df4affbfca07ff92ce38b0de43f87e Author: Ryan McCabe Date: Fri Sep 21 03:09:04 2007 +0000 fix bz277781 by accepting "nodename" as a synonym for "node" commit 673896d5ea1aebd62a4ec011c049a99bd7e7c991 Author: Ryan O'Hara Date: Tue Mar 6 19:10:34 2007 +0000 Fix help message to refer to script as 'fence_scsi_test'. commit 76d27b41a014efd9d9df974ac6ab640099b49c85 Author: Ryan O'Hara Date: Thu Dec 14 21:18:09 2006 +0000 Attempt to register the node in the case where it must perform fence_scsi fencing but is not registered with the device(s) that must be fence. With SCSI persistent reservations, in order to do a "preempt and abort" (which we are using to fence a node), the node doing this operation must be registered with the device. This fix will check to see the the node that is performing the fencing is registered with the device(s). If it is not, then it attempts to register with the device(s) so that it can then continue with the fence operation normally. Note that this situation should never happen, especially if things are configured properly. commit 12f9632c7964891ca5a75ccad1cb29eec718dd20 Author: Ryan O'Hara Date: Thu Dec 14 18:32:43 2006 +0000 Allow 'stop' to release the reservation if and only if there are no other keys registered with a given device. Prior to this fix, if was not possible for 'scsi_reserve stop' to unregister/release on the node that was holding the reservation. commit aa9b413135d2488c52a110dcb2d389080b81cc33 Author: Ryan O'Hara Date: Tue Dec 12 23:20:47 2006 +0000 Record devices that are successfully registered to /var/run/scsi_reserve. commit b9eb7cfb68a26248702ef00b8b1c6d33c86fd549 Author: Ryan O'Hara Date: Tue Dec 12 22:22:51 2006 +0000 Rewrite of get_scsi_devices function. It is no longer possible to use lvs to get a list of cluster volumes (and underlying devices) at fence time. For this reason we must "keep state" by recording which devices we register with at startup. The init script (scsi_reserve) will record each device it successfully registered with to a file (/var/run/scsi_reserve). Then, and fence time, the fence_scsi agent will unregister each device listed in the state file. commit 0a6c6b19a6fe5c688c34d578542df8cce781f687 Author: Ryan O'Hara Date: Tue Dec 12 21:32:53 2006 +0000 Fix success/failure reporting when registering devices at startup. If our node (key) is already registered with a given device, do not report failure since this is misleading. commit b3cce8a1a61ff8e7b3d5ef9888e391abed672c91 Author: Ryan O'Hara Date: Fri Dec 8 22:22:34 2006 +0000 Replace /var/lock/subsys/${0##*/} with /var/lock/subsys/scsi_reserve. commit c828cabab9cc82b38b0f039b0d5f72682c4f836e Author: Ryan O'Hara Date: Fri Dec 8 21:23:00 2006 +0000 Fix split calls to be consistent. Remove the optional LIMIT parameter. commit a17c7093b22fcbfc363f1808dc57b06aa47eb09d Author: Ryan O'Hara Date: Fri Dec 8 20:48:00 2006 +0000 Fix code to use get_key subroutine. Fix sg_persist commands to specify device via -d parameter. commit 6c7aa8b977cb44a9c642778b3654af8f79da0d97 Author: Ryan O'Hara Date: Fri Dec 1 17:53:22 2006 +0000 Remove "self" parameter. This was used to specify the name of the node performing the fence operation, and was passed to the agent. This is no longer used. Instead, we get the name of the local node in the agent by parsing the output from 'cman_tool status'. commit a46629aff677ce45cacc4aa6dab26061fca004d3 Author: Ryan O'Hara Date: Wed Nov 29 20:54:42 2006 +0000 Fix unregister code to report failure correctly. commit 3fb2ed252790c8c68f92b4a61abaa472a36a69bc Author: Ryan O'Hara Date: Wed Nov 29 20:50:22 2006 +0000 Variable should be quoted in conditional statement. commit a14739496c021b6f950d46c38be04a1f44ae8bda Author: Fabio M. Di Nitto Date: Wed Apr 9 05:40:44 2008 +0200 Revert "fix bz277781 by accepting "nodename" as a synonym for "node"" This reverts commit 57a07697afeb2e5d3bb2a4220e844bcdc44598cb. commit 13aeff6b6e6b4d439a0de0f1e006b8feceaeb9d1 Author: Fabio M. Di Nitto Date: Wed Apr 9 05:40:35 2008 +0200 Revert "Fix help message to refer to script as 'fence_scsi_test'." This reverts commit e9b17a088668ceaba5c6ea35f8d22ec2613cfe96. ----------------------------------------------------------------------- Summary of changes: fence/agents/scsi/fence_scsi.pl | 248 ++++++++++++++++++++++++---- fence/agents/scsi/fence_scsi_test.pl | 171 ++++++++++---------- fence/agents/scsi/scsi_reserve | 300 +++++++++++++++++++++++----------- 3 files changed, 506 insertions(+), 213 deletions(-) diff --git a/fence/agents/scsi/fence_scsi.pl b/fence/agents/scsi/fence_scsi.pl index 89822e8..b9aea99 100755 --- a/fence/agents/scsi/fence_scsi.pl +++ b/fence/agents/scsi/fence_scsi.pl @@ -1,19 +1,16 @@ #!/usr/bin/perl use Getopt::Std; +use XML::LibXML; use IPC::Open3; use POSIX; -my $verbose = 0; my @volumes; $_ = $0; s/.*\///; my $pname = $_; -# WARNING!! Do not add code bewteen "#BEGIN_VERSION_GENERATION" and -# "#END_VERSION_GENERATION" It is generated by the Makefile - #BEGIN_VERSION_GENERATION $RELEASE_VERSION=""; $REDHAT_COPYRIGHT=""; @@ -62,13 +59,137 @@ sub fail_usage exit 1; } +sub get_cluster_id +{ + my $cluster_id; + + my ($in, $out, $err); + my $cmd = "cman_tool status"; + + my $pid = open3($in, $out, $err, $cmd) or die "$!\n"; + + waitpid($pid, 0); + + die "Unable to execute cman_tool.\n" if ($?>>8); + + while (<$out>) + { + chomp; + print "OUT: $_\n" if $opt_v; + + my ($name, $value) = split(/\s*:\s*/, $_); + + if ($name eq "Cluster Id") + { + $cluster_id = $value; + last; + } + } + + close($in); + close($out); + close($err); + + return $cluster_id; +} + +sub get_node_id +{ + ($node)=@_; + + my $xml = XML::LibXML->new(); + my $tree = $xml->parse_file("/etc/cluster/cluster.conf"); + + my $xpath = "//cluster/clusternodes/clusternode[\@name='$node']/\@nodeid"; + + my $node_id = $tree->findvalue($xpath); + + return $node_id; +} + +sub get_node_name +{ + return $opt_n; +} + +sub get_host_id +{ + my $host_id; + + my ($in, $out, $err); + my $cmd = "cman_tool status"; + + my $pid = open3($in, $out, $err, $cmd) or die "$!\n"; + + waitpid($pid, 0); + + die "Unable to execute cman_tool.\n" if ($?>>8); + + while (<$out>) + { + chomp; + print "OUT: $_\n" if $opt_v; + + my ($name, $value) = split(/\s*:\s*/, $_); + + if ($name eq "Node ID") + { + $host_id = $value; + last; + } + } + + close($in); + close($out); + close($err); + + return $host_id; +} + +sub get_host_name +{ + my $host_name; + + my ($in, $out, $err); + my $cmd = "cman_tool status"; + + my $pid = open3($in, $out, $err, $cmd) or die "$!\n"; + + waitpid($pid, 0); + + die "Unable to execute cman_tool.\n" if ($?>>8); + + while (<$out>) + { + chomp; + print "OUT: $_\n" if $opt_v; + + my ($name, $value) = split(/\s*:\s*/, $_); + + if ($name eq "Node name") + { + $host_name = $value; + last; + } + } + + close($in); + close($out); + close($err); + + return $host_name; +} + sub get_key { ($node)=@_; - my $addr = gethostbyname($node) or die "$!\n"; + my $cluster_id = get_cluster_id; + my $node_id = get_node_id($node); + + my $key = sprintf "%x%.4x", $cluster_id, $node_id; - return unpack("H*", $addr); + return $key; } sub get_options_stdin @@ -93,7 +214,7 @@ sub get_options_stdin next unless $opt; - ($name, $val) = split /\s*=\s*/, $opt; + ($name, $val) = split(/\s*=\s*/, $opt); if ($name eq "") { @@ -111,10 +232,6 @@ sub get_options_stdin { $opt_n = $val; } - elsif ($name eq "self") - { - $opt_s = $val; - } elsif ($name eq "verbose") { $opt_v = $val; @@ -126,10 +243,45 @@ sub get_options_stdin } } +sub get_key_list +{ + ($dev) = @_; + + my ($in, $out, $err); + + my $cmd = "sg_persist -n -d $dev -i -k"; + my $pid = open3($in, $out, $err, $cmd) or die "$!\n"; + + waitpid($pid, 0); + + die "Unable to execute sg_persist.\n" if ($?>>8); + + while (<$out>) + { + chomp; + if ($_ =~ /^\s*0x/) + { + s/^\s+0x//; + s/\s+$//; + + $key_list{$_} = 1; + } + } + + close($in); + close($out); + close($err); + + return %key_list; +} + sub get_scsi_devices { my ($in, $out, $err); - my $cmd = "lvs --noheadings --separator : -o vg_attr,devices"; + + my $cmd = "vgs --config 'global { locking_type = 0 }'" . + " --noheadings --separator : -o vg_attr,pv_name 2> /dev/null"; + my $pid = open3($in, $out, $err, $cmd) or die "$!\n"; waitpid($pid, 0); @@ -141,12 +293,12 @@ sub get_scsi_devices chomp; print "OUT: $_\n" if $opt_v; - my ($vg_attrs, $device) = split /:/, $_, 3; + my ($vg_attrs, $device) = split(/:/, $_); if ($vg_attrs =~ /.*c$/) { $device =~ s/\(.*\)//; - push @volumes, $device; + push(@volumes, $device); } } @@ -176,43 +328,74 @@ sub check_sg_persist close($err); } +sub do_register +{ + ($dev, $key) = @_; + + my ($in, $out, $err); + my $cmd = "sg_persist -n -d $dev -o -G -S $key"; + my $pid = open3($in, $out, $err, $cmd) or die "$!\n"; + + waitpid($pid, 0); + + die "Unable to execute sg_persist ($dev).\n" if ($?>>8); + + while (<$out>) + { + chomp; + print "OUT: $_\n" if $opt_v; + } + + close($in); + close($out); + close($err); +} + sub fence_node { - # my $name = (POSIX::uname())[1]; + my $host_name = get_host_name(); + my $node_name = get_node_name(); + + my $host_key = get_key($host_name); + my $node_key = get_key($node_name); - my $host_key = get_key($opt_s); - my $node_key = get_key($opt_n); - - my $cmd; my ($in, $out, $err); foreach $dev (@volumes) { + my %key_list = get_key_list($dev); + + if (!$key_list{$host_key}) + { + do_register($dev, $host_key); + } + + if (!$key_list{$node_key}) + { + next; + } + if ($host_key eq $node_key) { - $cmd = "sg_persist -d $dev -o -G -K $host_key -S 0"; + $cmd = "sg_persist -n -d $dev -o -G -K $host_key -S 0"; } else { - $cmd = "sg_persist -d $dev -o -A -K $host_key -S $node_key -T 5"; + $cmd = "sg_persist -n -d $dev -o -A -K $host_key -S $node_key -T 5"; } my $pid = open3($in, $out, $err, $cmd) or die "$!\n"; waitpid($pid, 0); - if ($opt_v) + die "Unable to execute sg_persist ($dev).\n" if ($?>>8); + + while (<$out>) { - print "$cmd\n"; - while (<$out>) - { - chomp; - print "OUT: $_\n"; - } + chomp; + print "OUT: $_\n" if $opt_v; } - die "Unable to execute sg_persist.\n" if ($?>>8); - close($in); close($out); close($err); @@ -228,16 +411,15 @@ if (@ARGV > 0) { usage if defined $opt_h; version if defined $opt_V; - fail_usage "Unkown parameter." if (@ARGV > 0); + fail_usage "Unknown parameter." if (@ARGV > 0); + fail_usage "No '-n' flag specified." unless defined $opt_n; - fail_usage "No '-s' flag specified." unless defined $opt_s; } else { get_options_stdin(); fail "failed: missing 'node'" unless defined $opt_n; - fail "failed: missing 'self'" unless defined $opt_s; } diff --git a/fence/agents/scsi/fence_scsi_test.pl b/fence/agents/scsi/fence_scsi_test.pl index 0076144..b8e0cc0 100755 --- a/fence/agents/scsi/fence_scsi_test.pl +++ b/fence/agents/scsi/fence_scsi_test.pl @@ -1,121 +1,96 @@ #!/usr/bin/perl +use POSIX; use IPC::Open3; -use Sys::Hostname; +use XML::LibXML; use Getopt::Std; -use POSIX; my @devices; my %results; -# WARNING!! Do not add code bewteen "#BEGIN_VERSION_GENERATION" and -# "#END_VERSION_GENERATION" It is generated by the Makefile - #BEGIN_VERSION_GENERATION $RELEASE_VERSION=""; $REDHAT_COPYRIGHT=""; $BUILD_DATE=""; #END_VERSION_GENERATION -sub get_key +sub get_scsi_block_devices { - my $addr = gethostbyname($name) or die "$!\n"; + my $block_dir = "/sys/block"; - return unpack("H*", $addr); -} + opendir(DIR, $block_dir) or die "$!\n"; -sub register_device -{ - my $func = (caller(0))[3]; - my ($dev, $key) = @_; + my @block_devices = grep { /^sd*/ } readdir(DIR); - print "DEBUG: $func ($dev, $key)\n" if ($opt_d); + closedir(DIR); + for $block_dev (@block_devices) + { + push(@devices, "/dev/" . $block_dev); + } +} + +sub get_cluster_vol_devices +{ my ($in, $out, $err); - my $cmd = "sg_persist $dev -o -G -S $key"; + + my $cmd = "vgs --config 'global { locking_type = 0 }'" . + " --noheadings --separator : -o vg_attr,pv_name"; my $pid = open3($in, $out, $err, $cmd) or die "$!\n"; waitpid($pid, 0); - my $rval = WEXITSTATUS($?); + die "[error] unable to execute vgs command.\n" if WEXITSTATUS($?); - $results{$dev}[0] = $rval; + while (<$out>) + { + chomp; - print "DEBUG: [$rval] $cmd\n" if $opt_d; + my ($vg_attr, $pv_name) = split(/:/, $_); + + if ($vg_attr =~ /.*c$/) + { + ###### DEBUG ###### + print "DEBUG: pv_name = $pv_name\n"; + + push(@devices, $pv_name); + } + } close($in); close($out); close($err); - - return $rval; } -sub unregister_device +sub register_device { - my $func = (caller(0))[3]; my ($dev, $key) = @_; - - print "DEBUG: $func ($dev, $key)\n" if $opt_d; - my ($in, $out, $err); - my $cmd = "sg_persist $dev -o -G -K $key -S 0"; + my $cmd = "sg_persist -n -d $dev -o -G -S $key"; my $pid = open3($in, $out, $err, $cmd) or die "$!\n"; waitpid($pid, 0); - my $rval = WEXITSTATUS($?); - - $results{$dev}[1] = $rval; - - print "DEBUG: [$rval] $cmd\n" if $opt_d; + $results{$dev}[0] = WEXITSTATUS($?); close($in); close($out); close($err); - - return $rval; -} - -sub get_block_devices -{ - my $block_dir = "/sys/block"; - - opendir(DIR, $block_dir) or die "Error: $! $block_dir\n"; - - my @block_devices = grep { /^sd*/ } readdir(DIR); - - closedir(DIR); - - for $dev (@block_devices) - { - push @devices, "/dev/" . $dev; - } } -sub get_cluster_devices +sub unregister_device { + my ($dev, $key) = @_; my ($in, $out, $err); - my $cmd = "lvs --noheadings --separator : -o vg_attr,devices"; + + my $cmd = "sg_persist -n -d $dev -o -G -K $key -S 0"; my $pid = open3($in, $out, $err, $cmd) or die "$!\n"; waitpid($pid, 0); - die "Error: unable to exec lvs command.\n" if WEXITSTATUS($?); - - while (<$out>) - { - chomp; - - my ($vg_attr, $dev) = split /:/, $_, 3; - - if ($vg_attr =~ /.*c$/) - { - $dev =~ s/\(.*\)//; - push @devices, $dev; - } - } + $results{$dev}[1] = WEXITSTATUS($?); close($in); close($out); @@ -124,21 +99,38 @@ sub get_cluster_devices sub test_devices { - my $name = hostname() or die "$!\n"; - my $addr = gethostbyname($name) or die "$!\n"; - my $key = unpack("H*", $addr); + my $key = "0xDEADBEEF"; foreach $dev (@devices) { - if (register_device($dev, $key) != 0) - { - } - if (unregister_device($dev, $key) != 0) - { - } + register_device($dev, $key); + unregister_device($dev, $key); } } +sub check_config_fence +{ + my $xml = XML::LibXML->new(); + my $tree = $xml->parse_file("/etc/cluster/cluster.conf"); + my $root = "//cluster/fencedevices/fencedevice"; + + my $xpath_fence = "count(${root}[\@agent='fence_scsi'])"; + + return ( ! $tree->findvalue($xpath_fence)); +} + +sub check_config_nodes +{ + my $xml = XML::LibXML->new(); + my $tree = $xml->parse_file("/etc/cluster/cluster.conf"); + my $root = "//cluster/clusternodes/clusternode"; + + my $xpath_name = "count(${root}/\@name)"; + my $xpath_nodeid = "count(${root}/\@nodeid)"; + + return ($tree->findvalue($xpath_name) != $tree->findvalue($xpath_nodeid)); +} + sub print_results { my $device_count = scalar(@devices); @@ -195,9 +187,9 @@ sub print_usage print " -h Help. Prints out this usage information.\n\n"; } -### main ### +### MAIN ####################################################################### -if (getopts("cdhsv") == 0) +if (getopts("cdhst:v") == 0) { print_usage; exit(1); @@ -212,23 +204,34 @@ if ($opt_h) if ($opt_c) { print "\nTesting devices in cluster volumes...\n"; - get_cluster_devices; + get_cluster_vol_devices; + test_devices; + print_results; } if ($opt_s) { print "\nTesting all SCSI block devices...\n"; - get_block_devices; + get_scsi_block_devices; + test_devices; + print_results; } -if (!$opt_c && !$opt_s) +if ($opt_t) +{ + if ($opt_t eq "fence") + { + exit check_config_fence; + } + if ($opt_t eq "nodes") + { + exit check_config_nodes; + } +} + +if (!$opt_c && !$opt_s && !$opt_t) { print "\nPlease specify either cluster or SCSI mode.\n"; print_usage; exit(1); } - -test_devices; - -print_results; - diff --git a/fence/agents/scsi/scsi_reserve b/fence/agents/scsi/scsi_reserve index a4057c7..67c0d2d 100755 --- a/fence/agents/scsi/scsi_reserve +++ b/fence/agents/scsi/scsi_reserve @@ -1,140 +1,248 @@ #!/bin/bash # +# scsi_reserve: +# # chkconfig: 345 25 75 # description: start/stop persistent reservation service for lvm +# config: /etc/sysconfig/scsi_reserve . /etc/init.d/functions -# check for sg_persist command provided by sg3_utils package +# read in config file if it exists # -if ! sg_persist -V &> /dev/null ; then - echo "error: sg_persist not found" - exit 2 +if [ -f /etc/sysconfig/scsi_reserve ] ; then + . /etc/sysconfig/scsi_reserve fi -# check for gethostip command provided by syslinux package +# check if cluster is configured for fence_scsi # -if ! gethostip -h &> /dev/null ; then - echo "error: gethostip not found" - exit 3 +if ! fence_scsi_test -t fence ; then + logger -t scsi_reserve \ + "[error] cluster not configured for scsi reservations" + exit 1 fi -# get scsi devices that are part of clustered volumes +# check for nodeids in config file # -scsi_devices=$( lvs -o vg_attr,devices --noheadings \ - | awk --posix ' $1 ~ /[-a-z]{5}c/ { print $2 } ' \ - | sed -e 's/([0-9]*)//' | sort | uniq ) +if ! fence_scsi_test -t nodes ; then + logger -t scsi_reserve \ + "[error] cluster must define nodeid for all nodes" + exit 1 +fi -# if no scsi devices were found we can exit now +# check for sg_persist command provided by sg3_utils package # -[ -z $scsi_devices ] && exit 0 +if ! sg_persist -V &> /dev/null ; then + logger -t scsi_reserve \ + "[error] unable to exec sg_persist" + exit 1 +fi -# get the node name and node addr from cman +# check that cman is running # -node_name=$( cman_tool status | grep "Node name" | awk -F": " '{print $2}' ) -node_addr=$( cman_tool status | grep "Node addr" | awk -F": " '{print $2}' ) +if ! cman_tool status &> /dev/null ; then + logger -t scsi_reserve \ + "[error] cman does not appear to be running" + exit 1 +fi -# create unique key for this host +# get physical volumes (devices) that are part of cluster volumes # -key=$( gethostip -x $node_name ) - -case $1 in - -start) +scsi_devices=$( vgs --config 'global { locking_type = 0 }' \ + --noheadings -o vg_attr,pv_name 2> /dev/null \ + | awk ' $1 ~ /.*c$/ { print $2 } ' ) + +if [ -z "$scsi_devices" ] ; then + logger -t scsi_reserve \ + "[error] did not find devices in cluster volumes" + exit 1 +fi -rval=0 +# get the cluster id from cman +# +cluster_id=$( cman_tool status | grep -i "Cluster ID" \ + | awk -F": " '{ print $2 }' ) -touch /var/lock/subsys/${0##*/} +if [ -z "$cluster_id" ] ; then + logger -s -t scsi_reserve \ + "[error] unable to determine cluster id" + exit 1 +fi -# register each device using our key +# get the node id from cman # -for dev in $scsi_devices -do +node_id=$( cman_tool status | grep -i "Node ID" \ + | awk -F": " '{ print $2 }' ) - echo -n "Registering device: $dev" +if [ -z "$node_id" ] ; then + logger -t scsi_reserve \ + "[error] unable to determine node id" + exit 1 +fi - for error in 1 - do - sg_persist -d $dev -o -G -S $key &>/dev/null || break - # sg_persist -d $dev -o -R -K $key -T 5 &>/dev/null || break - error=0 - done +# generate unique key using cluster_id and node_id +# +key=$( printf "%x%.4x" $cluster_id $node_id ) - if [ $error -eq 0 ]; then - success - else - failure - rval=1 - fi +if [ -z "$key" ] ; then + logger -t scsi_reserve \ + "[error] unable to generate key" + exit 1 +fi - echo +################################################################################ - # attempt to create a reservation - # - sg_persist -d $dev -o -R -K $key -T 5 &>/dev/null +case $1 in -done -;; + start) -stop) + error=0 + count=0 -rval=0 + echo -n "Starting scsi_reserve:" -# unregister each device for this node -# -for dev in $scsi_devices -do - - if sg_persist -d $dev -i -r 2>/dev/null | grep -qiE "${key#0}" ; then - echo "Unable to unregister device: $dev" - error=1 - else - echo -n "Unregistering device: $dev" - for error in 1 + for dev in $scsi_devices do - sg_persist -d $dev -o -G -K $key -S 0 &>/dev/null || break + # check if our key is already resgistered with this device + # + if sg_persist -n -d $dev -i -k | grep -qiE "^[[:space:]]*0x$key" ; then + logger -t scsi_reserve \ + "[info] already registered with $dev (key=0x$key)" + continue + fi + + # create the scsi registration + # + if ! sg_persist -n -d $dev -o -I -S $key &> /dev/null ; then + logger -t scsi_reserve \ + "[error] unable to register device $dev (key=0x$key)" + : $[ count = $count + 1 ] + error=1 + else + logger -t scsi_reserve \ + "[info] registered with device $dev (key=0x$key)" + fi + + # check to see if reservation already exists + # + if sg_persist -n -d $dev -i -r | grep -qiE "^[[:space:]]*Key=0x" ; then + logger -t scsi_reserve \ + "[info] reservation already exists on $dev" + continue + fi + + # create the scsi reservation + # + if ! sg_persist -n -d $dev -o -R -K $key -T 5 &> /dev/null ; then + logger -t scsi_reserver \ + "[error] unable to create reservation on $dev (key=0x$key)" + : $[ count = $count + 1 ] + error=1 + fi + done + + # leave fence domain if any errors occured during registration + # + if [ $error -eq 0 ] ; then + success + else + logger -t scsi_reserve \ + "[info] $count errors during registration" + logger -t scsi_reserve \ + "[info] leaving the fence domain" + fence_tool leave + failure + fi + + echo + + ;; # end of start + + stop) + error=0 - done - fi + count=0 - if [ $error -eq 0 ]; then - success - else - failure - rval=1 - fi + echo -n "Stopping scsi_reserve:" - echo -done + for dev in $scsi_devices + do + # get list of keys registered with this device + # + key_list=$( sg_persist -n -d $dev -i -k | grep -iE "^[[:space:]]*0x" ) + + # check that our key is registered with this device + # + if ! sg_persist -d $dev -i -k | grep -qiE "^[[:space:]]*0x$key" ; then + logger -t scsi_reserve \ + "[info] not registered with $dev (key=0x$key)" + continue + fi + + # check if our key is the reservation holder + # + if sg_persist -n -d $dev -i -r 2>/dev/null | grep -qiE "$key" ; then + if echo "$key_list" | grep -qivE "$key" ; then + logger -t scsi_reserve \ + "[error] unable to remove registration on $dev (key=0x$key)" + : $[ count = $count + 1 ] + error=1 + continue + fi + fi + + # remove registration for this device + # + if ! sg_persist -n -d $dev -o -G -K $key -S 0 &> /dev/null ; then + logger -t scsi_reserve \ + "[error] failed to remove registration on $dev (key=0x$key)" + : $[ count = $count + 1 ] + error=1 + else + logger -t scsi_reserve \ + "[info] removed registration on $dev (key=0x$key)" + fi + + done + + # report success or failure + # + if [ $error -eq 0 ] ; then + success + else + logger -t scsi_reserve \ + "[info] $count errors occured during unregistration" + failure + fi + + echo + + ;; # end of stop + + status) -rm -f /var/lock/subsys/${0##*/} -;; + error=0 -status) + for dev in $scsi_devices + do + if sg_persist -n -d $dev -i -k | grep -qiE "$key" ; then + devices[${#devices[@]}]=$dev + fi + done -rval=0 + if [ -z "$devices" ] ; then + echo "No registered devices found." + else + echo "Found ${#devices[@]} registered device(s):" -# find devices that are registered with our key -# -for dev in $scsi_devices -do - if sg_persist -d $dev -i -k 2>/dev/null | grep -qiE "${key#0}" ; then - devices[${#devices[@]}]=$dev - fi -done - -if [ -z $devices ]; then - echo "No devices registered." -else - echo "Found ${#devices[@]} registered device(s):" - - for i in "${devices[@]}" ; do - echo $i - done -fi -;; + for i in "${devices[@]}" + do + echo $i + done + fi -esac + ;; # end of status -exit $rval +esac +exit $error hooks/post-receive -- Cluster Project