Added a semicolon to run-esx-report.sh that was left out and responsible for some ugly HTML formatting.
With the secure SSH access problem solved in Part 2, we'll move on to getting the data in the proper format and emailing it from the ESX Service Console. As you probably know, the Linux distribution installed with ESX 3.5 lacks
sendmail or an equivalent command, but we can roll our own from a perl script.The perl mailer script
We need to import two perl modules for the script, and both are included by default in the Service Console.
Getopt::Std provides a simple way to get command line options, and Net::SMTP will interface with an Exchange or SMTP server accessible from the console network:
use Getopt::Std;
use Net::SMTP;
This
getopt call is all that's necessary to declare the command line options (-f, -r, -s, etc.), and it will automatically populate a set of corresponding variables named opt_*. We'll do a quick check to make sure all the command line options were specified, and if not display the usage message:
getopt ('frsmb');
unless ($opt_f && $opt_r && $opt_s && $opt_m && $opt_b) {
print_usage();
exit 1;
}
Next we'll create a filehandle named
BODY, opening the file specified on the command line. After reading in each line to the variable body_data, we'll close the handle:
open(BODY, $opt_b) || error("Could not open file $opt_b.");
my @body_data=<BODY>;
close(BODY);
The
Net::SMTP module is pretty straightforward. To generate a HTML formatted email, we just need to specify the MIME version, the content type as HTML, and the character encoding as ISO 8859-1. If you would rather send the message as plain text, just remove those two lines:
my $smtp = Net::SMTP->new($opt_m) ||
error("SMTP connection to $opt_m failed.");
$smtp->mail($opt_f);
$smtp->to($opt_r);
$smtp->data();
$smtp->datasend("MIME-Version: 1.0\n");
$smtp->datasend("Content-Type: text/html; charset=iso-8859-1\n");
$smtp->datasend("To: $opt_r\n");
$smtp->datasend("From: $opt_f\n");
$smtp->datasend("Subject: $opt_s\n");
foreach $line (@body_data)
{
$smtp->datasend("$line");
}
$smtp->dataend();
$smtp->quit;
Here's the complete
html-mailer.pl script:
|
Enable outbound SMTP
Now that we've got a script that we can send test messages with, we need to enable outbound SMTP through the ESX firewall on the ESX server that will have the script scheduled from a
cron job. Just type this command as root to open the port:
|
If your Exchange or SMTP server is reachable from the Service Console network, execute
html-mailer.pl with the appropriate parameters, and specify any old text file:
./html-mailer.pl -f me@mydomain.dom \
-r me@mydomain.dom \
-s "Test message" \
-m exchange.mydomain.dom \
-b ./testfile.txt
The Service Console can't reach the Exchange server...
No worries, as long as you're able to reach the VirtualCenter server, we can install the SMTP service and set it up to forward to the Exchange server. To install SMTP on a Windows 2003 server, do the following:
- Open the Control Panel > Add or Remove Programs > Add/Remove Windows Components > double click Application Server > and then double click Internet Information Server (IIS). Put a check next to SMTP Service and click OK, OK, and Next
- After the SMTP install is complete, open the Start Menu > Programs > Administrative Tools > Internet Information Services (IIS) Manager, then right click Default SMTP Virtual Server and select Properties
- In the General tab, drop down the IP address: to the IP address in the Service Console network, if different from the LAN. This will prevent the SMTP service from popping up on your network security guy's port scans :)
- In the Access tab, click the Connection button, choose the Only the list below radio button, then click Add to add the appropriate subnet address and mask to the Group of computers option, or add each ESX server one at a time
- In the Access tab again, click the Relay button, choose the Only the list below radio button, then click Add to add the appropriate subnet address and mask to the Group of computers option, or add each ESX server one at a time. Uncheck the option Allow all computers which successfully authenticate to relay
- On the Delivery tab, click the Advanced button and add your Exchange server information in the Smart host: box. By specifying a smart host, the SMTP server will simply forward everything to the Exchange server, letting it make all the decisions about which domains to accept mail for, etc.
- Now test out the SMTP forwarder by using
telnetto initiate a SMTP session from the ESX server that will be sending the messages:
telnet virtualcenter.lab.local 25 ehlo mail from:spongebob@lab.local rcpt to:administrator@lab.local data Subject:test . quit
Almost there, so let's recap what we've done so far. In Part 1, we created the health check script that will run on each ESX server and send key performance stats and scaled histograms to the terminal. Then in Part 2, we covered how to distribute public keys so the script can be executed on several ESX servers via SSH. So far in Part 3, we've looked at a perl script that will email the combined script output, and now we need to create a script to tie it all together, and then schedule the script from a
cron job.Let's break down the main components of the script. First of all, if
ssh-agent isn't running, the script isn't going to get very far, so we'll use pgrep to check for the process and exit if it's not found:
if ! pgrep ssh-agent >/dev/null; then
echo "The ssh-agent process does not appear to be running, exiting"
exit 1
fi
We need to source
.ssh-agent, the file with the ssh-agent PID and socket info set up by the start-ssh-agent.sh script, or exit if it doesn't exist:
source "${HOME}/.ssh-agent" >/dev/null || exit 1
Since we'll be running everything from an ESX Service Console, and that server is likely to be part of the health check, we should compare the list of ESX hosts to the local hostname so we don't open a SSH connection to the local machine. We use
cut here to strip off the domain name so we'll match whether the FQDN or just the bare hostname is specified:
THISHOST=$(hostname | cut -d . -f 1)
for host in $@; do
if [ $(echo $host | cut -d . -f 1) = $THISHOST ]; then
"${RUNDIR}/esx-report.sh" >> "$TEMPTEXT"
else
ssh -q $host "$(cat "${RUNDIR}/esx-report.sh")" >> "$TEMPTEXT" || \
printf "WARNING: SSH connection to $host failed\n\n\n\n" >> "$TEMPTEXT"
fi
done
After the health check script has looped through the list of ESX hosts, we'll start building the HTML file with the necessary tags. Setting the font size for the
pre tag is the secret sauce for getting the email to display perfectly on a BlackBerry:
cat > "$TEMPHTML" <<-'HEADEREOF'
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css">
body { font-family: monospace; font-size: 12px }
pre { font-family: monospace; font-size: 12px }
</style>
</head>
<body>
<pre>
HEADEREOF
If you need to use < or > symbols in a HTML document, you have to specify the actual ASCII code of the character, as HTML considers words wrapped in those symbols to be tags. We'll use a
sed filter to replace all the >'s with the ASCII equivalent, and add a color tag to any lines with the word WARNING to make it stand out:
cat "$TEMPTEXT" | \
sed -e 's/>/\>/g' \
-e 's/WARNING:.*/<span style="color: red">&<\/span>/' >> "$TEMPHTML"
cat >> "$TEMPHTML" <<-'FOOTEREOF'
</pre>
</body>
</html>
FOOTEREOF
And finally, we'll execute
html-mailer.pl with the appropriate parameters. You'll need to change this section of the script for your environment:
"${RUNDIR}/html-mailer.pl" -f esx-report@lab.local \
-r administrator@lab.local \
-s "ESX Health Report" \
-m lab-vc \
-b "$TEMPHTML"
Here's the
run-esx-report.sh script. Remember to change the email address and mail server parameters for your environment:
|
To cron foo, thanks for everything
Still with us? One more step, and it's an easy one. We'll add a
cron job to run the script at 7:10 AM every morning. Remember to add the job for the user account you distributed SSH keys for.To edit the
cron entries for the user, type:
crontab -e
This starts
vi and opens up the user's crontab. To enter insert mode, type iAssuming you've set everything up using the code segments used in this series, to add an entry for 7:10 AM, type this line, replacing ESX LIST with a space separated list of ESX hosts:
10 7 * * * ${HOME}/esx-report/run-esx-report.sh ESX LIST >/dev/null 2>&1
After adding the entry, press the
Esc key, and type :wq to write the crontab and quit.If you have a long list of hosts, put them all in a text file, separated by spaces or each on its own line, and use command substitution to feed the list to
run-esx-report.sh
run-esx-report.sh $(cat ${HOME}/esx-report/hostlist.txt)
There's more?!
What if we wanted to trigger an email warning if an ESX host exceeds a threshold value? As we'll see in Part 4, we can do this easily with a quick modification to the
run-esx-report.sh script.
What modifications should be made to the script if not using the ssh-agent functionality? That is, it would be run from a single host.
ReplyDeleteThanks.
It's pretty easy to modify it for one host. First of all, just ignore everything from Part 2, since you won't need any of the SSH functionality.
ReplyDeleteThen from the run-esx-report.sh script in Part 3, remove this check for a list of hosts to query, as we'll always be running on the local host:
if [ -z $1 ]; then
echo "No ESX hosts specified, exiting"
exit 1
fi
And remove this check to see if ssh-agent is running:
if ! pgrep ssh-agent >/dev/null; then
echo "The ssh-agent process does not appear to be running, exiting"
exit 1
fi
And remove the source command on the .ssh-agent file, since it won't be there:
source "${HOME}/.ssh-agent" >/dev/null || exit 1
We don't need to capture the local hostname, as we're always going to run anyway, so remove this:
THISHOST=$(hostname | cut -d . -f 1)
You could leave this loop, but it's probably better to replace this whole section:
for host in $@; do
if [ $(echo $host | cut -d . -f 1) = $THISHOST ]; then
"${RUNDIR}/esx-report.sh" >> "$TEMPTEXT"
else
ssh -q $host "$(cat "${RUNDIR}/esx-report.sh")" >> "$TEMPTEXT" || \
printf "WARNING: SSH connection to $host failed\n\n\n\n" >> "$TEMPTEXT"
fi
done
...with this command to just run the script on the local host:
"${RUNDIR}/esx-report.sh" >> "$TEMPTEXT"
And when you schedule the cron job, you won't need to specify a list of hosts, just execute run-esx-report.sh:
10 7 * * * ${HOME}/esx-report/run-esx-report.sh >/dev/null 2>&1
If you make these changes, let me know if it works or not. Thanks!