Dear old ESXTOP aka How to schedule ESXTOP batch mode

Recently I had to record a night activity of a specific VM running on a specific host for troubleshooting reasons because vCenter data wasn’t just enough for that.

Using a number of blog posts around from Duncan Epping and others (it was many, I don’t even have the links anymore) I’ve put up my personal guide about how to take over this task because every time it’s like I have to start from scratch so I decided to document it.

First thing I created a script with the specific run time and collection data I needed:

vi <path>/record-esxtop.sh
esxtop -b -a -d 2 -n 3600 > /esxtopoutput.csv

OR

esxtop -b -a -d 2 -n 3600 | gzip -9c > /esxtopoutput.csv.gz

(-d=sampling rate, -n=number of iterations; the total run time is “d*n” in seconds)
(second version creates a zipped version of the output)

Let’s make this script executable:

chmod +x <path>/record-esxtop.sh

Then, since in latest versions of ESXi there is no crontab, you’ll need to edit the cron file for the user you want to run the script with:

vi /var/spool/cron/crontabs/root

Then add a line similar to this:

30 4 * * * <path>/record-esxtop.sh

Now kill crond and reload:

cat /var/run/crond.pid
ps | grep 
kill -HUP 
ps | grep 
/usr/lib/vmware/busybox/bin/busybox crond
cat /var/run/crond.pid
ps | grep 

Now your script will get executed and you’ll find a file with your data, but how to read it?
It’s dead simple, just open PerfMon on Windows, clear all running counters then right-click on “Performance Monitor” and in the tab “Sources” add your CSV file (need to unpack it first); in the data tab you will then be able to choose metrics and VMs you want to add to your graph.

It would be nice to have a tool that does the same on Mac but I couldn’t find one and I had to use a Windows VM; if you know a Mac alternative for PerfMon please add a comment.

This procedure is supported by VMware as per KB 103346.

Advertisements

How to backup, restore and schedule vCenter Server Appliance vPostgres Database

Now that we are moving away from SQL Express in favor of vPostgres for vCenter simple install on Windows and since vPostgres is the default database engine for (not so simple) install of vCSA I thought it would be nice to learn how to backup and restore this database.

Since it’s easier to perform these tasks on Windows and since there are already many guides on the Internet I will focus on vCSA because I think that more and more production environment (small and big) will be using vCSA since now it’s just as functional as vCenter if not more. (more on this in another post…)

You will find all instructions for both Windows and vCSA versions of vCenter on KB2091961, but more important than that you will find there also the python scripts that will work all the magic for you so grab the “linux_backup_restore.zip” file and copy it to the vCSA:

scp linux_backup_restore.zip root@<vcenter>:/tmp

For the copy to work you must have previously changed the shell configuration for the root user in “/etc/passwd” from “/bin/appliancesh” to “/bin/bash”

Then:

unzip linux_backup_restore.zip
chmod +x backup_lin.py
mkdir /tmp/linux_backup_restore/backups
python /tmp/linux_backup_restore/backup_lin.py -f /tmp/linux_backup_restore/backups/VCDB.bak

All you will see when the backup is completed is:

Backup completed successfully.

You should see the backup file now:

vcenter:/tmp/linux_backup_restore/backups # ls -lha
total 912K
drwx------ 2 root root 4.0K Jun 3 19:41 .
drwx------ 3 root root 4.0K Jun 3 19:28 ..
-rw------- 1 root root 898K Jun 3 19:29 VCDB.bak

At this point I removed a folder in my vCenter VM and Templates view, then I logged off the vSphere WebClient and started a restore:

service vmware-vpxd stop
service vmware-vdcs stop
python /tmp/linux_backup_restore/restore_lin.py -f /tmp/linux_backup_restore/backups/VCDB.bak
service vmware-vpxd start
service vmware-vdcs start

I logged back in the WebClient and my folder was back, so mission accomplished.

Now how do I schedule this thing? Using the good old crontab but before that I will write a script that will run the backup and also give a name to the backup file corresponding to the weekday so I can have a rotation of 7 days:

#!/bin/bash
_dow="$(date +'%A')"
_bak="VCDB_${_dow}.bak"
python /tmp/linux_backup_restore/backup_lin.py -f /tmp/linux_backup_restore/backups/${_bak}

I saved it as “backup_vcdb” and made it executable with “chmod +x backup_vcdb”.

Now to schedule it just run “crontab -e” and enter a single line just like this:

0 23 * * * python /tmp/linux_backup_restore/backup_vcdb

This basically means that the system will execute the script every day of every week of every year at 11pm.

After the crontab job runs you should see a new backup with a name of this sort:

vcenter:/tmp/linux_backup_restore/backups # ls -lha
total 1.8M
drwx------ 2 root root 4.0K Jun 3 19:46 .
drwx------ 3 root root 4.0K Jun 3 19:28 ..
-rw------- 1 root root 898K Jun 3 19:29 VCDB.bak
-rw------- 1 root root 900K Jun 3 19:46 VCDB_Wednesday.bak

You will also have the log files of these backups in “/var/mail/root”.

Enjoy your new backup routine 🙂

%d bloggers like this: