Beyond Linux cloning, what's in it for DR?

Steve Carl and I presented ideas around this theme a few years back at SHARE. Steve mentioned this in his blog a while back. The presentation was on cloning Linux on Z Series mainframes. It had occurred to me that the cloning of Linuxes, making copies, was basically the same thing as backup and restore. Except the idea had several important differences. One critical difference was that backups could be saved as compressed tarfiles on hierarchical cheaper storage like NFS servers than keeping them resident on mainframe storage. Here in R&D Support we have built some very reliable (nearly a year of uptime so far), low cost (less than $1000 USD / Terabyte) NAS servers based on commodity hardware and Linux. These are perfect for holding the tarfiles. Also with replication across a wan of these smaller compressed files, it was possible to implement a disaster recovery at an alternate site.

Just like in cloning, if you were going to restore it to another virtual machine, you needed to edit the restored copy for any IP/hostname differences, etc.

We had scripts that could do that. However, it soon occurred to me what I wanted was a web portal that could help facilitate this process for our R&D user community. The web interface would allow R&D developers and analysts to manage any assigned VMs they had by restoring any Linux image from a standard catalog of backups. In fact, after they had restored an image, they could add applications or modify it any way they liked and backup the image to their user catalog of backups. These backups could be shared with other users or become checkpoints for their own processes if they should need to fall back to a good image.

So I wrote the application using HTML form files and PHP scripts on a z series Linux to allow this to happen. With the help of Steve's ideas, we called it VMLMAT, Virtual Machine Linux Management and Archival Tool. It authenticates the user by using open source tools like SAMBA server, smbclient, and WINBIND service to verify the login credentials against a Windows Active Directory. After that, the tool provides the above mentioned services with an ownership model of the user's virtual machines. Admin owners of their virtual machine may grant privileges to other users to just bring up/down their machine, or to also backup/restore to their machine, or to have admin privileges as well.

VMLMAT incorporated DR by allowing automatic backups on schedules that CRON can handle and the application numbers the backup for generation support and automatic deletion of older backups. With behind the scenes replication and installing this portal at an alternative site, the user can point their browser to that URL and restore that image to a VM located there. Since the tool automatically changes IP/Hostname, you can test the DR while your original VM is still running.

Of course there are limitations to this. The restored VM must have the same configuration of volumes that the original had. Also, any support structure like external fileservers, DNS, etc must be in place to support your alternative site.

There are many opportunities for both companies and users to implement open source tools into other tools that facilitate the management of IT environment. Many years ago, a distributed systems administrator once told me while I was learning that environment that repetitive tasks are best handled by scripts and any good admin should know how to write them. It reminded me of the mainframe CLISTs (command lists) or REXX execs we wrote that also facilitated our admin duties as well. I believe nothing is new just the need to further automate our processes.

The proof is that VMLMAT has returned a great deal of time to my day. Where I used to barely be able to keep up with all the requests R&D had for new Linux Virtual Machines on the mainframe, now I not only can keep up, but have had thirty hours or more returned to my average work week. When I needed to re IP address all the Linux images on the mainframe as part of a larger project, I was able to take what I had done in VMLMAT and slightly modify it to automate this process too. Again I got many hours of otherwise repetitive work back.  VMLMAT is open source, free, and available here: http://vmlmat.sourceforge.net 

Keep in touch....