The new URL for Rollout is: http://code.google.com/p/rollout.

Unfortunately they don’t support git, so I have to deal once again with Subversion.

The new checkout instructions are:
svn checkout http://rollout.googlecode.com/svn/trunk/ rollout

The manual is online there, and an issue tracker, so submit bugs!

You can download it here. You will need to edit a few configuration items at the top of the script, to include your Amazon Web Services IDs and MySQL details. You will also need the Amazon::S3 and DBI perl modules. You can get them from CPAN.

$ wget -Omysql_s3.pl http://dparrish.com/files/mysql_s3.pl
$ vi mysql_s3.pl (enter your details)
$ chmod 755 mysql_s3.pl
$ ./mysql_s3.pl

It’s probably best to add it to cron so it will run daily. Something like this works nicely:

0 3 * * * perl /home/dparrish/mysql_s3.pl

I’d also suggest using S3 Firefox Organizer to manage your S3 account. It’s a Firefox plugin.

You can get the 1.0 stable release at http://dparrish.com/files/rollout-1.0.tar.gz.

Although this is stable and shouldn’t destroy any of your systems, it does still (and will always) require configuration before it will work for you.

As always, the latest version is available by git:

git clone http://git.dparrish.com/rollout

Comments / bug reports / flames are more than welcome.

Tags: rollout

I tracked it down to the gzip encoding module, which waits for all the data to be output, compresses it and sends it on to the browser.

You can disable gzip encoding for a particular directory in a .htaccess file (with Apache) like this:

RemoveOutputFilter DEFLATE
SetEnv no-gzip

So here, in it’s entirety: the Rollout Manual
(more…)

Rollout is a system developed to automate system administration on UNIX servers. It is primarily focused towards Linux, but could be adapted to Solaris, HP-UX, AIX, etc.

It is written purely in Perl, and the configuration is also a Perl source file. Some Perl knowledge is required to edit the configuration, but copy-and-paste may suffice.

System Administration can generally be reduced to a set of steps, which must be completed in order. Some example steps include:

* Copy files, create symlinks
* Edit configuration files
* Start / stop services
* Add / modify users & groups
* Install applications

The idea behind Rollout is to automate all these steps in a configurable way. Rollout is by design idempotent - it can be run many times on a single server and produce the same results.

There is a certain amount of overlap between this and other projects such as puppet and cfengine. They all have similar goals. Rollout is designed to be very easy to extend (as long as you can cope with Perl) and very fast.

You can get rollout from my git repository at http://git.dparrish.com/rollout. If you’re using Cogito, you can run:

cg clone http://git.dparrish.com/rollout

This is the first public release, and as such is not expected to be everything to everybody. However it is in active production use and hasn’t destroyed anything yet. Use it at your own risk. The documentation included contains instructions on setting up a server and configuring clients.

Also, I’m enabling comments on this post (and I feel dirty doing it) so that people can submit bugs/patches/requests/complaints.

rename 's/^([a-z./]+).(d+).(d+).(d+-d+-d+-d+-d+-d+).(meta|data)$/
      sprintf("%s.%03d.%03d.%s.%s", $1, $2, $3, $4, $5)/ex' */*

This should be run as all one line. It requires the rename perl script, which is in the perl package on Debian/Ubuntu at least. If you don’t have it, grab it here.

You can download the new version at:

Along with this release, I’m also opening up my git repository, so you can grab the latest source yourself. If you’re using Cogito, which I highly recommend, you can run:

cg clone http://git.dparrish.com/dbackup

For instructions on using dbackup, see my original post.

The single reason I don’t want a blog is that I hate them. People generally don’t have anything interesting to say, and I generally don’t want to read what they do have to say. There’s enough junk on Slashdot, Fark and Digg to keep me amused for the rest of my life.

So, the reason I have a blog now is so I can have my own useful OpenID for use on other supporting sites. OpenID is cool.

Comments will be forever disabled on my homepage. If you want to comment on anything I say, do so on your own blog and send a TrackBack ping.

It works on the principal that disks are cheaper and more reliable than tapes.

Backups are started by cron probably on a daily basis by the client. The client backs up individual filesystems / directories with tar and sends the result to the server, which stores them in a simple tree-based directory structure.

Restores are trivial, either by using the supplied restore client, or by simply copying the appropriate tar files off the server and uncompressing them.

Features

• Backup very large filesystems as long as you have the disk space
• Works with very large numbers of servers / filesystems
• No state is kept on the client, it’s all on the server
• Backup to multiple backup servers in round-robin
• Configuration can be global or per-server
• Authentication for client restores (optional)
• Supports optional gzip or bzip2 compression
• Supports client-side gpg encryption
• Trivial procedure for restores
• Daily / Full incremental backups

(more…)