Recently I launched Snospace which runs on a Media Temple dedicated server. At first it didn't really cross my mind to make backups. I'm used to managed hosting where they usually take care of this sort of stuff for you. I know it's highly important to backup your important files, but sometimes I forget... like eating vegetables.

I wanted a clear and simple solution that would backup my entire server, or at least the most vital information such as website source files and databases. For me it was really important to get these backups, without any human intervention to either an FTP account or my Amazon S3 account. Obviously in the event that something does go wrong, it should be easy to restore from a backup.

The dedicated server has Plesk installed, a fairly simple control panel to manage your server. Plesk does have scheduled backups but unfortunately it's not as stable as I would like. As Eric Nagel pointed out on his blog: "Plesk restorations SUCK. Plesk removes the current (live) site, then attempts the restore. Oh, and if the restore fails? Sorry - you have nothing now. Talk about back-asswards." Besides this major flaw, Plesk backups cannot be transferred automatically from Plesk to an FTP or Amazon S3 server. I'm sure this can be scripted but I'd rather not dive into this messy world of Plesk.

In the same blog post by Eric Nagel, he finally saved my day. He presented a script that would automatically backup everything, including databases to my Amazon S3 account. Perfect!

Indeed, the script ran perfectly. I set it up as a daily cronjob and saw that my Amazon S3 bucket was filling up with daily backup files.

However as the site grew, so did the backup files. At the same time my site had a major outage and I couldn't find out what happened. I restarted my server, checked all the logs and found out that this script timed out while trying to upload a backup file of 2.1 GB to my Amazon S3 account at 100% CPU usage. Oops!

Luckily or unfortunately (depending on how you look at it) I wasn't the only one with this problem. It seems like files larger than 2GB are not supported on 32 bit servers due to PHP's signed integer bug. And looking at the comments of the PHP class, timeouts with the Amazon S3 PHP class can occur, even with smaller files.

So this wasn't the way to go, for me at least. However I did think that Eric Nagel's script was a perfect start to getting automated backups to S3. I decided to build on his script and implement a couple features that would fix my problems, and hopefully yours too.

The improvements are:

  • Large files are split into separate smaller files. These files are uploaded as part files (.p01, .p02, etc.) with a maximum file size, you can change the maximum file size yourself. Default is 300MB.
  • Rewritten as a process. The jobs in the script have been split into separate smaller tasks.
  • Writes a temporary file to your working directory to prevent double executions of the same script to avoid data corruption and too much load. This means you can run it as a cronjob every minute. If the current task isn't finished, new tasks won't run until it's finished.
  • Auto exit after 1 hour, just in case the S3 times out. You can change this yourself ofcourse, but for most this will be enough.

The new version is fairly easy to setup. You can download it here. Just open it and change all the configuration variables. You can run it manually in your console by entering php /path/to/script/server_backup.php or set it up as a cronjob. I recommend running it every 2 minutes. Let it run as a user that has access to all directories you want to backup.

Restoring
Restoring is easy too. Just download the backup files from your S3 server. Upload them to your server and untar them. If you downloaded split files (.p01, .p02, etc.) then you can merge them again as one big file by running the command cat prefixoffile.* > largefile.tar.gz, then you can untar that file as well and restore your backup.

The script works for my server and is provided as-is. Please don't ask for support, always ask your system administrator for help first if you can't get it to work or you can figure it out by adjusting the script with trial & error. There are a lot of comments and debug messages provided in the script that can help you further. If not, you can also try to leave a comment below.

Good luck with your server backups!