Smarter Backups

All posts, RSS Feed

A while ago we explained how we do backups for RPGpad.

In that earlier blog we explained that we want to be absolutely sure that your campaign data is safely backed up. To that end every single backup stores:

  • The complete history of all campaigns, account profiles, and private messages.
  • All images ever used anywhere in the system.
  • The finger print of the code RPGpad was running at that moment.

And that was the cure for all our worries. Turns out that it was also the cure for all our free storage space—our daily backup has grown from a thin 20 megabytes to a heavy 323 megabyte affair, consuming roughly 1gb of storage for every three days of backups.

To investigate what was going on, we sketched out a graph of the backup size over time (this is a cleaned up sketch to show the trend):

Backup size graph

The graph shows the relative total size of the backups over time with the following stacked parts: Image data, All other data.

As it turns out this unexpected explosion of backup sizes is due to the popularity of the gallery feature among some of the players. Of the 323 megabyte, 290 megabyte consist of gallery images, while the other 33 megabyte is the complete history of all other things.

This week we reworked the backup system to be smarter about how it backs up the different things. Since image data is never removed or changed because old versions of pages and character sheets still refer to the old images, our backup system now maintains a “mirror” of all images we have ever seen.

We can safely do this because previously each backup contained “All images ever used anywhere”—and this means that newer backups also contain all images required for any older backup. So now we only maintain a single copy of “All images ever used anywhere” that is neatly backed up.

We have carefully tested the system to make sure that the backups are complete and that we can recover all data from the backups. After all tests turned out positive, we switched over completely to the new backup system. This means we can keep on keeping your campaign data safe without having to worry about storage space for the foreseeable future!

This week we have a lot of changes that are, as always, detailed in this week’s changelog!