Sep 2, 2010

Remotely Back-up Subversion Repository

It happens from time-to-time and for a variety of reasons that one might want to have a back-up of a Subversion repository. This includes the normal back-up reasons for disaster recovery, but could also provide a way to share repositories over slow connections, wanting a local copy of a remote repository, and probably more.

There are a few sure-fire ways to do it, but only one to do it without direct access to the original repository server, and (after initial creation of the repository) not even direct access to the back-up server. Well, you need HTTP access, but that’s all. We’re assuming proper access is allowed, and that neither server minds the occasional suck of the bandwidth.

To start, you need a Subversion repository on which to store the copy. Somewhere on this blog is a quickie how-to that will help get one configured. Once you have a server configured, you need to make a repository to receive the back-up; be sure that the server configuration allows exposing the repository path as necessary.

Unless you have some web-based tool to create the repository, you need to pop into the back-up “to” server (hereafter named “backup.server”) and create the repository (cleverly named “Repository”). Note my paths are for not-Windows, so if you’re on that OS, flip the slashes over. I’m also making some assumptions about the system configuration and permissions. If you’re not allowed to do this, of course, it will fail.

svnadmin create /path/to/svn/Repository

The command won’t return anything, but you should be able to look and see the new Repository folder in the /path/to/svn, or whever your subversion repository will live (in agreement with your Apache configuration). There’s one thing that Subversion doesn’t do by default, and that’s configure the hooks in a way that allow us to use the provided Subversion svnsync tool. In the new repository folder is a folder named “hooks.” By default in that folder are a number of templates. One in particular is of interest, the pre-revprop-change.tmpl file. The template file does some checking to ensure that a particular entity is limited to a particular action. The documentation notes that the file must exist and return zero for the hook to pass; you can just create an empty, executable script file named pre-revprop-change in that folder, and it works fine. Something very simple like #!/bin/sh works. Of course, it doesn’t do any protection (more on that), but gets us flying into the back-up.

After creating the repository and creating the pre-revprop-change script, direct access to the server is no longer required. Somewhere in the middle, either on the source server, destination server, or some other system in the middle with HTTP access to both and Subversion installed, we can kick off a pretty straight-forward svnsync command.. Here’s what it looks like, and following is a discussion of the bits we’re using.

svnsync init http://backup.server/Repository http://source.server/Repository --source-username=sourceUser --sync-username=syncUser

The “init” tells svnsync to configure the destination server so that it knows . It also does a little file manipulation on the system to create configurations to link the two together, hold usernames and passwords, and other Subversion magic. For the curious, watch the files in your home folder’s .subversion folder (again, on not-Windows) to see what changes. Slightly uncomfortable is that there are files there with passwords (assuming there are passwords associated with the servers) that are stored in clear-text; they are stored with owner-only permissions, but their presence can make some people squirm. Used as described above, svnsync will prompt for the passwords as required; add the parameters of –source-password and –sync-password to avoid being prompted (and have the passwords in your history…).

The first URL is the destination, or the back-up server Repository. Of course, the full and correct URL should be used.

The second URL is the source. Also here the full and correct URL should be used.

The source- and sync-username parameters (each preceded with double-hyphens) are optional. They’re used to provide the user to gain access where authentication is required. If authentication is required and these are not provided, the current username will be used. As noted in the comments, there are similar source- and sync-password parameters, too, to avoid being prompted if the server requires passwords.

If this is successful, a very simple response will confirm that the properties are configured.

Copied properties for revision 0.

Once done, trigger the copy command. This will be the same command to copy the first and each iteration in the middle. Issuing the command will bring the repository up to date, in full Subversion fashion, maintaining all of the history along the way. Since the init starts at revision 0, it may be some time if the repository is large or has a lot of revisions. While it’s working, there should be a cycle of “copying properties” and “transmitting data” and “committed version” messages as each revision is copied. When it reaches the current revision it will stop.

svnsync sync http://backup.server/RepositoryPath --source-username=sourceUser --sync-username=syncUser --sync-password=syncPassword

Here note that the sync-password parameter is provided. This line can be added to a cron job or other automatically run script to provide periodic updates to the repository. Of course, again, use the correct values for your system.

That’s the guts of it. Easy, four steps, and done. Make a repository. Prep the hooks. Run an init. Run a sync (repeat as necessary).

About that hook. Reading the svnsync documentation and the contents of the template file, many warnings abound about the trouble that happens if a sync’d-to repository is otherwise updated; and they’re true. Consider if a user uses the back-up repository and commits a change; they get to use the next revision number. The next sync will come along and try to use the same next revision number and fail because it already exists. For that simple reason it’s worth protecting with a more robust pre-revprop-change file. That’s a little outside the scope of this document, but it’s not much harder than changing the template file to use your sync username instead of the one in the file. Here’s a trivial example that will work with our example syncUser.

#!/usr/bin
if [ "$3" == "syncUser" ] ; then exit 0 ; fi
exit 1

Note this really only limits changing properties (and therefore revision numbers) to the one user, so if you’re going to do this, make that user be a non-user, one used only for the back-up.

About the Author

Object Partners profile.
Leave a Reply

Your email address will not be published.

Related Blog Posts
Natively Compiled Java on Google App Engine
Google App Engine is a platform-as-a-service product that is marketed as a way to get your applications into the cloud without necessarily knowing all of the infrastructure bits and pieces to do so. Google App […]
Building Better Data Visualization Experiences: Part 2 of 2
If you don't have a Ph.D. in data science, the raw data might be difficult to comprehend. This is where data visualization comes in.
Unleashing Feature Flags onto Kafka Consumers
Feature flags are a tool to strategically enable or disable functionality at runtime. They are often used to drive different user experiences but can also be useful in real-time data systems. In this post, we’ll […]
A security model for developers
Software security is more important than ever, but developing secure applications is more confusing than ever. TLS, mTLS, RBAC, SAML, OAUTH, OWASP, GDPR, SASL, RSA, JWT, cookie, attack vector, DDoS, firewall, VPN, security groups, exploit, […]