Here we’ll try to get a Hudson Continuous Integration server building from a monitored Subversion directory by adding these softwares and their Apache HTTP and Tomcat hosts.

Assumptions

Much of this discussion assumes you’re working with Linux (particularly assumed is that you’re running Ubuntu or other Debian-based distributions), but with a little care it can be used for Windows, Mac, Solaris, or other Linux distributions. The particular differences will most likely be around the installation paths and methods used to add prerequisites (e.g., what you do instead of “apt-get” or using Synaptic).

Additionally, it is assumed that the system on which this is being attempted has at least the basic software building tools. An installation of Ubuntu (the test system used to document this) comes with at least the basic C compiler, headers, and libraries necessary to do this. The exceptions encountered are noted, and should be enough to indicate where trouble might be.

Apache Serving Subversion

For the moment we’re assuming that an Apache web server is not installed, or that it’s acceptable to replace what is installed with one that has the required modules to make Subversion happy.

Get the Apache HTTP (web) server source right from the Apache HTTPD site. I recommend getting the latest production-ready source, which as of this writing is 2.2.14. Download the UNIX source, expand it (anywhere, really; It’ll be safe to deleting it later), with the likes of “tar -xzf httpd-2.2.14.tar.gz” and it’ll by default create a folder with the version name, in this case, httpd-2.2.14 in the current directory.

Change to that expanded directory and run “./configure –help” to get a list of the things that can be added. Use this list to double-check the names of the plug-ins and such in the following sections, adjusting where they’ve made changes. These changes are necessary for Subversion only, so if other things are needed for your installation, feel free to add them as well.

Ensure the option for mod_dav is enabled by adding “–enable-dav” (or whatever the –help exposed) to the ./configure command, such as “./configure –enable-dav” as this is one of the requirements for Subversion to work through the Apache server.

Another requirement is mod_ssl, which is likewise enabled by adding “–enable-ssl” to the ./configure command. This requires that an SSL package (and its headers) be installed. A good one is the OpenSSL. Using the likes of Ubuntu, just add the OpenSSL and libopenssl-dev packages using the Synaptic Package manager before trying to make. Installing OpenSSL from scratch is a little more involved than this document has room for, but it’s the basic “./configure && make && sudo make install” deal, if you do it from source.

By default, Apache will install to /usr/local. A last handy option to know (and look for in the –help output) is to change the output directory, using the “–prefix=(dir)” option, as in “–prefix=/opt/apache” or “–prefix=~/apache”. One good reason to change the install path is to avoid conflicting a pre-existing installation. Another is if you don’t have (or want to use) super-user rights on the system. A final good one is to have several Apache versions at the same time, perhaps for different development needs.

When all of the desired options are decided, run the configure command with the options, which will at a minimum be “./configure –enable-dav –enable-ssl” to create the Makefile. Run “make” in that directory to be sure everything builds. Running make as your user instead of the super user will create all of the files with your permissions, making clean-up and re-running easier.

Run “sudo make install” (or skip the sudo if you’ve chosen a folder to which your user has permission) to copy the files into place.

Go to the /usr/local/apache2/bin folder (or the bin directory inside the one in which you installed Apache), verify installation by running “./apachectl -l” and make sure mod_dav.c, mod_ssl.c, and http_core.c are in the list. While we didn’t specify HTTP as an option, it is on by default; it’s assumed you want HTTP in the HTTPD server, so it does that for you.

Adding Subversion

Subversion will require the Apache be configured with the minimum discussed above. It will also require a XML parsing library, such as expat or libxml is installed and available. You can get the libxml from source at xmlsoft.org, or if you prefer using a package manager, find and add the “expat” and “libexpat-dev” (for example by running sudo apt-get install expat libexpat-dev from a shell prompt) packages before configuring Subversion. Again, the building of these from scratch is outside of our scope today.

Download both Subversion Source and Subversion Dependencies from subversion.tigris.org, currently version 1.6.6, and expand them both into the same folder (which happens by default if you un-tar them in the same folder), and go to the directory (e.g. subversion-1.6.6).

Checking in that expanded folder, double-check the INSTALL file (with the likes of “more INSTALL”) and find the ./configure command suggestion which would look something like “./configure –with-apr=/usr/local/apache2 –with-apr-util=/usr/local/apache2” so we can be sure we have the right APR parameter names. We’ll be using the path of the Apache server, configured above.

Also, you can run “./configure –help” to see other options that might be helpful, again such as “–prefix=dir” (for the same reasons as above) to put the subversion in a different directory than the default, which is /usr/local.

Run the ./configure command, changing the apr paths to match the Apache installation path from above (if you changed it with the –prefix= option). Watch for errors and peek at the INSTALL folder for tips. If the Apache is configured as described and an XML library is found, there should be no errors.

Run “make” to build the files, and “sudo make install” to add the Subversion programs and libraries to the system, and the modules to Apache. Again, if not installing to a location requiring root permissions, the sudo is not required.

Quickie Repository

Quickly make a Subversion repository so that Apache has something to show us for our efforts. Decide where the repository should go, and run the command “svnadmin create /var/svn/testrepository” using the correct path and permissions. Note if you chose to install Subversion in a different directory than the default, you may need to specify the full path to the executable, such as ~/subversion/bin/svnadmin or add the bin folder to your path. You also may have to create the path and correct the permissions (e.g., “mkdir /var/svn && chmod 777 /var/svn”) before executing the command. The svnadmin program will silently succeed, so you can check for the existence of the repository by double-checking “ls /var/svn/testrepository” to be sure it exists with all kinds of subversion goodness in it.

The permissions need to be set to allow access for the user under which your Apache runs. This user will need full access. A few different ways to do this include changing the owner or group on the folder, or as the example above does, simply allow everyone in there. Securing the Subversion through Apache is a whole other topic to be handled a whole other day.

Joining Subversion and Apache

A slight tweak to the Apache configuration needs to be done so that Apache knows to serve the subversion repository. Edit (as super-user if necessary) the Apache conf/httpd.conf file and add the following block, correcting the path to match the one above.

<Location /svn>
    DAV svn
    SVNParentPath /var/svn
</Location>
<Directory "/var/svn">
    Order allow,deny
    Allow from all
</Directory>

Using the SVNParentPath will allow this single configuration to host all of the repositories created in the /var/svn folder. Whenever a repository is created in that same folder, it will automatically appear in the Apache server without further configuration, nor will a restart be required. If you use the SVNParentPath, however, note that you can’t visit /svn (or whatever you named it)

One quick note: if you’re not going to be starting Apache as root (e.g., on a system you don’t have sudo access), as long as you’re in the httpd.conf file, change the port from the default 80 to something bigger than 1024, and remember to use that port whenever accessing this server. And if you’re going that far, you can avoid the “can’t find the server name” warning by finding and setting the ServerName in the configuration file.

Run “/usr/local/apache2/bin/apachectl -t” (again, correct for the path used to install above) to check the syntax of the updated Apache configuration see if there are any errors. (An error I encountered meant I had to edit /usr/local/apache2/bin/envvars and add /usr/local/lib in the LD_LIBRARY_PATH because it was giving me an error “httpd: Syntax error on line 53 of /usr/local/apache2/conf/httpd.conf: Cannot load /usr/local/apache2/modules/mod_dav_svn.so into server: libsvn_repos-1.so.0: cannot open shared object file: No such file or directory.”)

If all is good, run “sudo /usr/local/apache2/bin/apachectl start” to fire up Apache. Fire up a web browser and visit http://localhost/ to get a brief message that should tell you that it worked. Again, skip the sudo if you don’t need it, and add the port if you changed it, of course.

Using a web browser to visit http://localhost/svn/testrepository (using the Location noted in the Apache config, and the name of the repository from the svnadmin command) should show your repository (finally!). Initially it might just indicate the revision number as zero, and show no files; but that’s right as we didn’t add anything to the repository yet.

Installing Tomcat

Tomcat requires that a Java runtime be installed and available to the system. This is almost assumed these days, but if in doubt, visit java.sun.com and get a JRE or JDK and install it. Note that previously Tomcat required a JDK be installed, but now it uses its own compiler for JSPs, and so a JDK is not required. However, since we’re using this as a tutorial for preparing for use with Hudson, get the JDK to be sure that whatever Hudson tries to build will work.

Visit the Apache Tomcat website at tomcat.apache.org and download the latest version. Expand the downloaded file (untar or unzip as appropriate for what you downloaded). Done. There’s nothing that needs to be done to install Tomcat.

Depending on where you want Tomcat to live, you may need to do the expanding and other bits as the super-user. Common places to put the Tomcat distribution are /usr/local (requires super-user) and /opt (usually doesn’t require super-user). There’s no reason this couldn’t work from your user’s home directory, either.

Additional configuration is probably a good idea (such as changing/adding the manager name and password in the tomcat-users.xml, changing the path in the server.xml, etc), but nothing more is required to make the rest of this work.

A quick peek at the conf/server.xml file and we can see that by default the WAR files can be added to the webapps folder and that they will automatically be deployed.

Start the server now (or after copying the hudson.war file in the next section) by simply executing the bin/startup.sh script. Visiting http://localhost:8080/ should bring up a confirmation page that Tomcat is running.

Finally, Hudson

Get Hudson from hudson-ci.org and download the latest and greatest hudson.war. Put the war file under Tomcat webapp directory. Simply copy the file. If Tomcat is running, after a moment Tomcat will see it and expand and deploy it.

Start the Tomcat server (if not already running) and go to http://localhost:8080/hudson/ and the Hudson interface (or “loading, please wait” message) will let you know it’s been installed.

Hudson Job

The final piece of the puzzle is to put a project into Subversion for Hudson to find and act on. Let’s first create a simple “hello world” project, with an Ant build script, commit it to Subversion, and make a job for Hudson to run.

A few assumptions will be stated here (to avoid all of the potential asides, like this one). First is that you’ve installed things to the default or as mentioned above. That is, the HTTPD is running on port 80 and Tomcat on 8080, which are their defaults. Also that you’ve chosen to use the “parent” view for subversion as noted above, so we can just create repositories on a whim. We’re also assuming everything is running on one machine, so the URLs will be localhost, which you will have to change if you’re doing this on separate machines.

Make a folder for the project (e.g., mkdir hudsonProject). This will become the root of our project.

In that folder, create a source folder (src) and in there create a trivial class to print “hello world” or some other such simple stuff. I’ll leave out the Javaness (e.g., “name the file HelloWorld.java” and so on, as I’m assuming you’re that far along.

class HelloWorld {
    public static void main(String[] args) {
        System.out.println("Hello, world.");
    }
}

Also create a simple Ant build script (or Maven if you prefer, but I’m documenting Ant…). Put this in the root of our project, and name it, of course, build.xml.

<project name="HelloWorld" default="compile" basedir=".">
  <target name="compile">
    <mkdir dir="build"/>
    <javac srcdir="src" destdir="build"/>
  </target>
</project>

This is about as simple of a build.xml as one can have; it will only compile our Java source files found in the src folder into the build folder. In the end we should have the .java file we created, and the .class file the Ant script creates. We haven’t installed Ant yet, so unless you’ve taken that initiative, you can’t test the script. That’ll happen in a bit and we’ll use Hudson to do our debugging for that…gives us something to play with.

Now we need to introduce this project to Subversion. The following command will prepare everything for our initial project creation. We could have created the project empty, without any files at all, which makes a fine “revison 0” but I’ll leave discussions of repository management theory for another post.

svnadmin create /var/svn/helloWorld
svn import . http://localhost/svn/helloWorld -m "Initial import"
svn list http://localhost/svn/helloWorld

The first line will create the repository in our parent folder (unless you want to use the one created before…then just change the name as necessary). The second line will copy the contents of our current directory (the dot) to the repository through our web server. The final line will show us what’s in the repository, which should mimic our trivial src/HelloWorld.java and build.xml structure. We can verify the repository exists and has our files by visiting http://localhost/svn/helloWorld with the web browser and the same stuff should be there.

Now we’ve got our project in Subversion; let’s create a Hudson job to work on it. Fire up a web browser and visit http://localhost:8080/hudson again. By now it should be running and greet you with a “create jobs” message. We’ve got a little housekeeping to do to make it work with our install.

On the right side find and click the “manage Hudson” link. At the top of the list in the middle, click the “configure system” link. Near the middle, find and click the “add jdk” button. De-select the “install automatically” if it’s selected, and put the path to the JDK installed above (e.g. /home/youruser/jdk1.6.0_17 if you unpacked it in your home folder), and give it a name.

Since we’re doing an Ant project, we need Ant “installed” and configured. The latest versions of Hudson look like they’ll try to install it for us, but I can’t get it to work, so make a quick trip to ant.apache.org and download the latest (currently 1.7.1). Simply extract that file somewhere. Back in our Hudson configuration, click the “add Ant” button, de-select the “install automatically” box, if it’s selected, and put the path to the Ant as unpacked (e.g., /home/youruser/apache-ant-1.7.1) and give it a name.

Hit the save button at the bottom of the form.

Bonk the “new job” link and start the process. Give the job a name (something like “helloWorld” — it isn’t really related to the poject) and select “build freestyle” to use the Ant job, and bonk the “ok” button. Much of the busy next form is optional, and we’ll leave Hudson management to another article, focusing now on the required fields to make this work.

Find and select Subversion. This will spring open the configuration bits necessary for our job. Put in the URL to the repository (in our example, http://localhost/svn/helloWorld).

Find the “add build step” drop down and select Ant from the list, then select the Ant installation by the name used when we added the configuration above (the “default” never seems to work, even if only one Ant is installed). By default it will use the build.xml in the root of the project, and the project’s default target. For ours, recall, the simple script has only one target, and it is the default. Nothing more is required.

Bonk the save button at the bottom of the form. The job will be saved. The Hudson view will now show the current state of the helloWorld job, which is pretty blank. On the left side is a “build now” link. Bonk that to trigger Hudson to do an SVN check-out and run the Ant script.

A few moments will pass and some activity will occur in the “build history” area on the left side. If we’ve done all of our configuration correctly, and there are no errors in our Java or Ant files, we should be rewarded with a successful build. Click on the build version (by date and time) from the “build history” area and we can see what happened to the build. For the most information, given our simple job configuration, check the console output (on the left) to see what Hudson did.

Had our source and build script had more, like unit tests and build a WAR or JAR, or if there are errors in the source or build script, Hudson would show us all that went right or wrong.

Remaining Work

Subversion and Hudson together with their appropriate Apache tools is a powerful and easy-to-use suite of version control and continuous integration. Brushing over as we did to get a skeleton working, the projects are exposed to the world. Both of the Apache servers, Subversion, and Hudson all have security to help control and restrict access to the servers, repositories, and jobs. Hudson has plug-ins for monitoring unit tests and code coverage and can put notices on Twitter and events to Google Calendar…

Much more than there’s time for here…but, heck, this is already five pages long…