I finally got my act together a couple of weeks ago and set up a
Jenkins server so we could start auto-deploying our applications to staging servers. Since we're doing agile/scrum on our projects now the product owners tend to like to see changes as they happen and we also have dedicated testers involved with some of our projects, so automating deployment to a staging server is saving us a lot of time and headaches.
We run all our CFML applications on
OpenBD and
Tomcat and for the most part deploy applications as self-contained WAR files, so the deployment steps in this environment are:
- Get latest code from Subversion trunk (we use trunk for ongoing development and branch for releases)
- Create WAR file
- Transer WAR file to target server for deployment
Pretty simple. I should note at this point that I will not be covering incorporating unit tests into the build/deploy process both because I want to focus only on the Jenkins stuff for this post, as well as because that aspect of things is covered quite well elsewhere. (And I'll be honest: we aren't yet doing unit testing consistently enough in our code that it can be part of our build process, but we're working towards that.)
I also won't cover installing Jenkins since there are many resources on that as well. In my case on Ubuntu Server it was a simple matter of adding Jenkins to sources.list, doing sudo apt-get install jenkins, and then doing a bit of Apache configuration to get up and running. You can
read more about installing Jenkins on Ubuntu here, and if you have specific questions about that aspect of things I can answer I'm happy to try.
Step 1: Create an Ant Build Script
As for the specifics of setting this up the first step is to create an Ant script to tell Jenkins what to do when the job runs (we'll create the Jenkins job in a bit). This is key because without a build script Jenkins doesn't really do much, so we'll create a build.xml in the root of our project and then when we create the Jenkins job we can tell it which target from the build script to run.
Since Jenkins "knows" about Subversion you do not have to include anything in your build script to pull the code from Subversion. So given that our applications get deployed as self-contained WAR files, all our Ant script has to do is build the WAR file from the code Jenkins pulls from Subversion.
I should clarify that even though I'm explaining the Ant script first, Jenkins actually runs the build script after it pulls code from Subversion. I'm only pointing that out so you don't think the Ant script runs first even though I'm covering it first. Since you can specify an Ant target when you create the Jenkins job I figured I better explain that first.
Here's a sample build script.
The script isn't nearly as daunting as it may look so let's walk through it.
In the properties section at the top, that's declaring variables we'll use later so we don't have to hard-code those values in multiple places in the script.
The next section is the targets and these are specific tasks that can be executed. Note that these targets may have dependencies, so if you execute a target that has dependencies the dependencies will run first in the order they are declared, and then the target you specified will run.
In the case of this script we have three targets: build, war, and init. The war target depends on build, and build depends on init, so when we specify 'war' as our target in Jenkins later that means init will run, then build, then war, so let's look at these in order.
The init target at the bottom does basic cleanup by deleting the directories into which the source code is dumped and where the WAR file is built so we start with a clean slate.
The build target runs two copy jobs to get the application files into the build directory, which is just a directory to temporarily hold the files that will be included in the WAR. First the build target copies all the non-image files into the build directory, and then it copies all the image files into the build directory.
The reason for doing this in two steps is if you copy plain text and image files in the same copy job, the image files become corrupted in the process. As you can see the first copy operation excludes image files and the second includes only the image files as identified by file extension in the imageFiles property declared at the top of the script. If you have other binary files in your applications that may become corrupted (note that JAR files seem unaffected by this issue ...) you'll want to add those file extensions to the property that indicates which files in your application are binary.
Also note that I'm excluding the build and dist directories, the build.xml file, the .project file that Eclipse adds to projects, and all the .svn files so those aren't included in the build.
So at this point after init runs we have clean directories for doing our build, and then the build target copies all the files (other than those being excluded) from the root of the project into the build directory.
The last step is to create the WAR file, and this is (not surprisingly) done in the war target in the build script. Since Ant knows how to build WAR files this is pretty simple; you just point the war command to the directory where the application files are located (the build directory in this case) and tell it the target name and location of the WAR file, which we're putting into a dist directory.
To review, what we'll tell Jenkins to do in a minute is to run the war target (which in turn is dependent upon the init and build targets) in our build script, which will:
- Run the init target which deletes and recreates the build and dist directories so we start with a clean slate
- Run the build target which copies all the code and binary files from the root to the build directory
- Run the war target which creates a WAR file from the code in the build directory and puts it in the dist directory
Once you have your Ant script created, save it as build.xml in the root of your project and commit that to SVN so Jenkins will have it available when it runs the build.
Step 2: Create a Jenkins Job
With the hard part out of the way, next you'll need to create a job in Jenkins by clicking on "New Job" in the top left of the Dashboard.
Give the job a name, select "Build a free-style software project" and click "OK."
Step 3: Point Jenkins to Your SVN Repository
On the next screen you can configure some additional settings like whether or not to keep previous builds, if there are any build parameters you need to specify, etc., but we'll focus on the SVN configuration for the purposes of this post.
Select Subversion under Source Code Management and enter the details about your repository. This tells Jenkins where it's going to get the code to do the build.
Be sure to give Jenkins the full path to the appropriate directory in Subversion. For example if you build from trunk and your project name is foo, your URL would be something like http://subversionserver/foo/trunk not just http://subversionserver/foo
As a reminder, since we deploy our CFML applications as WAR files using OpenBD, our SVN repository includes not only our application's source code but also the OpenBD engine, so this is traditional Java web application deployment. This is a
great way to do things because the application is truly self-contained and all the configuration such as datasources, mappings, etc. is all in SVN. This way you can pull down the project from SVN and be up and running instantly, and it makes deployment really simple.
Step 4: Set the Jenkins Build Trigger
At this point Jenkins knows where your code is but we need to tell Jenkins what triggers the build process to run. You can do this multiple ways but in my case I simply set up Jenkins to poll SVN every 5 minutes to check for new code. Another common way to do this is to use a post-commit hook in SVN to hit a Jenkins URL that triggers the build, but polling is working well for us.
Scroll down to the Build Trigger section of the configuration screen.
Check the box next to "Poll SCM," and then you can set the polling schedule using crontab style notation. Mouse over the ? next to the box if you need a refresher on the syntax, but in the example I have here that syntax tells Jenkins to poll SVN every five minutes to see if there are any changes in SVN. If there are changes the build will be triggered. We'll review what happens when the build is triggered at the end of this post.
Step 5: Set the Ant Build Target
Just a couple more steps in configuring the Jenkins job. Next we need to tell Jenkins which target in build.xml to run as part of the build. Calling build.xml is kind of an implied step with Jenkins since you don't have to explicitly tell it to look for build.xml. It's assumed you'll have an Ant script in the root of your project and that either the default target or a specific target will be run as part of the build process.
In the Build section of the configuration page, specify 'war' as the target to run from your build.xml file in the root of your project.
At this point Jenkins will:
- Poll SVN every 5 minutes to check for changes
- If there are changes, Jenkins will pull everything from SVN
- After everything is pulled from SVN, Jenkins will execute the war target from build.xml which will generate a WAR file that can be deployed to Tomcat (or any servlet container)
The last step is getting the generated WAR file to a target server.
Step 6: Configure the Post-Build Action to Deploy the WAR
One of the great things about Jenkins is the huge number of plugins available, and we'll be using the
SCP plugin in this final step. There are also deployment plugins for various servlet containers but since in the case of Tomcat that involves hitting the Tomcat manager and uploading the WAR over HTTP, I found SCP to be much more efficient and flexible.
After you install the SCP Plugin you need to go to "Manage Jenkins" and then "Configure System" to configure your SCP target, user name, and password. These are configured globally in Jenkins and then you simply select from a dropdown in the post-build action section of the Jenkins project configuration.
In the post-build action section of the project configuration:
- Check the box next to "Publish artifacts to SCP repository"
- Select the appropriate SCP site in the dropdown
- Specify the artifact to copy to the server. In our case this is the WAR file, and you specify the path and filename relative to the root of the Jenkins project. For example if you check things out from an SVN directory called 'trunk' and use the same directories in the Ant script above, your WAR file will be in trunk/dist/foo.war
- Specify the destination relative to the path you specified when you set up the SCP server, if necessary. If you specified Tomcat's webapps directory as the root in the SCP server and all your projects live in that directory you may not need to specify anything here.
One more configuration issue to note -- in the case of Tomcat you need to make sure and have the host for your application configured to auto-expand WARs and auto-deploy. This way when the WAR copy is complete Tomcat will deploy the new version of the application.
Summary
As with most things of this nature it took a lot longer to write this blog post than it will to set this stuff up. The only even remotely involved portion of all of this may be tweaking the Ant script to meet your needs but the rest of the process is pretty straight-forward.
At the end of all of this we wind up with a Jenkins job that polls SVN for changes every 5 minutes and if there are changes, this triggers the build. The build process:
- Pulls down changes from SVN
- Runs the war target in the build.xml in the root of the project, which ...
- Dumps a clean copy of the application into the build directory
- Creates a WAR from the build directory and puts the WAR into the dist directory
- Copies the WAR file to a target server using SCP
Once the WAR file is copied to the target server, provided that Tomcat is configured to do so it will redeploy the application using the new WAR file.
There's of course a bunch of different ways to configure a lot of this but this is working well for us. If you have other approaches or if anything I'm doing could be improved upon, I'd love to hear how you're using Jenkins.
