TL;DR;

If you just want a working solution for setting up Jenkins with Chef supporting Ruby and Rails projects, just follow the steps in this section.

If you are interested in the details, move to cookbook section.

Prerequisites

For OSX the following snippet can be used:

I’m assuming rvm, chef, kitchen and Berkshelf are already installed.
Also make sure that vagrant and VirtualBox are present so the cookbook can be run with kitchen.

How to use the u2i-jenkins cookbook?

Clone u2i/u2i-jenkins, get the right tag and run kitchen:

These commands run the Ubuntu virtual machine using VirtualBox and install Jenkins.

By default the kitchen suite in the cookbook sets up the CI with two example projects:

To set it up for your project check out the custom jobs section.

When kitchen is done visit localhost:8080 to view the Jenkins website.

Cookbook

First, the new cookbook will be created using Berkshelf.
Berkshelf is a cookbook dependency manager. If you know Bundler, getting familiar with this tool should be very easy.

Below I present the most interesting parts of my cookbook.
I’ve tried to describe every step in detail. However, if you are missing some information or the instructions are not clear feel free to check the source code of the cookbook which is available in our GitHub repository.

New cookbook

I’m assuming rvm, chef, kitchen and Berkshelf are already installed.
Also make sure that vagrant and VirtualBox are present so the cookbook can be run with kitchen.

First, let’s create a cookbook:

Change the cookbook maintainer and add copyright info in metadata.rb.

Commit and push to git remote:

Install Jenkins

For installing Jenkins, the jenkins cookbook is used.

I prefer to use a dedicated java cookbook over the jenkins::java recipe.
The dependencies need to be specified explicitly in metadata.rb:

I suggest you fix the version of Jenkins in the cookbook attributes.
Jenkins is updated quite frequently and often introduces compatibility changes which might cause problems with either the jenkins cookbook or simply the assumptions you made in your cookbook.

We can set the attributes in attributes/default.rb to specify Jenkins version.

If the war method is used instead of package then it’s probably a good idea to include the java cookbook. This way we have better control over the Java version and we can use Oracle Java.
Include the master recipe to install Jenkins in the default path (/var/lib/jenkins):

First run

Now we are ready to push the cookbooks to the Chef server and run chef-client on the new machine or Vagrant.
However, I find kitchen with the Vagrant driver the easiest way of testing cookbooks.

To run our recipes the .kitchen.yml must be modified.

  • change provisioner to chef_zero
  • fix chef version 11.18.0
  • specify one platform
  • modify driver configuration
  • increase memory and number of CPUs to speed up provisioning process
  • forward port so Jenkins is accesible from the host machine
  • if you’re running Linux you can consider using docker instead of vagrant but that’s a story for another guide
  • add the new recipe to the run_list

The .kitchen.yml should look like that:

Now the cookbook is ready to run kitchen converge.
Run:

and go for a cup of tea.

When done, verify Jenkins is installed by visiting localhost:8089.

Jenkins plugins

Now we’re ready to install some plugins. When using the Jenkins GUI, the CI manages to install the plugins together with its dependencies. Unfortunately, with the cookbook (which uses jenkins-cli), the dependencies have to be specified explicitly.

The code below installs the plugins and sets the restart_required flag when any plugin is installed during the current chef run.
If so, Jenkins is restarted.
This solution was found in the jenkins-chef-dsl::plugins recipe.

Now the plugins can be specified by attributes:

Run kitchen converge and verify the plugins were installed by visiting plugin manager.

Jenkins configuration

It took some time to find the right plugins to get things done.
Also figuring out which attributes in the XMLs are responsible for the configuration in Jenkins is pretty tedious. What I did was set things up in the UI and then analysed what had changed in the XML.

I don’t think it’s interesting to explain the structure of those configuration files in detail.
The XMLs for the plugins and the Jenkins configuration can be analysed in the repository

The code below copies all the XMLs to the Jenkins home directory:

Adjusting the configuration

If you want to create your own custom job or change the current configuration the best approach is to set it up in the Jenkins GUI first.

Once you’ve played around with the CI and adjusted it to your needs, analyse what has changed in the XML config files and introduce those changes in the Chef templates.

Analyse the code of _jobs_project recipe to prepare a recipe for your project and adjust the node config .kitchen.yml.
Remember to provide all the necessary ssh keys and oauth tokens in encrypted data bag.

Troubleshooting

Because of various things like changes in the Jenkins cookbook or the Jenkins API there is a high chance that the first chef run might finish with an error.
If that happens, log in to the machine and try restarting the service:

and then try to run kitchen converge again.

Improvements

At this point we have a simple Jenkins cookbook which stores parameterised XMLs for all jobs.
This way we have full control. However there is still a lot to improve:

  • more flexibility — in this basic version there is way to manage hardcoded options
  • make the cookbook more DRY — job configs are just copy-pasted.
  • no authentication — in most use cases authentication and authorization are must haves
  • support any kind of git repository instead of just GitHub

I am going to address some of those problems in the next blog post!