Jasmine and Jenkins Continuous Integration

I use Jasmine as my JavaScript unit/behavior testing framework of choice because it’s elegant and has a good community ecosystem around it. I recently wrote up how to get Jasmine-based autotesting set up with Guard, which is great for development time testing, but what about continuous integration?

Well, it turns out that it’s pretty difficult to get Jasmine integrated with Jenkins. This is not because of an inherent problem with either of those two, it’s just that no-one got around to writing an open source integration layer until now.

The main problem is that Jasmine tests usually expect to run in a browser, but Jenkins needs results to be exposed in .xml files. Clearly we need some bridge here to take the headless browser output and dump it into correctly formatted .xml files. Specifically, these xml files need to follow the JUnit XML file format for Jenkins to be able to process them. Enter guard-jasmine.

guard-jasmine

In my previous article on getting Jasmine and Guard set up, I was using the jasmine-headless-webkit and guard-jasmine-headless-webkit gems to provide the glue. Since then I’ve replaced those 2 gems with a single gem – guard-jasmine, written by Michael Kessler, the Guard master himself. This simplifies our dependencies a little, but doesn’t buy us the .xml file functionality we need.

For that, I had to hack on the gem itself (which involved writing coffeescript for the first time, which was not a horrible experience). The guard-jasmine gem now exposes 3 additional configurations:

  • junit – set to true to save output to xml files (false by default)
  • junit_consolidate – rolls nested describes up into their parent describe blocks (true by default)
  • junit_save_path – optional path to save the xml files to

The JUnit Xml reporter itself borrows heavily from larrymyers‘ excellent jasmine-reporters project. Aside from a few changes to integrate it into guard-jasmine it’s the same code, so all credit goes to to Larry and Michael.

Sample usage:

In your Guardfile:

guard :jasmine, :junit => true, :junit_save_path => 'reports' do
  watch(%r{^spec/javascripts/.+$}) { 'spec/javascripts' }
  watch(%r{^spec/javascripts/fixtures/.+$}) { 'spec/javascripts' }
  watch(%r{^app/assets/javascripts/(.+?)\.(js\.coffee|js|coffee)(?:\.\w+)*$}) { 'spec/javascripts' }
end

This will just run the full set of Jasmine tests inside your spec/javascripts directory whenever any test, source file or asset like CSS files change. This is generally the configuration I use because the tests execute so fast I can afford to have them all run every time.

In the example above we set the :junit_save_path to ‘reports’, which means it will save all of the .xml files into the reports directory. It is going to output 1 .xml file for each Jasmine spec file that is run. In each case the name of the .xml file created is based on the name of the top-level `describe` block in your spec file.

To test that everything’s working, just run `bundle exec guard` as you normally would, and check to see that your `reports` folder now contains a bunch of .xml files. If it does, everything went well.

Jenkins Settings

Once we’ve got the .xml files outputting correctly, we just need to tell Jenkins where to look. In your Jenkins project configuration screen, click the Add Build Step button and add a “Publish JUnit test result report” step. Enter ‘reports/*.xml’ as the `Test report XMLs` field.

If you’ve already got Jenkins running your test script then you’re all done. Next time a build is triggered the script should run the tests and export the .xml files. If you don’t already have Jenkins set up to run your tests, but you did already set up Guard as per my previous article, you can actually use the same command to run the tests on Jenkins.

After a little experimentation, people tend to come up with a build command like this:

bash -c ' bundle install --quiet \
&& bundle exec guard '

If you’re using rvm and need to guarantee a particular version you may need to prepend an `rvm install` command before `bundle install` is called. This should just run guard, which will dump the files out as expected for Jenkins to pick up.

To clean up, we’ll just add a second post-build action, this time choosing the “Execute a set of scripts” option and entering the following:

kill -9 `cat guard.pid`

This just kills the Guard process, which ordinarily stays running to power your autotest capabilities. Once you run a new build you should see a chart automatically appear on your Jenkins project page telling you full details of how many tests failed over time and in the current build.

Getting it

Update: The Pull Request is now merged into the main guard-jasmine repo so you can just use `gem ‘guard-jasmine’` in your Gemfile

This is hot off the presses but I wanted to write it up while it’s still fresh in my mind. At the time of writing the pull request is still outstanding on the guard-jasmine repository, so to use the new options you’ll need to temporarily use my guard-jasmine fork. In your Gemfile:

gem 'guard-jasmine'

Once the PR is merged and a new version issued you should switch back to the official release channel. It’s working well for me but it’s fresh code so may contains bugs – YMMV. Hopefully this helps save some folks a little pain!

5 Responses to Jasmine and Jenkins Continuous Integration

  1. Richard says:

    Did you mean to include `bundle update –quiet`? that would update all dependencies that are already locked in the gemfile.lock. I really don’t recommend doing this. Gem updates should not happen automatically. This will likely break your code at some point.

  2. Hi ed,

    I’m trying to follow your 2 posts about having jasmine and guard working together to run tests automatically on developer machine as well as on CI Server.

    I’m using Windows 7 64 and Ruby v2.0.0p247 and after sorting lots of blocks I got the guard watching my test files and running them. However I cannot go on from this due to the reports are not being generated.

    In the end of this post you say we need to get the guard-jasmine from your fork, however I was reviewing the code and it looks like your changes have been merged into the release code (I could also confirm this by checking the code I had in my local gem installation)

    I ended up with this Gemfile

    source “https://rubygems.org”

    gem ‘guard’
    gem ‘jasmine’
    gem ‘guard-jasmine’
    gem ‘wdm’
    gem ‘win32console’

    The Guardfile goes as yours and running the bundle exec guard command watch for changes on my tests and rerun them, but not reports folder is created.

    Do I need to add more gems on the Gemfile? (something like gem ‘jasmine-reporters’)
    Could you please provide the Gemfile and Guardfile files of your CI integrated environment?
    is there any other information you need from me to have a better understanding of my issue?

    Thank you in advance.

  3. edspencer says:

    Hey Jonathan,

    Yeah looks like the changes are merged into the main repo now so I’ve updated the post to reflect that. I’m running on OS X – it’s possible I got something wrong in the pull request that breaks windows 7… I don’t have a machine to test that on though.

    If you have access to an OS X machine you can try it on that would at least tell us if it’s a platform issue or not, might make debugging easier.

  4. edspencer says:

    @Richard I think you’re right, have removed the offending line

  5. Hi Ed,

    Unfortunately I don’t have an OS X to test the configuration. I’ve posted an issue with more details than I’ve provided in here. Hope it helps
    The issue url is https://github.com/netzpirat/guard-jasmine/issues/149

    So, besides OS, my gemfile and guardfile are ok or I am missing something?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 2,604 other followers

%d bloggers like this: