Blog
Configuring Test Kitchen output for Jenkins
The busser runner we are using for our Kitchen tests is busser-serverspec and at the time of this writing, the problem of having this busser output test results in a format other than the default one, is still open and it’s being tracked on this github issue
Some of the ideas in this ticket served as a starting point for our approach although we took a slightly different route, specifically in the step that takes care of transferring the junit-formatted output from the machine under test to the Jenkins build agent.
Here is a way you can get your serverspec Test Kitchen tests output to be successfully parsed by Jenkins.
Configure serverspec to install required dependencies
In order to install a JUnit formatter gem in the machine under test, create a Gemfile inside the serverspec folder of your recipe’s test folder, if you don’t have one already. The Gemfile should contain the following:
source 'https://rubygems.org'
gem ‘yarjuf’
# other dependencies
Yarjuf is a JUnit formatter for RSpec and it produces test results in a format that is fully compatible with Jenkins. If serverspec finds this Gemfile in the correct folder, it will install all of the dependencies defined in there and make them available during the tests.
Configure RSpec to use the JUnit formatter
Once Yarjuf is installed, we can instruct RSpec to use it by including the following lines in the spec_helper.rb file for your test machine:
require ‘yarjuf’
RSpec.configure do |c|
c.formatter = ‘JUnit’
end
This has the effect that the console output of ‘kitchen test vm-name’ will contain the test results formatted in JUnit. The next step takes care of extracting such report and save it in a xml file that can be directly passed to Jenkins for the parsing step.
Extract xml report out of Test Kitchen output
This step is where our approach differs from the one advised in the github ticket mentioned above. Instead of having the JUnit formatter write its output on a file local to the machine under test and then transfer the file to the Jenkins test agent, we let the formatter output the report within the console output.
Using the tee command, we copy the console output to a file, and then feed this file to a awk script which takes care of isolating the report into a valid xml file:
kitchen test -d always vm_name | tee kitchen_tests_vm_name.log;
awk -f filter_junit.awk kitchen_tests_vm_name.log > kitchen_tests_vm_name.xml
finally, this is the content of the filter_junit.awk script we wrote to extract the JUnit report:
#!/usr/bin/awk -f
BEGIN {
should_print=0;
}
{
if ($0 ~ /\?xml/ ) should_print=1;
gsub(/^[ \t]+/, “”, $0)
if ($0 ~ /<\/testsuites>/ ) {
print $0;
should_print=0;
}
if (should_print) print $0;
}
Configure Jenkins to parse the serverspec
JUnit xml files As the last step, we need to tell Jenkins where our JUnit reports are located. To do so, in the build configuration page click ‘Add post-build action’ at the bottom, select ‘Publish JUnit test result report’ and edit the location of the files according to your setup.
Conclusions
The steps outlined here helped us getting a step further in our Continuous Integration for our infrastructure code. Certainly having Jenkins understand the test results is much better than having to go through the whole Test Kitchen output to find the results of our tests. Of course this approach is far from ideal, but given the current state of the libraries involved, it was a good compromise for a decent test reporting without excessive effort in the setup. Hopefully this can all be avoided once serverspec will support JUnit reporting and Test Kitchen will accordingly export such reports to testing machine. Until then, please feel free to share with us your ideas and other possible approaches.
Latest entries
- Agilo for Scrum is retiring
- Django-treebeard and Wagtail page creation
- The Charity Sport Tournament in Lublin
- New Release of Agilo for Trac (0.9.15/1.3.15)
- Incontro DevOps Italia 2016
- Configuring Test Kitchen output for Jenkins
- Configuring Test Kitchen on Jenkins
- Better infrastructure management a.k.a. IAC (Infrastructure as Code)
- Our approach to automated visual regression testing
- Test parallelization with Lettuce, take 2