Test Kitchen Blog

Test Kitchen 1.3.0 Release Notes

by Fletcher Nichol on

For immediate release: Test Kitchen 1.3.0 is available on RubyGems.

Hey there, how’s it going? It’s been a while, huh? Also welcome—I’ll wager this is the first blog entry you’ve read here. There’s lots we can talk about, but for today let’s focus (and celebrate) a new release of Test Kitchen.

In terms of code churn and long term viability of the codebase, the biggest change this latest release brings is an upgrade to the project’s health tooling, a full backfilling of unit test coverage, a full reworking of the developer documentation in the form of YARD commenting, and even some refactoring as a result. Most of this work was contained in pull request 427 and will be a pretty dry read for most end users. However the reasoning behind it might be useful to some, so here’s an excerpt from the pull request text:

So what happens when a project comes to life by inspiration and accident, then gets picked up and starts getting used in the field? Well, it's a balancing act of adding new features and tending the codebase garden so that future features are possible.

Let's call this PR a bobcat re-landscaping effort.

The goal here is backfill missing unit test coverage in critical parts of the codebase for 3 big reasons:

  1. Make future refactoring quicker and lower stress. It becomes extremely expensive to properly refactor without a safety net of tests–they help enforce the previous contract for any new or modified production code.
  2. Allow more safety when accepting pull requests and contributions. Once a more complete code coverage is achieved, we will start to ask for accompanying unit and/or integration tests to accompany pull requests. This benefits everyone.
  3. Provide more use cases and developer documentation when navigating the codebase. Developer docs and code examples are one dimension of a successful "documented project".

Let’s take a peek at some of the highlights…

Highlights

PR 381/456: Add configurable defaults for chef-solo, chef-client, and chef omnibus paths

For Chef provisioners (namely, chef_solo and chef_zero) there are 2 (well 3) new configuration attributes available for customization:

  • chef_omnibus_root: is used when checking for a Chef Omnibus installation and to calculate the default path to the chef-client or chef-solo binaries. By default, this value is ”/opt/chef”. If you would rather use the ChefDK distribution, then you could set this value to ”/opt/chefdk”.
  • chef_client_path: is used with the chef_zero provisioner and lets you customize your path to the chef-client binary when performing a Chef run. By default, this value is ”/opt/chef/bin/chef-client” but will automatically adjusted if chef_omnibus_root is changed (see above).
  • chef_solo_path: is used with the chef_solo provisioner and lets you customize your path to the chef-solo binary when performing a Chef run. By default, this value is ”/opt/chef/bin/chef-solo” but will automatically adjusted if chef_omnibus_root is changed (see above).

PR 489: Introduce the :chef_omnibus_install_options config attribute to be able to pass additional arguments to the Chef installer script

This new configuration attribute allows you to pass additional arguments to the install.sh script which can install not only Omnibus Chef packages, but also the ChefDK distribution which is very useful for workstation development. Here’s a more complete example which also uses the new chef_omnibus_root configuration attribute as described above:

---
driver:
  name: vagrant

provisioner:
  name: chef_zero
  chef_omnibus_install_options: -P chefdk
  chef_omnibus_root: /opt/chefdk

platforms:
  - name: ubuntu-12.04
  - name: centos-6.4

suites:
  - name: default
    run_list:
      - workstation_amazing
    attributes:

Note that updating the chef_omnibus_root is required so that the path to the chef-client binary becomes ”/opt/chefdk/bin/chef-client”. Commence workstation testing!

PR 478: Buffer Logger output & fix Chef run output formatting

Speaking personally, this is one of the most exciting improvements in this release. This returns the Chef run output to be free of clutter, noise, and duplication.

Taken from the text of the pull request:

For more detail you can check out the commit message for the included commits.

Turns out there were competing issues:

  • Adding the explicit --log_level info option on chef-solo and chef-client were conflating both outputs together
  • The SSH connection is requesting a TTY (for older/stock sudoers policies) which caused the Chef runs to believe they were running in interactive mode
  • The interactive mode also enabled coloring which isn't necessary since Kitchen is coloring the output itself

When I attempted to use --force-logger --log_level info (in interactive) I was seeing double output of each line. When I force the Chef run to operate non-interactively (by piping through cat: chef-solo -force-logger --log_level info ... | cat) this resolved the double output.

It seems to me that fixing the output formatting will go so far to make things better here, but is it possible that a user will need to make use of the classic logger format and thus need an option to switch this out? I'm not sold either way, but know that any formatting cleanup will be an improvement and help bring clarity back to the Chef runs here.

PR 481: Disable color output when no TTY is present

This should help clean up Test Kitchen output in CI environments, using the kitchen command with unix pipes, etc.

In other words:

kitchen test | cat

Should not output colors.

Currently, the only catch is that colors are still used inside the instances for tools such as RSpec. This may require work in the Busser component.

PR 555: Correct global YAML merge order to lowest (from highest)

This addresses a merge order bug-of-intention between the following files:

  • .kitchen.yml (project config)
  • .kitchen.local.yml (local config)
  • $HOME/.kitchen/config.yml (global config)

The intended behavior was that:

  1. baseline common configuration can go in the global config
  2. any colliding values in the project config "win" over the global config
  3. any colliding values in the local config "win" over the project config (and consequently also the global config)

This pull request restores the intended merge semantics.

Full Changleog

Upstream changes

  • Pull request #558, pull request #557: Update omnibus download URL to chef.io. (@scarolan)
  • Pull request #531, pull request #521: Update mixlib-shellout dependency to be greater or equal to 1.2 and less than 3.0. (@lamont-granquist)

Bug fixes

  • Pull request #555, issue #524, issue #343: (Breaking) Correct global YAML merge order to lowest (from highest).
  • Pull request #416: Use Gem::GemRunner to install drivers with kitchen init generator (addresses opscode/chef-dk#20). (@mcquin)
  • Pull request #399: Sleep before retrying SSH#establish_connection. (@fnichol)
  • Pull request #527: Rescue SSH authentication failures to use retry/timeout connection logic. (@chrishenry)
  • Pull request #363: Ensure that integer chef config attributes get placed in solo.rb/client.rb properly. (@benlangfeld, @sethvargo)
  • Pull request #431: Check for zero byte state files. (@rhass)
  • Pull request #554, pull request #543: Replace / with - in Instance names, allowing instance names to be used as server hostnames in most cases. (@grubernaut, @fnichol)

New features

  • Pull request #373: Add new subcommand 'exec'. (@sawanoboly)
  • Pull request #397: Introduce the :chef_zero_port config attribute to the chef_zero provisioner. (@jtgiri)
  • Pull request #381, pull request #456: Add configurable defaults for chef-solo, chef-client, and chef omnibus paths. (@sethvargo, @robcoward, @fnichol)
  • Pull request #489: Introduce the :chef_omnibus_install_options config attribute to be able to pass additional arguments to the Chef installer script. (@ringods)
  • Pull request #549: Introduce the :chef_zero_host config attribute to the chef_zero provisioner. (@jochenseeber)
  • Pull request #454: Customize :ssh_timeout and :ssh_retries. (@ekrupnik)
  • Pull request #510, issue #166: Add support for site-cookbooks when using Librarian. (@jstriebel)

Improvements

  • Pull request #427: Backfilling spec coverage and refactoring: technical debt edition. Epically huge, finally complete. (@fnichol)
  • Pull request #478, issue #433, issue #352: Buffer Logger output & fix Chef run output formatting. (@fnichol)
  • Pull request #481: Disable color output when no TTY is present. (@fnichol)
  • Pull request #526, pull request #462, issue #375: Die on kitchen login if instance is not created. (@daniellockard, @fnichol)
  • Pull request #504: Fix 2 tests in SSHBase to reflect their intent. (@jgoldschrafe)
  • Pull request #450: Update help description for CLI subcommands. (@MarkGibbons)
  • Pull request #477: Bump 'kitchen help' into new Usage section in README and add how to use "-l". (@curiositycasualty)
  • Pull request #567: Pass the template filename down to Erb for __FILE__ et al. (@coderanger)
  • Pull request #498: Fix doc comment in Kitchen::Loader::YAML.new. (@jaimegildesagredo)
  • Pull request #507: Clarify comments in configuration loader. (@martinb3)
  • Pull request #366: Fix glaring "Transfering" -> "Transferring" typo. (@srenatus)
  • Pull request #457: Fix "confiuration" -> "configuration" typo in README. (@michaelkirk)
  • Pull request #370: Use Ruby 2.1 instead of 2.1.0 for CI. (@justincampbell)

Heads up

  • Drop Ruby 1.9.2 from TravisCI build matrix (support for Ruby 1.9.2 will be a "best effort"). (@fnichol)
Made in North America

© 2013, Heavy Water Operations, LLC (OR).