Test Kitchen 1.3.0 Release Notes
by Fletcher Nichol on
Hey there, how’s it going? It’s been a while, huh? Also welcome—I’ll wager this is the first blog entry you’ve read here. There’s lots we can talk about, but for today let’s focus (and celebrate) a new release of Test Kitchen.
In terms of code churn and long term viability of the codebase, the biggest change this latest release brings is an upgrade to the project’s health tooling, a full backfilling of unit test coverage, a full reworking of the developer documentation in the form of YARD commenting, and even some refactoring as a result. Most of this work was contained in pull request 427 and will be a pretty dry read for most end users. However the reasoning behind it might be useful to some, so here’s an excerpt from the pull request text:
So what happens when a project comes to life by inspiration and accident, then gets picked up and starts getting used in the field? Well, it's a balancing act of adding new features and tending the codebase garden so that future features are possible.
Let's call this PR a bobcat re-landscaping effort.
The goal here is backfill missing unit test coverage in critical parts of the codebase for 3 big reasons:
- Make future refactoring quicker and lower stress. It becomes extremely expensive to properly refactor without a safety net of tests–they help enforce the previous contract for any new or modified production code.
- Allow more safety when accepting pull requests and contributions. Once a more complete code coverage is achieved, we will start to ask for accompanying unit and/or integration tests to accompany pull requests. This benefits everyone.
- Provide more use cases and developer documentation when navigating the codebase. Developer docs and code examples are one dimension of a successful "documented project".
Let’s take a peek at some of the highlights…
PR 381/456: Add configurable defaults for chef-solo, chef-client, and chef omnibus paths
For Chef provisioners (namely,
chef_zero) there are 2 (well 3) new configuration attributes available for customization:
chef_omnibus_root: is used when checking for a Chef Omnibus installation and to calculate the default path to the
chef-solobinaries. By default, this value is
”/opt/chef”. If you would rather use the ChefDK distribution, then you could set this value to
chef_client_path: is used with the
chef_zeroprovisioner and lets you customize your path to the
chef-clientbinary when performing a Chef run. By default, this value is
”/opt/chef/bin/chef-client” but will automatically adjusted if
chef_omnibus_rootis changed (see above).
chef_solo_path: is used with the
chef_soloprovisioner and lets you customize your path to the
chef-solobinary when performing a Chef run. By default, this value is
”/opt/chef/bin/chef-solo” but will automatically adjusted if
chef_omnibus_rootis changed (see above).
PR 489: Introduce the :chef_omnibus_install_options config attribute to be able to pass additional arguments to the Chef installer script
This new configuration attribute allows you to pass additional arguments to the
install.sh script which can install not only Omnibus Chef packages, but also the ChefDK distribution which is very useful for workstation development. Here’s a more complete example which also uses the new
chef_omnibus_root configuration attribute as described above:
--- driver: name: vagrant provisioner: name: chef_zero chef_omnibus_install_options: -P chefdk chef_omnibus_root: /opt/chefdk platforms: - name: ubuntu-12.04 - name: centos-6.4 suites: - name: default run_list: - workstation_amazing attributes:
Note that updating the
chef_omnibus_root is required so that the path to the
chef-client binary becomes
”/opt/chefdk/bin/chef-client”. Commence workstation testing!
PR 478: Buffer Logger output & fix Chef run output formatting
Speaking personally, this is one of the most exciting improvements in this release. This returns the Chef run output to be free of clutter, noise, and duplication.
Taken from the text of the pull request:
For more detail you can check out the commit message for the included commits.
Turns out there were competing issues:
- Adding the explicit
--log_level infooption on
chef-clientwere conflating both outputs together
- The SSH connection is requesting a TTY (for older/stock sudoers policies) which caused the Chef runs to believe they were running in interactive mode
- The interactive mode also enabled coloring which isn't necessary since Kitchen is coloring the output itself
When I attempted to use
--force-logger --log_level info(in interactive) I was seeing double output of each line. When I force the Chef run to operate non-interactively (by piping through cat:
chef-solo -force-logger --log_level info ... | cat) this resolved the double output.
It seems to me that fixing the output formatting will go so far to make things better here, but is it possible that a user will need to make use of the classic logger format and thus need an option to switch this out? I'm not sold either way, but know that any formatting cleanup will be an improvement and help bring clarity back to the Chef runs here.
PR 481: Disable color output when no TTY is present
This should help clean up Test Kitchen output in CI environments, using the
kitchen command with unix pipes, etc.
In other words:
kitchen test | cat
Should not output colors.
Currently, the only catch is that colors are still used inside the instances for tools such as RSpec. This may require work in the Busser component.
PR 555: Correct global YAML merge order to lowest (from highest)
This addresses a merge order bug-of-intention between the following files:
The intended behavior was that:
- baseline common configuration can go in the global config
- any colliding values in the project config "win" over the global config
- any colliding values in the local config "win" over the project config (and consequently also the global config)
This pull request restores the intended merge semantics.
- Pull request #558, pull request #557: Update omnibus download URL to chef.io. (@scarolan)
- Pull request #531, pull request #521: Update mixlib-shellout dependency to be greater or equal to 1.2 and less than 3.0. (@lamont-granquist)
- Pull request #555, issue #524, issue #343: (Breaking) Correct global YAML merge order to lowest (from highest).
- Pull request #416: Use
Gem::GemRunnerto install drivers with
kitchen initgenerator (addresses opscode/chef-dk#20). (@mcquin)
- Pull request #399: Sleep before retrying SSH#establish_connection. (@fnichol)
- Pull request #527: Rescue SSH authentication failures to use retry/timeout connection logic. (@chrishenry)
- Pull request #363: Ensure that integer chef config attributes get placed in solo.rb/client.rb properly. (@benlangfeld, @sethvargo)
- Pull request #431: Check for zero byte state files. (@rhass)
- Pull request #554, pull request #543: Replace
-in Instance names, allowing instance names to be used as server hostnames in most cases. (@grubernaut, @fnichol)
- Pull request #373: Add new subcommand 'exec'. (@sawanoboly)
- Pull request #397: Introduce the
:chef_zero_portconfig attribute to the chef_zero provisioner. (@jtgiri)
- Pull request #381, pull request #456: Add configurable defaults for chef-solo, chef-client, and chef omnibus paths. (@sethvargo, @robcoward, @fnichol)
- Pull request #489: Introduce the
:chef_omnibus_install_optionsconfig attribute to be able to pass additional arguments to the Chef installer script. (@ringods)
- Pull request #549: Introduce the
:chef_zero_hostconfig attribute to the chef_zero provisioner. (@jochenseeber)
- Pull request #454: Customize
- Pull request #510, issue #166: Add support for site-cookbooks when using Librarian. (@jstriebel)
- Pull request #427: Backfilling spec coverage and refactoring: technical debt edition. Epically huge, finally complete. (@fnichol)
- Pull request #478, issue #433, issue #352: Buffer Logger output & fix Chef run output formatting. (@fnichol)
- Pull request #481: Disable color output when no TTY is present. (@fnichol)
- Pull request #526, pull request #462, issue #375: Die on
kitchen loginif instance is not created. (@daniellockard, @fnichol)
- Pull request #504: Fix 2 tests in
SSHBaseto reflect their intent. (@jgoldschrafe)
- Pull request #450: Update help description for CLI subcommands. (@MarkGibbons)
- Pull request #477: Bump 'kitchen help' into new Usage section in README and add how to use "-l". (@curiositycasualty)
- Pull request #567: Pass the template filename down to Erb for
__FILE__et al. (@coderanger)
- Pull request #498: Fix doc comment in
- Pull request #507: Clarify comments in configuration loader. (@martinb3)
- Pull request #366: Fix glaring "Transfering" -> "Transferring" typo. (@srenatus)
- Pull request #457: Fix "confiuration" -> "configuration" typo in README. (@michaelkirk)
- Pull request #370: Use Ruby 2.1 instead of 2.1.0 for CI. (@justincampbell)
- Drop Ruby 1.9.2 from TravisCI build matrix (support for Ruby 1.9.2 will be a "best effort"). (@fnichol)