Mischa Taylor's Coding Blog

On the edge of time and reason

Chef Server 12

The source code for this article can be found at https://github.com/learningchef/learningchef-code/tree/master/chefserver12.

Make sure you install the latest Chef Development Kit 0.3.5 (or higher) and/or Chef Client 11.18 (or higher) before trying to run knife commands against Chef Server 12. It appears that some changes were made to the clients for administrators to work with Chef Server 12, so these newer versions (at the time of this writing) are required that came out since the Learning Chef book was published.

Introduction

In the Learning Chef book we cover Chef Server 11 in Chapters 9 and 10, as that was the version of Chef Server available when we were writing the book. Since then, Chef Server 12 has been released. This blog post covers how the material presented in Chapter 9 can be adapted for Chef Server 12. No changes are needed in Chapter 10 for Chef Server 12.

Chef Server 12 merges the code bases for what were three separate flavors of Chef Server:

  • Open Source Chef Server
  • On-Premises Enterprise Chef
  • Hosted Enterprised Chef

Also the pricing for Chef Server has changed as well to match. For more information on the changes, refer to Chef plans and pricing.

From a technical standpoint, the great thing about Chef Server 12 is that is now shares the same core, whether or not you choose to use what used to be the open source option or you pay for a subscription to take advantage of Chef Server’s Premium features.

Installing Chef Server 12 Manually

To install Chef Server, go to https://downloads.getchef.com/ and click on the “Get It” button, as shown in the following screenshot:

From there, you are presented with a download link page where you can choose to download Chef Server 12. Chef Server 12 currently provides install packages for both the Red Hat Enterprise Linux and Ubuntu Linux platforms (sorry, no Windows support for the server piece, only for Windows clients):

To manually replicate a basic Chef Server install in a cookbook, we first need to download the Chef Server 12 install package for Red Hat Enterprise Linux 6, as we’ll be installing on CentOS 6.5. To match what is being written in this article, use version 12.0.0. Use the Copy Link Address option on the download link to copy the full download URl to your clipboard.

Here’s the rest of the steps necessary to install Chef Server:

  1. Install the chef-server package.
  2. Run sudo chef-server-ctl reconfigure.

NOTE: The name of the command line application to configure Chef Server has changed from private-chef-ctl to chef-server-ctl with version 12.

Install Chef Server 12

Assuming you have sufficient resources to install Chef Server 12 locally along with a test node, let’s create a chef-server cookbook that will install Chef Server 12. To maintain consistency with Hosted Enterprise Chef, create the directory chef-repo/cookbooks and create the chef-server cookbook in that directory. Having a top-level chef-repo directory will help you handle the additional files necessary to manage Chef Server 12 beyond the cookbooks themselves. You’ll definitely be using more than one cookbook in your organization, so we suggest putting them in a chef-repo/coobkooks subdirectory.

Create the chef-repo/cookbooks directory and make it the current working directory.

Linux/Mac OS X:

$ mkdir -p chef-repo/cookbooks
$ cd chef-repo/cookbooks

Windows:

> mkdir chef-repo\cookbooks
> cd chef-repo\cookbooks

Then generate the chef-server cookbook with chef generate cookbook or knife cookbook create, dependening on whether you are using the Chef Development Kit or Chef Client. We’re going to go through the cookbook creation steps quickly in this article. If you need a refresher on what each of these commands mean and the expected output, refer to Chapter 7 of the Learning Chef book.

Chef Development Kit:

$ chef generate cookbook chef-server
$ cd chef-server

Chef Client:

$ knife cookbook create chef-server --cookbook-path .
$ cd chef-server
$ kitchen init --create-gemfile
$ bundle install

As shown in the code example below, edit the .kitchen.yml file to use the CentOS 6.5 basebox we prepared specifically for the Learning Chef book. Also assign a private network address like we did in Chapter 7 of Learning Chef. This time we’re going to use the IP address 192.168.38.34. If this conflicts with an address already being used on your local network. change it to a nonconflicting one. We also need more memory than the default 512 MB allocated, so add a customize: block with a memory: statement to increase the memory to 1.5 GB (memory is specified in megabytes only).

NOTE: We also changed the suite name to server as we’ll be logging in to the virtual machine with Chef Server 12. This makes it more clear that the examples should be run on the Chef Server machine.

chef-repo/cookbooks/chef-server/.kitchen.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
 ---
 driver:
   name: vagrant

 provisioner:
   name: chef_solo

 platforms:
   - name: centos65
     driver:
       box: learningchef/centos65
       box_url: learningchef/centos65
       network:
       - ["private_network", {ip: "192.168.38.34"}]
       customize:
         memory: 1536

 suites:
   - name: server
     run_list:
       - recipe[chef-server::default]
     attributes:

Generate a default attributes file in attributes/default.rb.

Chef Development Kit:

$ chef generate attribute default

Chef Client:

$ touch attributes/default.rb

Add an attribute specifying the download URL for the Chef Server package that you obtained from the download link page. We recommend using the 12.0.0 version URL as shown below, as we wrote the exampels for this article using this Version of Chef Server.

vmware/workstation/.kitchen.yml
1
2
3
default['chef-server']['url'] = \
  'https://web-dl.packagecloud.io/chef/stable/packages/el/6/'\
  'chef-server-core-12.0.0-1.el6.x86_64.rpm'

From here, we’re just going to skip ahead to the final bit of code in the “Introducing Idempotence” section of Chapter 9 in Learning Chef, as everything remains the same for Chef Server 12. The only difference is the command line app for configuring Chef Server is now called chef-server-ctl instead of private-server-ctl. Refer to Chapter 9 in Learning Chef for more explanation on what this code does.

chefdk/chef-repo/cookbooks/chef-server/recipes/default.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
#
# Cookbook Name:: chef-server
# Recipe:: default
#

package_url = node['chef-server']['url']
package_name = ::File.basename(package_url)
package_local_path = "#{Chef::Config[:file_cache_path]}/#{package_name}"

# package is remote, let's download it
remote_file package_local_path do
  source package_url
end

package package_name do
  source package_local_path
  provider Chef::Provider::Package::Rpm
  notifies :run, 'execute[reconfigure-chef-server]', :immediately
end

# reconfigure the installation
execute 'reconfigure-chef-server' do
  command 'chef-server-ctl reconfigure'
  action :nothing
end

Try running kitchen converge against this recipe, and note that it reports 0/2 resources updated, which is the result we are looking for; no resources are updated after running kitchen converge for the second time:

$ kitchen converge
-----> Starting Kitchen (v1.2.1)
-----> Converging <default-centos65>...
...
Converging 3 resources
       Recipe: chef-server::default
         * remote_file[/tmp/kitchen/cache/chef-server-core-12.0.0-1.el6.x86_64.rpm] action create[2014-11-26T01:27:20+00:00] INFO: Processing remote_file[/tmp/kitchen/cache/chef-server-core-12.0.0-1.el6.x86_64.rpm] action create (chef-server::default line 11)
        (up to date)
         * package[chef-server-core-12.0.0-1.el6.x86_64.rpm] action install[2014-11-26T01:27:27+00:00] INFO: Processing package[chef-server-core-12.0.0-1.el6.x86_64.rpm] action install (chef-server::default line 15)
        (up to date)
         * execute[reconfigure-chef-server] action nothing[2014-11-26T01:27:28+00:00] INFO: Processing execute[reconfigure-chef-server] action nothing (chef-server::default line 22)
        (skipped due to action :nothing)
       [2014-11-26T01:27:28+00:00] INFO: Chef Run complete in 7.811144016 seconds

       Running handlers:
       [2014-11-26T01:27:28+00:00] INFO: Running report handlers
       Running handlers complete
       [2014-11-26T01:27:28+00:00] INFO: Report handlers complete
       Chef Client finished, 0/2 resources updated in 10.600168629 seconds
       Finished converging <default-centos65> (0m12.49s).
-----> Kitchen is finished. (0m13.51s)

Always check your recipes to see if they are idempotent before deploying them to production. If we had deployed the first version of this recipe in production, given that the chef-client usually runs on a periodic timer performing Chef runs, all our nodes would have been reinstalling the Chef Server package and reconfiguring the server every 30 minutes!

Configure Chef Server

The configuration of Chef Server has changed considerably with Chef Server 12. Now, the server does not enable a web UI by default, and you are expected to configure the Chef Server primarily through the command line.

You need to perform two actions in order to configure Chef Server 12:

  • Create an admin user
  • Create an organization

Both of these actions are now chef-server-ctl subcommands: user-create and org-create respectively.

NOTE: You may be tempted to skip ahead and install the management UI and try to configure an admin user/organization in the web UI, just like you did with Chef Server 11. Unfortunately this approach does not work with version 12.0.0. You must create one admin user and an initial organization on the command line first, then you can create the rest in the web UI.

The chef-server-ctl user-create command is used to create a user The help for the command usage follows. As of this writing the help mistakenly displays usage for the legacy knife opc user create command, but it is really now supposed to be chef-server-ctl user-create:

USAGE: knife opc user create USERNAME FIRST_NAME [MIDDLE_NAME] LAST_NAME EMAIL PASSWORD
-f, --filename FILENAME          Write private key to FILENAME rather than STDOUT

The chef-server-ctl org-create command is used to create an organization. The help for the command usage follows. It currently has a similar issue with the help referencing the legacy command, similar to user-create:

USAGE: knife opc org create ORG_SHORT_NAME ORG_FULL_NAME (options)
-f, --filename FILENAME          Write validator private key to FILENAME rather than STDOUT

In both cases, use the --filename parameter to write the *.pem files containing the user and organization keys. By default, they are just echoed to STDOUT.

Login to the server-centos65 instance to create the first admin user and the first organization. I created an admin user for myself, just like I did in Chapter 9 of Learning Chef. Here’s the results of the commands I ran:

$ kitchen login server-centos65
Last login: Wed Nov 26 01:59:12 2014 from 10.0.2.2
Welcome to your Packer-built virtual machine.
[vagrant@server-centos65 ~]$ sudo chef-server-ctl user-create misheska Mischa Taylor mischa@misheska.com chefrocks --filename misheska.pem
...
[vagrant@server-centos65 ~]$ sudo chef-server-ctl org-create learningchef "Learning Chef" --association misheska --filename learningchef-validator.pem
...
[vagrant@server-centos65 ~]$ ls *.pem
learningchef-validator.pem  misheska.pem
[vagrant@server-centos65 ~]$ exit
logout
Connection to 127.0.0.1 closed.

NOTE: You’ll need sudo or root access to run the user-create and org-create commands, because they need access to the default superuser key owned by root. This key is located in /etc/opscode/pivotal.pem.

After you have created the <username>.pem and <organization>-validator.pem files to the chef-repo/.chef directory on your host. One way to do this is to use the scp command to copy the files. Here’s what I did to create these files on my host after making chef-repo the current working directory:

$ mkdir .chef
$ scp -o stricthostkeychecking=no vagrant@192.168.38.34:/home/vagrant/misheska.pem .chef/misheska.pem
vagrant@192.168.38.34's password: vagrant
$ scp -o stricthostkeychecking=no vagrant@192.168.38.34:/home/vagrant/learningchef-validator.pem .chef/learningchef-validator.pem
vagrant@192.168.38.34's password: vagrant

For the initial organization, you’ll need to create your own knife.rb file by hand. Here’s the knife.rb file I used:

chef-repo/.chef/knife.rb
1
2
3
4
5
6
7
8
9
10
11
12
current_dir = File.dirname(__FILE__)
log_level                :info
log_location             STDOUT
node_name                "misheska"
client_key               "#{current_dir}/misheska.pem"
validation_client_name   "learningchef-validator"
validation_key           "#{current_dir}/learningchef-validator.pem"
chef_server_url          "https://server-centos65.vagrantup.com/\
organizations/learningchef"
cache_type               'BasicFile'
cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )
cookbook_path            ["#{current_dir}/../cookbooks"]

The chef_server_url field in the knife.rb uses a fake DNS hostname of server-centos65.vagrantup.com because that’s the hostname vagrant set up. If you try to visit the URL https://server-centos65.vagrantup.com/organization/learningchef, you will discover that it is not valid.

Chef Server requires that hosts have valid fully qualified domain names set up in your local domain name service (DNS). In production, you would have your Chef Server hosntame configured in your Domain Name System (DNS) server before installing Chef Server. Let’s add a temporary host entry for server-centos65.vagrantup.com in your local host database in lieu of making a DNS change, as we are just doing a book exercose.

Run one of the following commands to add a host entry. Following are the commands I ran on my machine. If you used an IP address other than 192.168.38.34, make sure it matches when you run the command.

Linux/Mac OS X:

$ sudo sh -c "echo '192.168.38.34 server-centos65.vagrantup.com' >> /etc/hosts"

Windows Command Prompt:

> echo 192.168.38.34 server-centos65.vagrantup.com >> %WINDIR%\System32\Drivers\Etc\Hosts

Windows PowerSHell:

PS> ac -Encoding UTF8 $env:windor\system32\drivers\etc\hosts "192.168.38.34 server-centos65.vagrantup.com"

Now if you try to visit https://default-centos65.vagrantup.com in your web browser, your local host should think that this is a valid hostname.

Testing the Connection

You should run the following commands from inside the Chef repository. Open your termianl or command prompt, and make chef-repo the current working directory. If you placed your chef-repo in a different location, use that instead:

$ cd ~/chef-repo

Now you can use knife, the command-line tool for Chef Server, to test your connection and authentication against Chef Server. At the time of this writing, Chef does not provide a “connection test” command. However, asking Chef Server to list the clients will very

  • Your network can connect to Chef Server.
  • The authentication files are in the correct location.
  • The authentication files can be read by Chef.
  • The response from Chef Server is received by your workstation.

Issue the knife client list command on your terminal. You should see the following:

$ knife client list
learningchef-validator

If you get an error, checking the following:

  1. You can access https://server-centos65.vagrantup.com:443 from a web browser.
  2. You are running commands from inside the chef-repo directory.
  3. The .chef directory contains two .pem files and a knife.rb.
  4. Your authentication fiels have the correct file permissions (they should be only user-readable).
  5. You are using the Chef Development Kit 0.3.5 and/or chef-client 11.18.0 (or higher). These tools needed some updates to work properly with Chef Server 12.

If you have confirmed the preceding steps and are still unable to connect to Chef Server, please consult the Chef online documentation.

From this point forward, the steps for bootstrapping a node are the same as with Chef Server 11. Refer to the “Bootstrapping a Node” section in Chapter 9 of Learning Chef for more information.

Installing the web UI

The web UI is now a premium feature of Chef Server. It is not installed by default. To install the web UI on your Chef Server, run the following commands to install the opscode-manage plugin and reconfigure both the web UI configuration and the Chef Server configuration to use the web UI:

$ cd cookbooks/chef-server
$ kitchen login
Last login: Wed Nov 26 04:09:56 2014 from 10.0.2.2
Welcome to your Packer-built virtual machine.
[vagrant@server-centos65 ~]$ sudo chef-server-ctl install opscode-manage
...
[vagrant@server-centos65 ~]$ sudo opscode-manage-ctl reconfigure
...
[vagrant@server-centos65 ~]$ sudo chef-server-ctl reconfigure
...
[vagrant@server-centos65 ~]$ exit
logout
Connection to 127.0.0.1 closed.

Once you have configured Chef Server to use the web UI, vist https://server-centos65.vagrantup.com. You should see something resembling the following screenshot. Since you already created an admin account, click on the Click here to sign in link:

Clicking on the link will take you to https://server-centos65.vagrantup.com/login where you can sign in with your administrator account, as shown in the following:

From there, you can access the management UI for Chef Server 12!

Conclusion

This blog post covered all the relevant changes needed to adapt the material presented in Chapter 9 of the Learning Chef book for Chef Server 12. Thankfully, besides the server configuration steps, not much changed.

In addition to the material presented in this article, you might want to consider automating the creation of the admin user and organization in your Chef cookbook. To see how this might be done, take a look at the Chef cookbook I use to demo Chef Server 12 for clients at https://github.com/misheska-cookbooks/chef-server12.

Survey of Test Kitchen Providers

Update November 10, 2014 * Update for DigitalOcean 0.8x provider using API V2

Introduction

Test Kitchen supports a wide variety of different providers via Test Kitchen drivers besides the default kitchen-vagrant driver. In this post, we’ll cover several popular alternatives.

Test Kitchen drivers are gem libraries available for download from http://rubygems.org . Use the kitchen driver discover command to list all the Test Kitchen gems currently available. Here is a list of all the Test Kitchen drivers as of this writing:

$ kitchen driver discover
    Gem Name                          Latest Stable Release
    kitchen-all                       0.2.0
    kitchen-ansible                   0.0.1
    kitchen-azure                     0.1.0
    kitchen-bluebox                   0.6.2
    kitchen-cabinet                   3.0.0
    kitchen-cloudstack                0.10.0
    kitchen-digital_ocean             0.3.0
    kitchen-digitalocean              0.8.0
    kitchen-docker                    1.5.0
    kitchen-docker-api                0.4.0
    kitchen-driver-vagrant_provision  1.0.0
    kitchen-ec2                       0.8.0
    kitchen-fifo                      0.1.0
    kitchen-fog                       0.7.3
    kitchen-gce                       0.2.0
    kitchen-goiardi                   0.1.1
    kitchen-inspector                 1.3.0
    kitchen-joyent                    0.1.1
    kitchen-libvirtlxc                0.4.0
    kitchen-local                     0.0.1
    kitchen-lxc                       0.0.1
    kitchen-openstack                 1.6.0
    kitchen-puppet                    0.0.13
    kitchen-rackspace                 0.12.0
    kitchen-rightscale                0.1.0
    kitchen-salt                      0.0.19
    kitchen-scribe                    0.3.1
    kitchen-sharedtests               0.2.0
    kitchen-ssh                       0.0.4
    kitchen-sshgzip                   0.0.3
    kitchen-sync                      1.0.1
    kitchen-vagrant                   0.15.0
    kitchen-vagrant_sandbox           0.1.1
    kitchen-vagrant_winrm             0.1.1
    kitchen-zcloudjp                  0.5.0
    test-kitchen-provisioners         0.1

By default, Test Kitchen defaults to using the kitchen-vagrant driver. When you run the kitchen init command to add Test Kitchen support to a project, you can add the --driver=<gem_name> option to have Test Kitchen generate configuration files using another driver of your choice. For example, the following command would use the kitchen-azure driver:

kitchen init --create-gemfile --driver=kitchen-azure

As shown in the following diagram the environments supported by Chef-releated drivers fall into four different categories: desktop virtual machines, public/private cloud providers, Linux containers and physical machines. We’ll cover representative examples from each category in this appendix.

Desktop Virtualization

Test Kitchen uses the kitchen-vagrant driver to work with desktop virtualization providers, like VirtualBox, VMWare Fusion, VMWare Workstation and Hyper-V. Since the kitchen-vagrant driver is just a shim on top of Vagrant for Test Kitchen, any provider that Vagrant supports should be supported by the kitchen-vagrant driver.

It is important to clarify that as of this writing, the kitchen-vagrant driver assumes that the virtualization provider is installed locally on the host machine. As shown in the following diagram, using the kitchen-vagrant driver, Test Kitchen creates a sandbox environment virtual machine locally on your host:

  1. Test Kitchen invokes the kitchen-vagrant driver to create a virtual machine instance.
  2. In the case of the kitchen-vagrant driver, Vagrant itself contains all the logic to work with different types of virtualization software. The kitchen-vagrant is just a small shim to allow Test Kitchen to use Vagrant to work with virtual machine instances. In this example, Vagrant uses the VirtualBox API to spin up a virtual machine instance for our sandbox environment.
  3. Once the sandbox environment is running, Test Kitchen links the instance for communication.

Test Kitchen treats the data center versions of VMware, like vCenter/vSphere/ESXi as a cloud provider. To Test Kitchen the data center editions are handled as if there were cloud instances, as vCenter/vSphere/ESXi merely a private cloud on a local LAN or corporate WAN instead of a public cloud over the Internet. As of this writing, the kitchen-openstack and kitchen-ssh drivers support vSphere data center virtualization with Test Kitchen.

kitchen-vagrant with VMware Fusion/VMware Workstation desktop virtualization

You can use VMware desktop virutalization with kitchen-varant instead of Oracle VM VirtualBox. It requires the purchase of the Vagrant VMware plugin from https://www.vagrantup.com/vmware which, at the time of this writing, costs USD $79 per seat. The VMware plugin works with VMware Workstation 9 and 10 on Windows/Linux and VMware Fusion 5, 6 and 7 on Mac OS X.

On Mac OS X/Linux, you may have multiple virtualization solutions installed alongside VMware. On these platforms, you can use both VMware and VirtualBox baseboxes at the same time, for example, if you have enough system resources. On Windows, you must make a choice, as only one virtualization solution can be installed at a time.

Once you have purchased the VMware plugin and received a license file, you can install the Vagrant plugin and license with the following:

For VMware Workstation (on Windows/Linux):

$ vagrant plugin install vagrant-vmware-workstation
$ vagrant plugin license vagrant-vmware-workstation license.lic

For VMware Fusion (on Mac OS X):

$ vagrant plugin install vagrant-vmware-fusion
$ vagrant plugin license vagrant-vmware-fusion license.lic

After you install the VMware plugin and license file and want to use VMware, you’ll need to get VMware baseboxes. Currently VirtualBox and VMware baseboxes are not interchangeable.

Once the VMware plugin and license has been installed, you’ll need to change your .kitchen.yml files slightly for VMware. You can specify the VMware provider name in the platforms section of your .kitchen.yml file.

Modify the .kitchen.yml file, adding a provider: line to the platforms driver section. If you are using VMware Workstation, use the vmware_workstation provider name. For VMware Fusion, the provider name should be vmware_fusion. You’ll also need to change the box_url line to point at a box file which has Vmware Tools installed, as box files are not guest tool agnostic. For this book, box files have been provided for both VMware and VirtualBox via VagrantCloud, so you can use the same box_url line.

Synced folders work the same as with VirtualBox. Just add a synced_folders: block to the driver: section with a list of folders to map between the guest and the host. Each entry in the list contains an array with two parameters. The first parameter is a path to the directory on the host machine. If the path is relative, it is relative to the .kitchen.yml file. The second parameter is an absolute path specifying where the folder is shared on the guest machine. The .kitchen.yml examples that follow map the current working directory on the host to the directory /vagrant on the guest, like so:

...
       synced_folders:
       - [".", "/vagrant"]
...

VMware Workstation .kitchen.yml example:

vmware/workstation/.kitchen.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
 ---
driver:
  name: vagrant

provisioner:
  name: chef_solo

platforms:
  - name: centos65
    driver:
      provider: vmware_workstation
      box: learningchef/centos65
      box_url: learningchef/centos65
      synced_folders:
        - [".", "/vagrant"]

suites:
  - name: default
    run_list:
    attributes:

VMware Fusion .kitchen.yml example:

vmware/fusion/.kitchen.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
 ---
driver:
  name: vagrant

provisioner:
  name: chef_solo

platforms:
  - name: centos65
    driver:
      provider: vmware_fusion
      box: learningchef/centos65
      box_url: learningchef/centos65
      synced_folders:
        - [".", "/vagrant"]

suites:
  - name: default
    run_list:
    attributes:

Once you modify the .kitchen.yml file appropriately the kitchen create, kitchen converge, etc. commands will use VMware instead of VirtualBox:

$ kitchen create default-centos65
-----> Starting Kitchen (v1.2.2.dev)
-----> Creating <default-centos65>...
       Bringing machine 'default' up with 'vmware_fusion' provider...
       ==> default: Cloning VMware VM: 'learningchef/centos65'. This can take some time...
       ==> default: Checking if box 'learningchef/centos65' is up to date...
       ==> default: Verifying vmnet devices are healthy...
       ==> default: Preparing network adapters...
       ==> default: Fixed port collision for 22 => 2222. Now on port 2200.
       ==> default: Starting the VMware VM...
       ==> default: Waiting for the VM to finish booting...
       ==> default: The machine is booted and ready!
       ==> default: Forwarding ports...
           default: -- 22 => 2200
       ==> default: Setting hostname...
       ==> default: Configuring network adapters within the VM...
       ==> default: Waiting for HGFS kernel module to load...
       ==> default: Enabling and configuring shared folders...
           default: -- /Users/misheska/github/learningchef/learningchef-code/chapa01/vmware/fusion: /vagrant
       ==> default: Machine not provisioning because `--no-provision` is specified.
       Vagrant instance <default-centos65> created.
       Finished creating <default-centos65> (0m39.42s).
-----> Kitchen is finished. (0m39.66s)

Test Kitchen Cloud Drivers

The following diagram shows how the Test Kitchen cloud drivers create a sandbox environment. The main difference between using a cloud provider and desktop virtualization is that the sandbox environment lives remotely on another machine. Test Kitchen communicates with the sandbox environment remotely over SSH, usually on the Internet.

  1. Test Kitchen invokes the specified driver (like kitchen-ec2) to create an instance on the cloud provider. Cloud provider drivers communicate with the cloud provider using the appropiate cloud API. Normally this is an HTTP API.
  2. The cloud provider spins up an instance to serve as our sandbox environment.
  3. Once the sandbox environment is running, Test Kitchen links the instance to your local development workstation for remote communication, usually over SSH. All Test Kitchen commands work with the remote sandbox environment transparently. As far as the user experience with Test Kitchen goes, it behaves as if it were a local desktop virtualization environment.

As of this writing, all of the Test Kitchen Cloud drivers do not support synchronized folders. All kitchen commands automatically copy your project files to the sandbox environment, as Test Kitchen uses scp to transfer files from your host to the remote cloud instance. For any other file sharing beyond what is supported by Test Kitchen, you’ll need to use a Cloud Provider-specific mechanism, such as Amazon Elastic Block Store (EBS).

DigitalOcean Cloud Provider (kitchen-digitalocean)

kitchen-digitalocean Setup

Go to https://cloud.digitalocean.com/settings/applications and click on the Generate new token button to generate a new Personal Access Token using the v2.0 API. Make sure you check the optional Write scope when you generate the token. Write scope is necessary for the DigitalOcean Cloud provider to function correctly.

Record the personal access token that is generated when you click on the Generate new token button, as shown below:

Add the new access token to your ~/.bash_profile (or equivalent for your platform) as the environment variable DIGITALOCEAN_ACCESS_TOKEN:

export DIGITALOCEAN_ACCESS_TOKEN="1234567890abcdefg"

Collect SSH public keys from the computers which need access to your sandbox instances. Visit https://cloud.digitalocean.com/ssh_keys and add the SSH keys. Once you’ve added the SSH keys(s), use the following curl command to get the DigitalOcean SSH Key IDs:

curl -X GET https://api.digitalocean.com/v2/account/keys -H "Authorization: Bearer $DIGITALOCEAN_ACCESS_TOKEN"

Record the id field for each of your SSH keys. Add the list of SSH Key IDs to the environment variable DIGITALOCEAN_SSH_KEY_IDS. If you have more than one SSH key ID, separate each ID by a comma followed by a space:

export DIGITALOCEAN_SSH_KEY_IDS="12345, 67890"

Run the following kitchen init command to add Test Kitchen support to your project using the kitchen-digitalocean driver:

$ kitchen init --driver=kitchen-digitalocean --create-gemfile
      create  .kitchen.yml
      create  test/integration/default
      create  Gemfile
      append  Gemfile
      append  Gemfile
You must run `bundle install' to fetch any new gems.

Run bundle install to download and install any required gems.

kitchen-digitalocean .kitchen.yml Example

As of the 0.8.x release, the kitchen-digitalocean provider automatically looks for the access token in the DIGITAL_ACCESS_TOKEN and the ssh key IDs in the DIGITALOCEAN_SSH_KEY_IDS environment variables. Since the access token and SSH key IDs are sensitive information, it is recommended that you store them in these environment variables instead of directly in your .kitchen.yml file. This way, you can share your .kitchen.yml file with others and store it in source control.

Here is an example kitchen.yml which spins up a CentOS 6.5 sandbox environment, loading the Access Token and SSH Key IDs from corresponding environment variables:

digitalocean/.kitchen.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
 ---
driver:
  name: digitalocean

provisioner:
  name: chef_zero

platforms:
  - name: centos65
    driver:
      name: digitalocean
      image: centos-6-5-x64

suites:
  - name: default
    run_list:
    attributes:

Before running any Test Kitchen commands, make sure you set the appropriate environment variables as shown below (with your own values):

Linux and Mac OS X:

export DIGITALOCEAN_ACCESS_TOKEN="01234567890abcdef01234567890abcdef"
export DIGITALOCEAN_SSH_KEY_IDS="12345, 67890"

Windows Command Prompt:

set DIGITALOCEAN_ACCESS_TOKEN=01234567890abcdef01234567890abcdef
set DIGITALOCEAN_SSH_KEY_IDS=12345, 67890

Windows Powershell:

$env:DIGITALOCEAN_ACCESS_TOKEN="01234567890abcdef01234567890abcdef"
$env:DIGITALOCEAN_SSH_KEY_IDS="12345, 67890"

The output of kitchen list should resemble the following:

$ kitchen list
Instance           Driver        Provisioner  Last Action
default-centos65  Digitalocean   ChefZero     <Not Created>

Spin up the node with kitchen create:

$ kitchen create default-centos65
-----> Starting Kitchen (v1.2.1)
-----> Creating <default-centos65>...
       Digital Ocean instance <3129943> created.
       Waiting for 192.241.185.202:22...
       Waiting for 192.241.185.202:22...
       (ssh ready)

       Finished creating <default-centos65> (2m42.61s).
-----> Kitchen is finished. (2m42.82s)

Install Chef Client with kitchen setup. kitchen destroy will delete your Droplet on DigitalOcean.

Refer to the kitchen-digitalocean driver documentation on https://github.com/test-kitchen/kitchen-digitalocean for more information on additional .kitchen.yml settings.

Amazon EC2 Cloud Provider (kitchen-ec2)

kitchen-ec2 Setup

In order to use the kitchen-ec2 driver, you’ll need to create an Amazon Web Services access key, consisting of an access key ID plus a secret key. You can create a new access key ID and secret _key or retrieve an existing access key ID on the AWS Identity and Access Management (IAM) page in the AWS Console. Once you select a user, click on the Manage Access Keys button as shown in the following:

In the Manage Access keys dialog, click on the Create Access Key button to create a new access key ID and secret _key as shown in the following:

AWS will create your access key. You can click on Show User Security Credentials to display the Access Key ID and the Secret Access Key. Make note of these as this is the last time they will be displayed. You can also click on the Download Credentials button to download the credentials as a .csv file as shown below:

Create a key pair to use when you launch instances. Amazon EC2 supports a variety of ways to work with key pairs. Refer to http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html for more information.

Make sure you set permissions on the key pair. Otherwise kitchen-ec2 will ignore the file.

chmod 400 my-key-pair.pem

Run the following kitchen init command to add Test Kitchen support to your project using the kitchen-ec2 driver:

$ kitchen init --driver=kitchen-ec2 --create-gemfile
      create  .kitchen.yml
      create  test/integration/default
      create  Gemfile
      append  Gemfile
      append  Gemfile
You must run `bundle install' to fetch any new gems.

Run bundle install to fetch any new gems.

kitchen-ec2 .kitchen.yml Example

Since the Access Key ID, Secret Access Key and SSH Key ID contain sensitive information, it is recommended that you store these values in environment variables instead of directly in your .kitchen.yml file. This way, you can share your .kitchen.yml file with others and store it in source control. You can use embedded Ruby templates in a .kitchen.yml file to load values from the environment. Here is an example kitchen.yml which spins up a CentOS 6.5 sandbox environment, loading the Access Key ID, Secret Acces Key and SSH Key ID from corresponding environment variables:

ec2/.kitchen.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
 ---
driver:
  require_chef_omnibus: true
  name: ec2
  aws_access_key_id: "<%= ENV['AWS_ACCESS_KEY_ID']%>"
  aws_secret_access_key: "<%= ENV['AWS_SECRET_ACCESS_KEY']%>"
  aws_ssh_key_id: "<%= ENV['AWS_SSH_KEY_ID']%>"
  ssh_key: "<%= ENV['AWS_SSH_KEY']%>"

provisioner:
  name: chef_solo

platforms:
  - name: centos65
    driver:
      image_id: ami-8997afe0
      username: root
      region: us-east-1
      availability_zone: us-east-1c

suites:
  - name: default
    run_list:
    attributes:

Before running any Test Kitchen commands, make sure you set the appropriate environment variables as shown below (with your own values):

Linux and Mac OS X:

export AWS_ACCESS_KEY_ID="ABCDEFGHI123JKLMNOPQ"
export AWS_SECRET_ACCESS_KEY="abcdefghijklmnopqrstuvwyz"
export AWS_SSH_KEY_ID="keyid1234"
export AWS_SSH_KEY="$HOME/ec2/$AWS_SSH_KEY_ID.pem"

Windows Command Prompt:

set AWS_ACCESS_KEY_ID=ABCDEFGHI123JKLMNOPQ
set AWS_SECRET_ACCESS_KEY=abcdefghijklmnopqrstuvwyz
set AWS_SSH_KEY_ID=keyid1234
set AWS_SSH_KEY=%USERPROFILE%/ec2/%AWS_SSH_KEY_ID%.pem

Windows Powershell:

$env:AWS_ACCESS_KEY_ID="ABCDEFGHI123JKLMNOPQ"
$env:AWS_SECRET_ACCESS_KEY="abcdefghijklmnopqrstuvwyz"
$env:AWS_SSH_KEY_ID="keyid1234"
$env:AWS_SSH_KEY="$env:userprofile/ec2/$env:aws_ssh_key_id.pem"

The output of kitchen list should resemble the following:

$ kitchen list
Instance           Driver  Provisioner  Last Action
default-centos65  Ec2     ChefSolo     <Not Created>

Spin up the node with kitchen create:

$ kitchen create default-centos65
-----> Starting Kitchen (v1.2.2.dev)
-----> Creating <default-centos65>...
       EC2 instance <i-5b6f2b70> created.
...........       (server ready)
       Waiting for ec2-54-197-34-184.compute-1.amazonaws.com:22...
       Waiting for ec2-54-197-34-184.compute-1.amazonaws.com:22...
       Waiting for ec2-54-197-34-184.compute-1.amazonaws.com:22...
       Waiting for ec2-54-197-34-184.compute-1.amazonaws.com:22...
       (ssh ready)\n
       Finished creating <default-centos65> (3m2.97s).
-----> Kitchen is finished. (3m3.40s)

NOTE:

You may be prompted to opt in and accept the terms and subscribe to using the AWS Marketplace CentOS image the first time you spin up an image. The kitchen-ec2 driver will provide you with a link to the opt in URL.

NOTE:

You might not be able to create CentOS images in all availability zones. The kitchen-ec2 driver will advice you of your availability zone options if there is an issue with your availability zone choice.

Install Chef Client with kitchen setup. kitchen destroy will delete your EC2 instance.

Refer to the kitchen-ec2 driver documentation on https://github.com/test-kitchen/kitchen-ec2 for more information additional .kitchen.yml settings.

Google Compute Engine Cloud Provider (kitchen-gce)

kitchen-gce Setup

Create a Google Compute Engine project in the Google Developers Console at https://console.developers.google.com. Create a Service Account Key by navigating to APIs & auth > Credentials. Under OAuth start the process by clicking on the CREATE NEW CLIENT ID button as shown here:

On the Create Client ID dialog, choose Service account then click on Create Client ID as shown below. This will generate a private key file along with a password. Record this information, as it is the only time it will be displayed.

Make note of the Email address field for the Service Account (not to be confused with the project owner’s Email Address at the top of the page) as shown in the following. You’ll be recording this in the google_client_email field in the .kitchen.yml.

If you do not already have an SSH key pair to login, create them using ssh-keygen or an equivalent tool. Register the public key in the Google Developer Console. The default file name for a public key is $HOME/.ssh/id_rsa.pub. Navigate to Compute > Compute Engine > Metadata on the Google Developers Console. Make sure the SSH_keys is selected in the panel on the right, then click on the Add SSH key button as shown in the following:

Copy the public key id_rsa.pub file contents to the clipboard and paste it into the Enter entire key data field. Click on the Done button to save.

Run the following kitchen init command to add Test Kitchen support to your project using the kitchen-gce driver:

$ kitchen init --driver=kitchen-gce --create-gemfile
      create  .kitchen.yml
      create  test/integration/default
      create  Gemfile
      append  Gemfile
      append  Gemfile
You must run `bundle install' to fetch any new gems.

Run bundle install to fetch any new gems.

kitchen-gce .kitchen.yml Example

Since the project, client e-mail and key location are sensitive information and differ between users, it is recommended that you store them in environment variables instead of directly in your .kitchen.yml file. This way, you can share your .kitchen.yml file with others and store it in source control. You can use an embedded Ruby template in a .kitchen.yml file to load values from the environment. Here is an example kitchen.yml which spins up a CentOS 6.5 sandbox environment, loading the project and client e-mail from corresponding environment variables:

gce/.kitchen.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
 ---
driver:
  name: gce
  google_project: "<%= ENV['GOOGLE_PROJECT']%>"
  google_client_email: "<%= ENV['GOOGLE_CLIENT_EMAIL']%>"
  google_key_location: "<%= ENV['GOOGLE_KEY_LOCATION']%>"

provisioner:
  name: chef_solo

platforms:
  - name: centos65
    driver: gce
      area: us
      image_name: centos-6-v20140619

suites:
  - name: default
    run_list:
    attributes:

Before running any Test Kitchen commands, make sure you set the appropriate environment variables as shown below (with your own values):

Linux and Mac OS X:

export GOOGLE_PROJECT="alpha-bravo-123"
export GOOGLE_CLIENT_EMAIL="123456789012@developer.gserviceaccount.com"
export GOOGLE_KEY_LOCATION="$HOME/gce/1234567890abcdef1234567890abcdef12345678-privatekey.p12"

Windows Command Prompt:

set GOOGLE_PROJECT=alpha-bravo-123
set GOOGLE_CLIENT_EMAIL=123456789012@developer.gserviceaccount.com
set GOOGLE_KEY_LOCATION=%USERPROFILE%/gce/1234567890abcdef1234567890abcdef12345678-privatekey.p12

Windows Powershell:

$env:GOOGLE_PROJECT="alpha-bravo-123"
$env:GOOGLE_CLIENT_EMAIL="123456789012@developer.gserviceaccount.com"
$env:GOOGLE_KEY_LOCATION="$env:userprofile/gce/1234567890abcdef1234567890abcdef12345678-privatekey.p12"

The output of kitchen list should resemble the following:

$ kitchen list
Instance          Driver  Provisioner  Last Action
default-centos65  Gce     ChefSolo     <Not Created>

Spin up the node with kitchen create:

$ kitchen create default-centos65
-----> Starting Kitchen (v1.2.2.dev)
-----> Creating <default-centos65>...
       GCE instance <default-centos65-31681aab-e6a2-494b-99cb-9b920a1f6284> created.
..       (server ready)
       (ssh ready)
       Finished creating <default-centos65> (1m26.70s).
-----> Kitchen is finished. (1m28.18s)

Install Chef Client with kitchen setup. kitchen destroy will delete your Google Compute Engine instance.

Refer to the kitchen-gce driver documentation on https://github.com/anl/kitchen-gce for more information on additional .kitchen.yml settings.

Rackspace Cloud Provider (kitchen-rackspace)

kitchen-rackspace Setup

Login to the Cloud Sites Control Panel at https://manage.rackspacecloud.com/pages/Login.jsp Navigate to Your Account > API Access to display your username and API key as shown in below:

Run the following kitchen init command to add Test Kitchen support to your project using the kitchen-rackspace driver:

$ kitchen init --driver=kitchen-rackspace --create-gemfile
      create  .kitchen.yml
      create  test/integration/default
      create  Gemfile
      append  Gemfile
      append  Gemfile
You must run `bundle install' to fetch any new gems.

Run bundle install to fetch any new gems.

kitchen-rackspace .kitchen.yml Example

Since the username and API Key are sensitive information and differ between users, it is recommended that you store them in environment variables instead of directly in your .kitchen.yml file. This way, you can share your .kitchen.yml file with others and store it in source control. You can use an embedded Ruby template in a .kitchen.yml file to load values from the environment. Here is an example kitchen.yml which spins up a CentOS 6.5 sandbox environment, loading the project and client e-mail from corresponding environment variables:

rackspace/.kitchen.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
 ---
driver:
  require_chef_omnibus: true
  name: rackspace
  rackspace_username: "<%= ENV['RACKSPACE_USERNAME']%>"
  rackspace_api_key: "<%= ENV['RACKSPACE_API_KEY']%>"
  public_key_path: "<%= ENV['RACKSPACE_PUBLIC_KEY_PATH']%>"

provisioner:
  name: chef_solo

platforms:
  - name: centos65
    driver: rackspace
      image_id: "592c879e-f37d-43e6-8b54-8c2d97cf04d4"
      flavor_id: "performance1-1"

suites:
  - name: default
    run_list:
    attributes:

Before running any Test Kitchen commands, make sure you set the appropriate environment variables as shown below (with your own values):

Linux and Mac OS X:

export RACKSPACE_USERNAME="alice"
export RACKSPACE_API_KEY="abcdef0123456789abcdef0123456789"
export RACKSPACE_PUBLIC_KEY_PATH="$HOME/.ssh/id_rsa.pub"

Windows Command Prompt:

set RACKSPACE_USERNAME=alice
set RACKSPACE_API_KEY=abcdef0123456789abcdef0123456789
set RACKSPACE_PUBLIC_KEY_PATH=%USERPROFILE%/.ssh/id_rsa.pub

Windows Powershell:

$env:RACKSPACE_USERNAME="alice"
$env:RACKSPACE_API_KEY="abcdef0123456789abcdef0123456789"
$env:RACKSPACE_PUBLIC_KEY_PATH="$env:userprofile/.ssh/id_rsa.pub"

The output of kitchen list should resemble the following:

$ kitchen list
Instance          Driver     Provisioner  Last Action
default-centos65  Rackspace  ChefSolo     <Not Created>

Spin up the node with kitchen create:

$ kitchen create default-centos65
-----> Starting Kitchen (v1.2.2.dev)
-----> Creating <default-centos65>...
       Rackspace instance <9456b985-3a41-4cb0-a3cf-7536cc15baf7> created.
       (server ready)
       (ssh ready)
       Finished creating <default-centos65> (0m37.77s).
-----> Kitchen is finished. (0m38.21s)

Then install Chef Client with kitchen setup. kitchen destroy will delete your instance on Rackspace.

Refer to the kitchen-gce driver documentation on https://github.com/test-kitchen/kitchen-rackspace for more information on additional .kitchen.yml settings.

Linux Container Drivers

You can regard Linux Containers to be a resource-efficient variant of virtual machines. As shown in the following diagram, Linux containers trade off the flexibility (and overhead) of being able to run different operating systems in each guest to minimize resource consumption by having all guests share the same OS kernel. In container environments, guests are isolated like virtual machines using more lightweight mechanisms around Linux processes instead.

This idea has its origins in attempts to provide better process isolation to chroot jails. chroot is a Unix command that facilitates creating a separate virtualized copy of the operating system by changing the apparent root directory (/) to processes running within this copy of the operating system. Other variants of Unix have added extensions to this chroot mechanism to provide better isolation of the guest process, such as FreeBSD jails and Solaris Containers. Linux Containers bring this process-based isolation mechanism to the standard Linux kernel via a recently added kernel feature called control groups.

As of this writing, there are no container-like Test Kitchen drivers for Windows. Microsoft is working on adding similar lightweight virtualization technology to Windows via its Drawbridge virtalization technology[http://research.microsoft.com/en-us/projects/drawbridge/]. The only equivalent to Linux Containers in Windows at this moment is Microsoft Applications Virtualization (App-V), which has been around for quite some time, but it has a major drawback in requiring modification of target applications in order to work with the system, so it is not widely used.

The following diagram shows the steps in the sandbox environment creation process for containers. It is identical to the host-based model presented previously, just using lightweight, isolated container processes instead of full-blown virtual machines.

  1. Test Kitchen invokes the container driver (kitchen-docker or kitchen-lxc) to create a container instance.
  2. The Test Kitchen driver uses the operating system APIs for Linux Containers to create a new instance for our sandbox environment.
  3. Once the sandbox environment is running, Test Kitchen links the instance for communication.

As of this writing, Test Kitchen drivers for Linux Containers do not support functionality equivalent to synchronized folders. All Test Kitchen commands use scp to transfer files from your host to the container instance. For any other file sharing beyond what is supported by Test Kitchen, you’ll need to make direct use of the file sharing mechanisms provided by the container driver being used. This is where Docker shines, as it supports data volume containers which bypass container image layering. Data volume containers are an ideal way to share data between containers. It is also possible to mount host directories in a container, but that has more limited use cases. Refer to the documentation on your container provider for more information.

You can combine together virtual machines with Linux containers to use containers on platforms that do not have native container support, like Mac OS X and Windows. The following diagram presents an overview of the setup. With virtual machines, it is usually not possible to nest virtualization software instances. Running virtualization software inside guest OS instances is either prohibited or painfully slow. However, it’s perfectly fine to run Linux Containers within a virtual machine. To the outer virtualization software, the container instances are merely Linux processes.

In the next section on Docker, we’ll show you how to use this technique for readers running Mac OS X or Windows. Neither platform supports Linux containers natively on the host. Chef Software uses a Docker-based VM in training classes, so that students with laptops running Mac OS X or Windows can use the same setup as the students using Linux. This approach also saves money, as Chef Software uses cloud providers for training, and these providers charge based the number of instances and resources used. The lightweight Docker instances consume fewer resources and only require one running instance on the cloud provider - all the other instances are just lightweight container instances, which cloud providers (currently) do not charge extra. You may want to consider using Linux Containers in a similar fashion to save money if you make heavy use of third-party virtualization or cloud providers, like we do.

Docker Driver (kitchen-docker)

If you are using Linux, refer to the Docker installation guide for instructions on how to install and configure Docker in your environment: http://www.docker.com/.

Chef Training Environment Setup

Skip ahead to the next secion if you are using Linux and already have Docker installed. Otherwise, you’ll need to spin up a virtual machine with Docker installed in order to play around with a container environment.

We’ve created a Chef training environment that has Docker and the Chef Development Kit used in this book preinstalled on a Linux virtual machine. We use this same instance in official Chef training. It’s also a handy environment for playing around with containers using Test Kitchen.

First, make sure you install Vagrant and VirtualBox or Vagrant and VMware.

Create a directory for the Chef training environment project called chef and make it the current directory.

$ mkdir chef
$ cd chef

Add Test Kitchen support to the project using the default kitchen-vagrant driver by running kitchen init. Then run bundle install to install the necessary gems for the Test Kitchen driver.

$ kitchen init --create-gemfile
      create  .kitchen.yml
      create  test/integration/default
      create  Gemfile
      append  Gemfile
      append  Gemfile
You must run `bundle install' to fetch any new gems.

$ bundle install
Fetching gem metadata from https://rubygems.org/..........
Fetching additional metadata from https://rubygems.org/..
Resolving dependencies...
Using mixlib-shellout (1.4.0)
Using net-ssh (2.9.1)
Using net-scp (1.2.1)
Using safe_yaml (1.0.3)
Using thor (0.19.1)
Using test-kitchen (1.2.1)
Using kitchen-vagrant (0.15.0)
Using bundler (1.5.2)
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.

Modify the .kitchen.yml file to use the Chef training image as shown in the following .kitchen.yml:

chef/.kitchen.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
 ---
driver:
  name: vagrant

provisioner:
  name: chef_solo

platforms:
  - name: learningchef
    driver:
      box: learningchef/cheftraining
      box_url: learningchef/cheftraining

suites:
  - name: default
    run_list:
    attributes:

Run kitchen create to spin up the image:

$ kitchen create
-----> Starting Kitchen (v1.2.2.dev)
-----> Creating <default-learningchef>...
       Bringing machine 'default' up with 'virtualbox' provider...
       ==> default: Importing base box 'learningchef/chefdk-box'...
       ==> default: Matching MAC address for NAT networking...
       ==> default: Checking if box 'learningchef/chefdk-box' is up to date...
       ==> default: Setting the name of the VM: default-learningchef_default_1404728110875_23069
       ==> default: Fixed port collision for 22 => 2222. Now on port 2200.
       ==> default: Clearing any previously set network interfaces...
       ==> default: Preparing network interfaces based on configuration...
           default: Adapter 1: nat
       ==> default: Forwarding ports...
           default: 22 => 2200 (adapter 1)
       ==> default: Booting VM...
       ==> default: Waiting for machine to boot. This may take a few minutes...
           default: SSH address: 127.0.0.1:2200
           default: SSH username: vagrant
           default: SSH auth method: private key
           default: Warning: Remote connection disconnect. Retrying...
       ==> default: Machine booted and ready!
       ==> default: Checking for guest additions in VM...
       ==> default: Setting hostname...
       ==> default: Machine not provisioning because `--no-provision` is specified.
       Vagrant instance <default-learningchef> created.
       Finished creating <default-learningchef> (0m36.99s).
-----> Kitchen is finished. (0m37.44s)

Then run kitchen login to use Docker! Note that the image also has the latest Chef Development Kit installed (as of this writing). You will be running the Test Kitchen Docker driver inside this virtual machine. It has been pre-populated with all the necessary files to spin up the CentOS 6.5 images used in the exercises for this book:

$ kitchen login
Welcome to Ubuntu 14.04 LTS (GNU/Linux 3.13.0-24-generic x86_64)

 * Documentation:  https://help.ubuntu.com/
Welcome to the Learning Chef training environment
Last login: Fri May 23 13:49:31 2014 from 10.0.2.2
vagrant@default-learningchef:~$ docker --version
Docker version 0.11.1, build fb99f99
vagrant@default-learningchef:~$ kitchen --version
Test Kitchen version 1.2.2.dev
vagrant@default-learningchef:~$

NOTE:

Sharp-eyed readers might notice that this is an Ubuntu image. It is perfectly OK to spin up CentOS images on Ubuntu, as long as you use a version that shares the same kernel!

TIP:

At first, the multiple layers of instances might be a little confusing. Refer back to the Docker diagram shown previously so that you can keep the big picture of this setup in mind. Also, modifying the command prompts so they clearly indicate which environment is the VM and which environment is a container instance is strongly recommended.

kitchen-docker Setup

Run the following kitchen init command to add Test Kitchen support to your project using the kitchen-docker driver:

$ kitchen init --driver=kitchen-docker --create-gemfile
      create  .kitchen.yml
      create  test/integration/default
      create  Gemfile
      append  Gemfile
      append  Gemfile
You must run `bundle install' to fetch any new gems.

Run bundle install to download and install any required gems.

kitchen-docker .kitchen.yml Example

The following .kitchen.yml presents an example which spins up a CentOS 6.5 sandbox environment:

docker/.kitchen.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
 ---
driver:
  name: docker

provisioner:
  name: chef_solo

platforms:
  - name: centos65
    driver:
      image_id: 3448641
      region_id: 4

suites:
  - name: default
    run_list:
    attributes:

The output of kitchen list should resemble the following:

$ kitchen list
Instance          Driver  Provisioner  Last Action
default-centos65  Docker  ChefSolo     <Not Created>

Spin up the node with kitchen create:

$ kitchen create
-----> Starting Kitchen (v1.2.2.dev)
-----> Creating <default-centos65>...
       Step 0 : FROM centos:latest
       Pulling repository centos
        ---> 0c752394b855
...
       Waiting for localhost:49153...
       Waiting for localhost:49153...
       Finished creating <default-centos65> (1m19.28s).
-----> Kitchen is finished. (1m19.34s)

At the time of this writing, due to some issues with kitchen-docker, you may be prompted for kitchen@localhost's password. The password is kitchen

$ kitchen login
kitchen@localhost's password: kitchen
Last login: Mon Jul  7 11:37:14 2014 from 172.17.42.1
[kitchen@55f29336b435 ~]$ cat /etc/redhat-release
CentOS release 6.5 (Final)
[kitchen@55f29336b435 ~]$ exit
logout
Connection to localhost closed.

Install Chef Client with kitchen setup. kitchen destroy will delete container instance.

Refer to the kitchen-docker driver documentation on https://github.com/portertech/kitchen-docker for more information on additional .kitchen.yml settings.

Physical Machine Drivers

As of this writing, Test Kitchen does not currently support chef-metal. It is currently planned to provide robust support for managing sandbox environments running on physical machines using chef-metal (though plans sometimes change).

Until Test Kitchen supports chef-metal, the only way to use Test Kitchen with physical machines currently (other than your local host) is to use the kitchen-ssh driver. This is actually a generic way to integrate any kind of machine with Test Kitchen, not just physical machines. As long as the machine accepts ssh connections, it will work.

The following diagram shows an overview of the Test Kitchen instance creation process using kitchen-ssh. It is similar to the creation process used for cloud instances with the Test Kitchen environment being run on a remote machine, but there is only one step because an isolated sandbox instance is not created. The kitchen-ssh driver merely links up an SSH communication channel with Test Kitchen in the remote machine’s host environment.

It is assumed that you are using some other method outside of Test Kitchen to be able to easily reset the environment. Also, since it does not spin up a new instance, you will need to make sure the machine that you are linking to has CentOS 6 installed to match the exercises in this book.

Driver for any server with an SSH address (kitchen-ssh)

Run the following kitchen init command to add Test Kitchen support to your project using the kitchen-ssh driver:

$ kitchen init --driver=kitchen-ssh --create-gemfile
      create  .kitchen.yml
      create  test/integration/default
      create  Gemfile
      append  Gemfile
      append  Gemfile
You must run `bundle install' to fetch any new gems.

Run bundle install to fetch any required gems.

kitchen-ssh .kitchen.yml Example

The following .kitchen.yml assumes that you are connecting to an existing CentOS 6.5 environment with an SSH server running. Change the hostname:, username: and password: fields accordingly to match your remote machine’s settings:

ssh/.kitchen.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
 ---
driver:
  name: ssh

provisioner:
  name: chef_solo

platforms:
  - name: centos65
    driver:
      hostname: 192.168.33.33
      username: alice
      password: averysecretpassword

suites:
  - name: default
    run_list:
    attributes:

The output of kitchen list should resemble the following:

Instance          Driver  Provisioner  Last Action
default-centos65  Ssh     ChefSolo     Created

Initiate a connection to the node with kitchen create. You could also run kitchen login without needing to run kitchen create in this case, as kitchen create does nothing:

$ kitchen create
-----> Starting Kitchen (v1.2.2.dev)
-----> Creating <default-centos65>...
       Kitchen-ssh does not start your server '192.168.33.33' but will look for an ssh connection with user 'alice'
---
       Kitchen-ssh found ssh ready on host '192.168.33.33' with user 'alice'

       Finished creating <default-centos65> (0m0.01s).
-----> Kitchen is finished. (0m0.02s)

Install Chef Client with kitchen setup. For this driver, kitchen destroy does nothing, just like kitchen create, besides updating the status in Test Kitchen.

Refer to the kitchen-ssh driver documentation on https://github.com/neillturner/kitchen-ssh/blob/master/lib/kitchen/driver/ssh.rb for more information on additional .kitchen.yml settings.

Set Up a Sane Ruby Cookbook Authoring Environment for Chef on Mac OS X, Linux and Windows

Updated January 3, 2014

  • Added instructions to create subl launching commandlet

Updated January 2, 2014

  • Per Seth Vargo switched from rbenv to chruby
  • Switched from SublimeText 2 to SublimeText 3 - it works with SublimeChef

You will need to set up a sane Ruby 1.9.x development to support your Chef cookbook authoring activity on Mac OS X, Linux or Windows. In this Ruby environment, you will manage all the required Ruby Gem libraries necessary to support Chef cookbook authoring. The LearnChef site recommends that you piggyback the Chef client’s omnibus Ruby environment to start learning how to write Chef cookbooks. This article assumes that you want to go beyond the basics, where you’ll need a Ruby environment dedicated for Chef cookbook development.

There are many different ways to set up a sane Ruby environment. This article covers how to set up a sane Ruby environment for Chef using Chruby for Mac OS X/Linux and RubyInstaller for Windows. The setup covered in this article should work for most people wanting to write Chef cookbooks. If you are more experienced with Ruby development, you may want to roll your own Ruby environment in another fashion. The only requirement is that Ruby 1.9.x must be used, the Chef client currently does not support Ruby 2.x.

Mac OS X

Out of the box, Ruby does not provide a mechanism to support multiple installed versions. Chruby makes it easy to manage multiple versions of Ruby. It’s a great way to set up a dedicated Ruby 1.9.x environment with all the required Gem libraries for Chef cookbook development.


NOTE: Before trying to install Chruby verify that you do not have another popular Ruby virtualization manager installed - RVM. If you try to run the following rvm command, it should say command not found:

$ rvm --version
-bash: rvm: command not found

If you want to switch to Chruby (which is recommended), make sure that you completely remove RVM first (as Chruby and RVM cannot coexist because RVM overrides the gem command with a function specific to RVM).


Install the Xcode Command Line Tools - Mac OS X

First you’ll need to install a C compiler and the Xcode Command Line tools to build Ruby from source. If you are using the latest version of Mac OS X Mavericks 10.9, it has support for downloading the Xcode command line tools directly from a Terminal window. Run the following on a command line:

$ xcode-select --install

You will be prompted to either click on Install to just install the command line developer tools or click on Get Xcode to install both Xcode and the command line developer tools. It can be handy to have Xcode as well, but it is a rather large multi-gigabyte download and not really necessary for Ruby development. So if you want to get going quickly, just click on the Install button:

If xcode-select installed the Xcode Command Line Tools, you should have git installed (among other things). Verify this with the following command:

$ git --version
git version 1.8.3.4 (Apple Git-47)

Install the Homebrew Package Manager - Mac OS X

Next, you’ll need to install the Homebrew package manager to get all the dependencies needed to compile Ruby from source. While you could manage these dependencies by hand, using a package manager is a better idea, as package managers know how to uninstall what they install.

First verify that you DO NOT currently have homebrew installed. brew --version should report command not found.

$ brew --version
-bash: brew: command not found

If you already have Homebrew installed, just Update Homebrew and Rbenv and skip to the next section.


NOTE: Before trying to install Homebrew verify that you do not have another popular package manager installed - MacPorts. If you try to run the following port command, it should say command not found:

$ port --version
-bash: port: command not found

If MacPorts is already installed, make sure that you completely remove MacPorts from your system before trying to install Homebrew.


Run the following command to install Homebrew:

$ ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"

Run brew doctor and address any issues it discovers. When all is well, you should see:

$ brew doctor
Your system is raring to brew.

Install Apple GCC 4.2 - Mac OS X

Next, install the additional dependencies to compile Ruby from source:

# For update-system
brew update
# Add the system duplicate formulae
brew tap homebrew/dupes
# Install legacy C compiler for building Ruby
brew install apple-gcc42

Install Chruby and Ruby-Build via Homebrew - Mac OS X

Now install chruby and ruby-install via Homebrew:

$ brew update
$ brew install chruby
$ brew install ruby-build

Add chruby.sh to your shell to enable chruby:

$ echo 'source /usr/local/share/chruby/chruby.sh' >> $HOME/.bash_profile

Add auto.sh to your shell to enble auto-switching of Rubies specified by `.ruby-version files:

$ echo 'source /usr/local/share/chruby/auto.sh' >> $HOME/.bash_profile

Compile Ruby 1.9.x from source - Mac OS X

Install the latest version of ruby 1.9.x (at the time of this writing 1.9.3-p484)

$ ruby-build 1.9.3-p484 --install-dir ~/.rubies/ruby-1.9.3-p484

Reload .bashrc with these new settings:

$ source $HOME/.bash_profile    

To switch to the Ruby required for Chef:

$ chruby ruby-1.9

To make this version the default version of Ruby, simply add this command to your $HOME/.bash_profile

$ echo 'chruby ruby-1.9' >> $HOME/.bash_profile

Verify the ruby install. If everything was installed correctly, the ruby -v command should report that version 1.9.3p484 is installed.

$ ruby -v
ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-darwin13.0.0]

Install Bundler - Mac OS X

You’ll need to use Bundler to manage gems. Installing a gem is also a good way to ensure that you’ve installed most of the Ruby prerequisites.

First, make sure you update to the latest version of Rubygems:

$ gem update --system
Updating rubygems-update
Fetching: rubygems-update-2.2.0.gem (100%)
Successfully installed rubygems-update-2.2.0
Installing RubyGems 2.2.0
RubyGems 2.2.0 installed
...

Then install the bundler gem. If the gem install command reports Successfully installed you’re good to go:

$ gem install bundler
Successfully installed bundler-1.5.1
Parsing documentation for bundler-1.5.1
1 gem installed

Install Sublime Text 3 (Optional) - Mac OS X

Miguel Cabeça has written an excellent plugin for the Sublime Text editor called SublimeChef, which is the closest thing to an Integrated Development Environment (IDE) that exists for Chef. Sublime Text costs $70 for a license, but has no restriction on the length of a trial period, so feel free try out Sublime Text to see if it works for you.

Download and install SublimeText 3 for your platform via http://www.sublimetext.com/3

It is very handy to be able to launch Sublime Text from the command line as you’ll find yourself going back and forth between the two in developing your cookbooks. Create a short-named link to the Sublime Text executable with the following commands:

$ sudo mkdir -p /usr/local/bin
$ sudo chown -R $(whoami) /usr/local/bin
$ ln -s /Applications/Sublime\ Text.app/Contents/SharedSupport/bin/subl /usr/local/bin/subl

Since /usr/local/bin should already bin your path, typing in the following command should launch Sublime Text:

$ subl

You can pass in a filename parameter as well, such as:

$ subl default.rb

Install Sublime Text 3 Package Control (Optional) - Mac OS X

In order to install the SublimeChef plugin, first you need to install Sublime Text Package Control.

  • Download Package Control.sublime-package

  • Choose Preferences > Browse Packages... from the Sublime Text 3 menu

  • Browse up a folder, then navigate into Installed Packages/

  • Copy Package Control.sublime-package file you downloaded into this Sublime Text 3\Installed Packages directory

  • Restart Sublime Text 3 to enable Package Control

Once Package Control is installed successfully, you should be able to display the Command Palette by pressing COMMAND+SHIFT+P:

Install Sublime Chef (Optional) - Mac OS X

After pressing CTRL+SHIFT+P to display the Command Palette, start typing install to select Package Control: Install Package:

Then type chef to display the SublimeChef entry - click to install:

Miguel created the following demo video to show how SublimeChef can be used.

Linux

Install Prerequisite Packages - Linux

Make sure the prerequisite packages are installed.

Ubuntu prerequisites:

$ sudo apt-get update
$ sudo apt-get install -y build-essential git
$ sudo apt-get install -y libxml2-dev libxslt-dev libssl-dev

RHEL/CentOS prerequisites:

$ sudo yum update
$ sudo yum install -y git
$ sudo yum install -y gcc-c++ patch readline readline-devel zlib zlib-devel
$ sudo yum install -y libyaml-devel libffi-devel openssl-devel make bzip2
$ sudo yum install -y autoconf automake libtool bison
$ sudo yum install -y libxml2-devel libxslt-devel

Install Chruby and Ruby-Build - Linux

Download the chruby source distribution:

$ wget -O chruby-0.3.8.tar.gz https://github.com/postmodern/chruby/archive/v0.3.8.tar.gz

Extract chruby and install:

$ tar xzvf chruby-0.3.8.tar.gz
$ cd chruby-0.3.8/
$ sudo make install

Feel free to remove the chruby source dir after installing:

$ cd ..
$ rm chruby-0.3.8.tar.gz
$ rm -rf chruby-0.3.8

Install ruby-build:

$ git clone https://github.com/sstephenson/ruby-build.git
$ cd ruby-build
$ sudo ./install.sh

Feel free to remove the ruby-build source dir after installing:

$ cd ..
$ rm -rf ruby-build

Add chruby.sh to your shell to enable chruby:

$ echo 'source /usr/local/share/chruby/chruby.sh' >> $HOME/.bashrc

Add auto.sh to your shell to enble auto-switching of Rubies specified by `.ruby-version files:

$ echo 'source /usr/local/share/chruby/auto.sh' >> $HOME/.bashrc

Compile Ruby 1.9.x from source - Linux

Install the latest version of ruby 1.9.x (at the time of this writing 1.9.3-p484)

$ ruby-build 1.9.3-p484 --install-dir ~/.rubies/ruby-1.9.3-p484

Reload .bashrc with these new settings:

$ source $HOME/.bashrc

To switch to the Ruby required for Chef:

$ chruby ruby-1.9

To make this version the default version of Ruby, simply add this command to your $HOME/.bashrc

$ echo 'chruby ruby-1.9' >> $HOME/.bashrc

Verify the ruby install. If everything was installed correctly, the ruby -v command should report that version 1.9.3p484 is installed.

$ ruby -v
ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-darwin13.0.0]

Install Bundler - Linux

You’ll need to use Bundler to manage gems. Installing a gem is also a good way to ensure that you’ve installed most of the Ruby prerequisites.

First, make sure you update to the latest version of Rubygems:

$ gem update --system
Updating rubygems-update
Fetching: rubygems-update-2.2.0.gem (100%)
Successfully installed rubygems-update-2.2.0
Installing RubyGems 2.2.0
RubyGems 2.2.0 installed
...

Then install the bundler gem. If the gem install command reports Successfully installed you’re good to go:

$ gem install bundler
Successfully installed bundler-1.5.0
Parsing documentation for bundler-1.5.0
1 gem installed

Install Sublime Text 3 (Optional) - Linux

Miguel Cabeça has written an excellent plugin for the Sublime Text editor called SublimeChef, which is the closest thing to an Integrated Development Environment (IDE) that exists for Chef. Sublime Text costs $70 for a license, but has no restriction on the length of a trial period, so feel free try out Sublime Text to see if it works for you.

Download and install SublimeText 3 for your platform via http://www.sublimetext.com/3

Unlike with Mac OS X, the Linux installer should have created a command line launch link for Sublime Text in /usr/bin/subl. Typing in the following command should launch Sublime Text:

$ subl

You can pass in a filename parameter as well, such as:

$ subl default.rb

Install Sublime Text 3 Package Control (Optional) - Linux

In order to install the SublimeChef plugin, first you need to install Sublime Text Package Control.

  • Download Package Control.sublime-package

  • Choose Preferences > Browse Packages... from the Sublime Text 3 menu

  • Browse up a folder, then navigate into Installed Packages/

  • Copy Package Control.sublime-package file you downloaded into this Sublime Text 3/Installed Packages directory

  • Restart Sublime Text 3 to enable Package Control

Once Package Control is installed successfully, you should be able to display the Command Pallete by pressing CTRL+SHIFT+P:

Install Sublime Chef (Optional) - Linux

After pressing CTRL+SHIFT+P to display the Command Pallette, start typing install to select Package Control: Install Package:

Then type chef to display the SublimeChef entry - click to install:

Miguel created the following demo video to show how SublimeChef can be used.

Windows

There is no need to install a Ruby version manager on Windows, like there is for Mac OS X or Linux. In fact, the chruby version manager does not work on Windows. Instead, you’ll use the RubyInstaller for Windows which can install different versions of Ruby on the same machine.

Install Ruby 1.9.x - Windows

Download and install the latest Windows RubyInstaller for Ruby 1.9.x from http://rubyinstaller.org/downloads (version 1.9.3-p484 as of this writing):

Verify that Ruby was installed correctly by running the following from a Command Prompt:

> ruby -v
ruby 1.9.3p484 (2013-11-22) [i386-mingw32]

Install Ruby DevKit - Windows

Download and install the Ruby Development Kit for use with Ruby 1.8.7 and 1.9.3.

Extract the archive to C:\Ruby\DevKit:

Enhance Rubies to use the DevKit - Windows

Run dk.rb init to generate a config.yml which includes all the installed Rubies to be enhanced to use the DevKit:

> cd /d c:\Ruby\DevKit
> ruby dk.rb init

If you want to review the list of Rubies before installing, run dk.rb review:

> cd /d c:\Ruby\DevKit
> ruby dk.rb review

Then run dk.rb to DevKit enhance your installed Rubies:

> cd /d c:\Ruby\DevKit
> ruby dk.rb install

Finally run devkitvars to add the DevKit tools to your command shell’s PATH and try to get the version of gcc to verify that the tools installed properly:

> c:\Ruby\DevKit\devkitvars.bat
Adding the DevKit to PATH...
> gcc --version
gcc (tdm-1) 4.5.2
Copyright (C) 2010 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS OR A PARTICULAR PURPOSE.

Install Bundler - Windows

You’ll need to use Bundler to manage gems. Installing a gem is also a good way to ensure that you’ve installed most of the Ruby prerequisites.

First, make sure you update to the latest version of Rubygems:

> c:\Ruby\DevKit\devkitvars.bat
Adding the DevKit to Path...
> gem update --system
Fetching: rubygems-update-2.2.0.gem (100%)
Successfully installed ruygems-update-2.2.0
Installing RubyGems 2.2.0
RubyGems 2.2.0 installed
...

Then install the bundler gem. If the gem install command reports Successfully installed you’re good to go:

> gem install bundler
Successfully installed bundler-1.5.0
Parsing documentation for bundler-1.5.0
1 gem installed

Install Git - Windows

While using source control is a recommended best practice, Chef does not require Git. If you use some other source control system besides Git, feel free to install it instead of Git. Use of Git source control just happens to be more common than others with Chef.

Download the latest stable release of the Git for Windows install from http://git-scm.com/downloads

Run the Git Windows install accepting the default choices, except on the PATH setup dialog. You’ll need to include the Unix tools on your Windows command prompt for some Chef commands to work:

Verify that Git installed correcting by running the following on a command prompt:

> git --version
git version 1.8.4.msysgit.0

Install Sublime Text 3 (Optional) - Windows

Miguel Cabeça has written an excellent plugin for the Sublime Text editor called SublimeChef, which is the closest thing to an Integrated Development Environment (IDE) that exists for Chef. Sublime Text costs $70 for a license, but has no restriction on the length of a trial period, so feel free try out Sublime Text to see if it works for you.

Download and install SublimeText 3 for your platform via http://www.sublimetext.com/3

Install Sublime Text 3 Package Control (Optional) - Windows

In order to install the SublimeChef plugin, first you need to install Sublime Text Package Control.

  • Download Package Control.sublime-package

  • Choose Preferences > Browse Packages... from the Sublime Text 3 menu

  • Browse up a folder, then navigate into Installed Packages/

  • Copy Package Control.sublime-package file you downloaded into this Sublime Text 3\Installed Packages directory

  • Restart Sublime Text 3 to enable Package Control

Once Package Control is installed successfully, you should be able to display the Command Pallete by pressing CTRL+SHIFT+P:

Install Sublime Chef (Optional) - Windows

After pressing CTRL+SHIFT+P to display the Command Pallette, start typing install to select Package Control: Install Package:

Then type chef to display the SublimeChef entry - click to install:

Miguel created the following demo video to show how SublimeChef can be used.

Getting Started Writing Chef Cookbooks the Berkshelf Way, Part 3

Update January 7, 2014 * Note about Michel Goetz’s blog series on ChefSpec

Updated December 29, 2013

  • Bumped Test Kitchen from 1.0.0.beta.3 to 1.1.1
  • Bumped CentOS to version 6.5
  • Per Kelly Setzer, updated os check with recent RSpec updates

Updated September 10, 2013

  • Bumped VirtualBox images from version 4.2.16 to 4.2.18
  • Bumped Vagrant from version 1.2.7 to 1.3.1

Updated September 1, 2013

  • Bumped Test Kitchen from 1.0.0.beta.2 to 1.0.0.beta.3

This is the third article in a series on writing Opscode Chef cookbooks the Berkshelf Way. Here’s a link to Part 1 and Part 2. The source code examples covered in this article can be found on Github: https://github.com/misheska/myface

In this installment, we’re going to learn how to use Test Kitchen to automate all the verification steps we did by hand for each iteration in Part 1 and Part 2. If not anything else, it’s worth learning Test Kitchen because OpsCode, the company that makes Chef, has encouraged the use of Test Kitchen to verify community cookbooks.

Test Kitchen is built on top of vagrant and supplements the Vagrantfile file you have been using so far in this series to do local automated testing. The main benefit to Test Kitchen is that it makes it easy to run tests on multiple platforms in parallel, which is more difficult to do with just a Vagrantfile. We’ll be showcasing this aspect of Test Kitchen by ensuring that Myface works on both the CentOS 6.4 and Ubuntu 12.04 Linux distributions.

Iteration #13 - Install Test Kitchen

Edit myface/Gemfile and add the following lines to load the Test Kitchen gems:

gem 'test-kitchen'
gem 'kitchen-vagrant'

Depending on when you went through this article series, your Gemfile may already have these additions. After editing, your myface/Gemfile should look like the following after editing:

myface/Gemfile
1
2
3
4
5
source 'https://rubygems.org'

gem 'berkshelf'
gem 'test-kitchen'
gem 'kitchen-vagrant'

After you have updated the Gemfile run bundle install to download the test-kitchen gem and all its dependencies:

$ bundle install
Fetching gem metadata from https://rubygems.org/........
Fetching gem metadata from https://rubygems.org/..
Resolving dependencies...
Using i18n (0.6.9)
Using multi_json (1.8.2)
Using activesupport (3.2.16)
...
Installing test-kitchen (1.1.1)
Using bundler (1.5.1)
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.

Testing Iteration #13 - Show the Test Kitchen version

If everything worked properly you should be able to run the kitchen --version command to see your installed Test Kitchen’s version information

$ kitchen --version
Test Kitchen version 1.1.1

Iteration #14 - Create a Kitchen YAML file

In order to use Test Kitchen on a cookbook, first you need to add a few more dependencies and create a template Kitchen YAML file. Test Kitchen makes this easy by providing the kitchen init command to perform all these initialization steps automatically

$ kitchen init
      create  .kitchen.yml
      append  Thorfile
      create  test/integration/default
      append  .gitignore
      append  .gitignore
      append  Gemfile
      append  Gemfile
You must run 'bundle install' to fetch any new gems.

Since kitchen init modified your Gemfile, you need to re-run bundle install (as suggested above) to pick up the new gem dependencies:

$ bundle install
Fetching gem metadata from https://rubygems.org/........
Fetching additional metadata from https://rubygems.org/..
Resolving dependencies...
Using i18n (0.6.9)
Using multi_json (1.8.2)
Using activesupport (3.2.16)
...
Using safe_yaml (0.9.7)
Using test-kitchen (1.1.1)
Installing kitchen-vagrant (0.14.0)
Using bundler (1.5.1)
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.

Most importantly, this new bundle install pass installed the kitchen-vagrant vagrant driver for Test Kitchen.

Now that you have created a .kitchen.yml Kitchen YAML file and loaded all the necessary gems, let’s customize the file so that we’ll use to verify that the Myface cookbook works on CentOS 6.4, like we were doing in the previous installments with the Vagrantfile.

myface/.kitchen.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
---
driver:
  name: vagrant

provisioner:
  name: chef_solo

platforms:
- name: centos65
  driver_config:
    box: centos65
    box_url: https://s3-us-west-2.amazonaws.com/misheska/vagrant/virtualbox4.3.6/centos65.box

suites:
- name: default
  run_list: ["recipe[myface]"]
  attributes:
    mysql:
      server_root_password: "rootpass"
      server_debian_password: "debpass"
      server_repl_password: "replpass"

Everything in the YAML file should be straightforward to understand, except perhaps the attributes item in the suites stanza. These values came from the Vagrantfile we used in the previous installments of this series. Here’s an excerpt from the Vagrantfile - at the end are some values that Berkshelf initialzed (which we used in Part 2).

...
  config.vm.provision :chef_solo do |chef|
    chef.json = {
      :mysql => {
        :server_root_password => 'rootpass',
        :server_debian_password => 'debpass',
        :server_repl_password => 'replpass'
      }
    }

   chef.run_list = [
        "recipe[myface::default]"
    ]
  end
end

Those Vagrantfile attributes were just converted into a format that the Test Kitchen YAML file format finds acceptable.

You can add even more Vagrantfile customizations to your kitchen.yml file if you like. For example, you can assign a host-only network IP so you can look at the MyFace website with a browser on your host. Add the following network: block to a platform’s driver_config::

...
driver_config:
  network:
  - ["private_network", {ip: "33.33.33.10"}]
...

After adding this block your .kitchen.yml should look like this:

myface/.kitchen.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
---
driver:
  name: vagrant

provisioner:
  name: chef_solo

platforms:
- name: centos65
  driver_config:
    box: centos65
    box_url: https://s3-us-west-2.amazonaws.com/misheska/vagrant/virtualbox4.3.6/centos65.box
    network:
    - ["private_network", {ip: "33.33.33.10"}]

suites:
- name: default
  run_list: ["recipe[myface]"]
  attributes:
    mysql:
      server_root_password: "rootpass"
      server_debian_password: "debpass"
      server_repl_password: "replpass"

For more information on the kitchen-vagrant settings, refer to the README.md file for kitchen-vagrant at https://github.com/opscode/kitchen-vagrant/blob/master/README.md

Testing Iteration #14 - Provision with Test Kitchen

You can do nearly everything that you were doing with vagrant just using Test Kitchen. The Test Kitchen equivalent of the vagrant up command is kitchen converge. Try running the kitchen converge command now to verify that your .kitchen.yml file is valid. When you run kitchen converge it will spin up a CentOS 6.5 vagrant test node instance and use Chef Solo to provision the MyFace cookbook on the test node:

$ kitchen converge 
-----> Starting Kitchen (v1.1.1)
-----> Creating <default-centos65>...
       Bringing machine 'default' up with 'virtualbox' provider...
       [default] Importing base box 'centos65'...
       [default] Matching MAC address for NAT networking...
       [default] Setting the name of the VM...
       [default] Clearing any previously set forwarded ports...
       [Berkshelf] Skipping Berkshelf with --no-provision
       [default] Fixed port collision for 22 => 2222. Now on port 2200.
       [default] Clearing any previously set network interfaces...
       [default] Preparing network interfaces based on configuration...
       [default] Forwarding ports...
       [default] -- 22 => 2200 (adapter 1)
       [default] Running 'pre-boot' VM customizations...
       [default] Booting VM...
       [default] Waiting for machine to boot. This may take a few minutes...
       [default] Machine booted and ready!
       [default] Setting hostname...
       Vagrant instance <default-centos65> created.
       Finished creating <default-centos65> (0m51.76s).
-----> Converging <default-centos65>...
       Preparing files for transfer
       Resolving cookbook dependencies with Berkshelf...
       Removing non-cookbook files before transfer
-----> Installing Chef Omnibus (true)
       downloading https://www.opscode.com/chef/install.sh
         to file /tmp/install.sh
       trying wget...
       Downloading Chef  for el...
       downloading https://www.opscode.com/chef/metadata?v=&prerelease=false&p=el&pv=6&m=x86_64
         to file /tmp/install.sh.2158/metadata.txt
       trying wget...
       url	https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-11.8.2-1.el6.x86_64.rpm
       md5	10f3d0da82efa973fe91cc24a6a74549
       sha256	044558f38d25bbf75dbd5790ccce892a38e5e9f2a091ed55367ab914fbd1cfed
       downloaded metadata file looks valid...
       downloading https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-11.8.2-1.el6.x86_64.rpm
         to file /tmp/install.sh.2158/chef-.x86_64.rpm
       trying wget...
       Checksum compare with sha256sum succeeded.
       Installing Chef
       installing with rpm...
       warning: /tmp/install.sh.2158/chef-.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 83ef826a: NOKEY
Preparing...                #####  ########################################### [100%]
   1:chef                          ########################################### [100%]
       Thank you for installing Chef!
       Transfering files to <default-centos65>
       [2013-12-29T11:44:52-08:00] INFO: Forking chef instance to converge...
       Starting Chef Client, version 11.8.2
       [2013-12-29T11:44:52-08:00] INFO: *** Chef 11.8.2 ***
       [2013-12-29T11:44:52-08:00] INFO: Chef-client pid: 2257
       [2013-12-29T11:44:53-08:00] INFO: Setting the run_list to ["recipe[myface::default]"] from JSON
       [2013-12-29T11:44:53-08:00] INFO: Run List is [recipe[myface::default]]
       ...
       Recipe: apache2::default
         * service[apache2] action restart[2013-12-29T11:47:41-08:00] INFO: Processing service[apache2] action restart (apache2::default line 210)
       [2013-12-29T11:47:43-08:00] INFO: service[apache2] restarted

           - restart service service[apache2]

       [2013-12-29T11:47:43-08:00] INFO: Chef Run complete in 170.575449983 seconds
       [2013-12-29T11:47:43-08:00] INFO: Running report handlers
       [2013-12-29T11:47:43-08:00] INFO: Report handlers complete
       Chef Client finished, 100 resources updated
       Finished converging <default-centos65> (3m43.06s).
-----> Kitchen is finished. (4m35.40s)

To display the results of the Chef Run, type in the kitchen list command:

$ kitchen list
Instance           Driver   Provisioner  Last Action
default-centos65   Vagrant  Chef Solo    Converged

If the run succeeded, it should display Converged in the Last Action field.

The Test Kitchen equivalent of the vagrant ssh command is kitchen login. Since Test Kitchen supports multiple instances, you will need to pass in the instance name for which you wish to login as a parameter (which you can get from the kitchen list output). We want to login to the CentOS 6.5 instance (the only instance for now), so type in the command kitchen login default-centos65 to login:

$ kitchen login default-centos65
Last login: Sun Dec 29 13:16:33 2013 from 10.0.2.2
Welcome to your Packer-built virtual machine.
[vagrant@default-centos65 ~]$

Now you can poke around in the image the same way you did with vagrant ssh, for example, verifying that the httpd server is running:

[vagrant@default-centos65 ~]$ sudo /sbin/service httpd status
httpd (pid  4410) is running...

After you are done working in the test instance, make sure to run the exit command to log out so that you return back to your host prompt:

[vagrant@default-centos65 ~]$ exit
logout
Connection to 127.0.0.1 closed.

Should you need it, the Test Kitchen equivalent of vagrant destroy is kitchen destroy. If you make a change to the chef cookbook and want to re-deploy, the Test Kitchen equivalent of vagrant provision is kitchen converge.

Since you added a private IP for you instance, you can also view the MyFace website on your host with your favorite web browser:

http://33.33.33.10

Welcome to MyFace

Iteration #15 - Provisioning Ubuntu

We haven’t really made use of any unique Test Kitchen features yet, let’s start now. We’ll also deploy our cookbook locally to Ubuntu 12.04 for testing, in addition to CentOS 6.5.

Edit .kitchen.yml and add a reference to a Ubuntu 12.04 basebox alongside the existing CentOS 6.5 basebox in the platforms stanza:

- name: ubuntu1204
  driver_config:
    box: ubuntu1204
    box_url: https://s3-us-west-2.amazonaws.com/misheska/vagrant/virtualbox4.3.6/ubuntu1204.box
    network:
    - ["private_network", {ip: "33.33.33.11"}]

After editing, your .kitchen.yml file should resemble the following:

myface/.kitchen.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
---
driver:
  name: vagrant

provisioner:
  name: chef_solo

platforms:
- name: centos65
  driver_config:
    box: centos65
    box_url: https://s3-us-west-2.amazonaws.com/misheska/vagrant/virtualbox4.3.6/centos65.box
    network:
    - ["private_network", {ip: "33.33.33.10"}]
- name: ubuntu1204
  driver_config:
    box: ubuntu1204
    box_url: https://s3-us-west-2.amazonaws.com/misheska/vagrant/virtualbox4.3.6/ubuntu1204.box
    network:
    - ["private_network", {ip: "33.33.33.11"}]

suites:
- name: default
  run_list: ["recipe[myface]"]
  attributes:
    mysql:
      server_root_password: "rootpass"
      server_debian_password: "debpass"
      server_repl_password: "replpass"

Before we run kitchen converge to do a Chef run, we need to fix our cookbook so it will run successfully on Ubuntu 12.04. If you tried to deploy now you would notice that the MyFace cookbook would fail to deploy to Ubuntu 12.04 successfully due to a reference to the php-mysql package in myface/receipes/webserver.rb.

... 
include_recipe 'apache2'
include_recipe 'apache2::mod_php5'

package 'php-mysql' do
  action :install
  notifies :restart, 'service[apache2]'
end
...

On Ubuntu the package name should be php5-mysql instead of php-mysql.

As with most issues in the Chef world, there’s a cookbook for that! The Opscode php cookbook has conditionals to reference the correct name for the php-mysql package on a number of platforms.

Edit myface/metadata.rb and add a reference to the latest version of the php cookbook (currently 1.3.10):

myface/metadata.rb
1
2
3
4
5
6
7
8
9
10
11
12
name             'myface'
maintainer       'Mischa Taylor'
maintainer_email 'mischa@misheska.com'
license          'Apache 2.0'
description      'Installs/Configures myface'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version          '2.0.0'

depends 'apache2', '~> 1.8.0'
depends 'mysql', '~> 4.0.0'
depends 'database', '~> 1.6.0'
depends 'php', '~> 1.3.0'

In myface/recipes/webserver.rb replace the package "php-mysql" do ... end block with the following reference:

include_recipe 'php::module_mysql'

After editing, myface/recipes/webserver.rb should look like this:

myface/recipes/webserver.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
#
# Cookbook Name:: myface
# Recipe:: webserver
#
# Copyright (C) 2013 YOUR_NAME
#
# All rights reserved - Do Not Redistribute
#

group node['myface']['group']

user node['myface']['user'] do
  group node['myface']['group']
  system true
  shell '/bin/bash'
end

include_recipe 'apache2'
include_recipe 'apache2::mod_php5'

include_recipe 'php::module_mysql'

# disable default site
apache_site '000-default' do
  enable false
end

# create apache config
template "#{node['apache']['dir']}/sites-available/#{node['myface']['config']}" do
  source 'apache2.conf.erb'
  notifies :restart, 'service[apache2]'
end

# create document root
directory "#{node['myface']['document_root']}" do
  action :create
  mode '0755'
  recursive true
end

# write site
template "#{node['myface']['document_root']}/index.php" do
  source 'index.php.erb'
  mode '0644'
end

# enable myface
apache_site "#{node['myface']['config']}" do
  enable true
end

Testing Iteration #15 - Deploy locally to Ubuntu 12.04

Now that we’ve fixed up our cookbook to work on Ubuntu 12.04, let’s test it out! Run kitchen list to display the list of Test Kitchen instances:

Instance             Driver   Provisioner  Last Action
default-centos65     Vagrant  Chef Solo    Converged
default-ubuntu1204   Vagrant  Chef Solo    <Not Created>

Notice that after editing the .kitchen.yml we now have an Ubuntu 12.04 instance called default-ubuntu1204 and it is in the <Not Created> state.

Go ahead setup the Ubuntu 12.04 instance by running kitchen converge again:

$ kitchen converge default-ubuntu1204

Note that this time we added an optional instance parameter so that Test Kitchen only performs the action against the specified instance. If you do not specify this parameter, it defaults to all, running the command against all instances. After about 5-10 minutes or so, you should observe that Test Kitchen downloaded an Ubuntu 12.04 basebox, booted a VM with the basebox, and successfully deployed our chef cookbook.

Run the kitchen list command again to verify that the Ubuntu 12.04 instance is now in the Set Up state as well, showing that there were no errors:

Instance             Driver   Provisioner  Last Action
default-centos65     Vagrant  Chef Solo    Converged
default-ubuntu1204   Vagrant  Chef Solo    Converged

You just fixed an error with the MyFace cookbook that prevented deployment to Ubuntu 12.04, and verified that the cookbook correctly deploys to both Ubuntu 12.04 and Centos 6.5.

Use the kitchen login command to ssh into each instance and poke around if you like. You now have two local vagrant VMs instantiated to play with!

$ kitchen login default-ubuntu1204
Welcome to Ubuntu 12.04.3 LTS (GNU/Linux 3.8.0-29-generic x86_64)

 * Documentation:  https://help.ubuntu.com/
Last login: Sun Dec 29 13:30:36 2013 from 10.0.2.2
$ [...poke around, run some commands...]
$ exit
Connection to 127.0.0.1 closed.

$ kitchen login default-centos65
Last login: Sun Dec 29 13:21:10 2013 from 10.0.2.2
Welcome to your Packer-built virtual machine.
[vagrant@default-centos65 ~]$ [...poke around, run some commands...]
[vagrant@default-centos65 ~]$ exit
logout
Connection to 127.0.0.1 closed.

You can view the websites for each instance by viewing the appropriate private IP

CentOS 6.5:   http://33.33.33.10
Ubuntu 12.04: http://33.33.33.11

Iteration #16 - Writing your first Serverspec test

While it’s really helpful to know now that the Myface cookbook will converge on both a CentOS 6.5 and Ubuntu 12.04 setup, we haven’t written any tests yet. Let’s do that.

It’s helpful to know that Test Kitchen was designed as a framework for post-convergence system testing. You are supposed to set up a bunch of test instances, perform a Chef run to apply your cookbook’s changes to them, then when this is process is complete your tests can inspect the state of each test instance after the Chef run is finished. This is how we tested our nodes by hand in Part 1 and Part 2. Now we are going to automate the process.

NOTE: It is a testing anti-pattern to rely too much on system tests, even if they are automated so make sure you are judicious in your use of system tests. It can be difficult to maintain a lot of system tests over time and keep them relevant. The tests you performed by hand in Part 1 and Part 2 to verify MyFace are at just the right level of detail for a system test. Do just enough to verify that the system works correctly after it was configured. In a future post, I’ll cover unit tests in more detail using Chefspec and Guard. For now, let’s focus on system tests.

Test Kitchen finds tests by following a directory naming convention. When you installed Test Kitchen, it created the test/integration/default subdirectory underneath your main cookbook. It looks for test code in the following directory underneath test/integration:

<COOKBOOOK-PATH>/test/integration/<TEST-SUITE-NAME>/<PLUGIN-NAME>

A collection of tests is called a test suite. Following Test Kitchen install’s lead, we’ll just call our first suite of tests default. Test Kitchen has a number of plugins which will install and setup any components necessary for running tests. We’ll be using the severspec plugin for our tests. So you will be placing your test files in the following directory:

myface/test/integration/default/serverspec

Create the myface/test/integration/default/serverspec subdirectory now.

To start, we need to add a Ruby helper script which loads our Test Kitchen plugin. We’ll call it myface/test/integration/default/serverspec/spec_helper.rb:

myface/test/integration/default/serverspec/spec_helper.rb
1
2
3
4
5
6
7
8
9
10
11
require 'serverspec'
require 'pathname'

include Serverspec::Helper::Exec
include Serverspec::Helper::DetectOS

RSpec.configure do |c|
  c.before :all do
    c.os = backend(Serverspec::Commands::Base).check_os
  end
end

This code is modeled after the examples provided on the serverspec website.

Create a subdirectory underneath myface/test/integration/default/serverspec called localhost:

myface/test/integration/default/serverspec/localhost

It is a serverspec convention to put tests (a.k.a. “specs”) underneath spec_helper.rb in a subdirectory denoting the host name to be tested. Serverspec supports testing via SSH access to remote hosts. We won’t be using this capability as we will be testing local images, so we’ll just use localhost for the host name.

Now, let’s write our first test! If you recall in Testing Iteration #1, we ran the following command to verify that the myface user was created:

$ getent password myface
myface:x:497:503::/home/myface:/bin/bash

Create the a file named myface/test/integration/default/serverspec/localhost/webserver_spec.rb that contains a serverspec test to perform the same action:

myface/test/integration/default/serverspec/localhost/webserver_spec.rb
1
2
3
4
5
6
7
8
9
require 'spec_helper'

describe 'MyFace webserver' do

  it 'should have a myface user' do
    expect(command 'getent passwd myface').to return_stdout /myface:x:\d+:\d+::\/home\/myface:\/bin\/bash/
  end

end

Serverspec provides extensions to Rspec to help you more easily test servers. If you’re not familiar with Rspec syntax, Code School has an excellent tutorial on Testing with Rspec. Even if you don’t know Rspec, you should still be able to follow along with provided examples.

You can find a list of serverspec resources at the following link: http://serverspec.org/resource_types.html. We’re using the command resource to run the command getent password myface and match the resultant output with a Ruby regular expression (because the uid and gid field could be any number between 100-999, because myface is a system account).

Testing Iteration #16 - Running your first Serverspec test

OK, let’s run our first test!

To start you need to run kitchen setup so that Test Kitchen loads and configures all the required plugins. In keeping with the restaurant theme, the component that manages Test Kitchen plugins is called Busser.

$ kitchen setup
-----> Starting Kitchen (v1.1.1)
-----> Setting up <default-centos65>...
Fetching: thor-0.18.1.gem (100%)
Fetching: busser-0.6.0.gem (100%)
       Successfully installed thor-0.18.1
       Successfully installed busser-0.6.0
       2 gems installed
-----> Setting up Busser
       Creating BUSSER_ROOT in /tmp/busser
       Creating busser binstub
       Plugin serverspec installed (version 0.2.5)
-----> Running postinstall for serverspec plugin
       Finished setting up <default-centos65> (0m52.12s).
-----> Setting up <default-ubuntu1204>...
Fetching: thor-0.18.1.gem (100%)
Fetching: busser-0.6.0.gem (100%)
Successfully installed thor-0.18.1
Successfully installed busser-0.6.0
2 gems installed
-----> Setting up Busser
       Creating BUSSER_ROOT in /tmp/busser
       Creating busser binstub
       Plugin serverspec installed (version 0.2.5)
-----> Running postinstall for serverspec plugin
       Finished setting up <default-ubuntu1204> (0m15.36s).
-----> Kitchen is finished. (1m7.77s)

After running kitchen setup, next run kitchen verify to run your test suite.

$ kitchen verify
-----> Starting Kitchen (v1.1.1)
-----> Verifying <default-centos65>...
       Suite path directory /tmp/busser/suites does not exist, skipping.
       Uploading /tmp/busser/suites/serverspec/localhost/webserver_spec.rb (mode=0644)
       Uploading /tmp/busser/suites/serverspec/spec_helper.rb (mode=0644)
-----> Running serverspec test suite
       /opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -S /opt/chef/embedded/bin/rspec /tmp/busser/suites/serverspec/localhost/webserver_spec.rb --color --format documentation

       MyFace webserver
         should have a myface user

       Finished in 0.04293 seconds
       1 example, 0 failures
       Finished verifying <default-centos65> (0m1.61s).
-----> Verifying <default-ubuntu1204>...
       Suite path directory /tmp/busser/suites does not exist, skipping.
Uploading /tmp/busser/suites/serverspec/localhost/webserver_spec.rb (mode=0644)
Uploading /tmp/busser/suites/serverspec/spec_helper.rb (mode=0644)
-----> Running serverspec test suite
 /opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -S /opt/chef/embedded/bin/rspec /tmp/busser/suites/serverspec/localhost/webserver_spec.rb --color --format documentation

MyFace webserver
  should have a myface user

Finished in 0.11307 seconds
1 example, 0 failures
       Finished verifying <default-ubuntu1204> (0m1.66s).
-----> Kitchen is finished. (0m3.80s)

Finally run kitchen list to display the results of your test run.

$ kitchen list
Instance             Driver   Provisioner  Last Action
default-centos-64    Vagrant  Chef Solo    Verified
default-ubuntu-1204  Vagrant  Chef Solo    Verified

If Test Kitchen displays the Last Action as Verified, all the tests passed.

Iteration #17 - Completing the webserver test suite

Now let’s dive in and encode all the rest of the tests from Part 1.

While we used the command resource to encode our first test, this isn’t the optimal way to encode this test as a serverspec. We can make use of the user resource to encode a test more succinctly:

it 'should have a myface user' do
  expect(user 'myface').to exist
end

The myface/test/integration/default/serverspec/localhost/webserver_spec.rb file should resemble the following:

myface/test/integration/default/serverspec/localhost/webserver_spec.rb
1
2
3
4
5
6
7
8
9
require 'spec_helper'

describe 'MyFace webserver' do

  it 'should have a myface user' do
    expect(user 'myface').to exist
  end

end

Run kitchen verify and kitchen list to re-run your test. You should see the same result as before - 1 example, 0 failures:

$ kitchen verify
$ kitchen list

Only use the command resource as a method of last resort. First check to see if serverspec has a better resource to perform a test.

Let’s move on to the next command from Testing Iteration #3:

$ vagrant ssh -c "sudo /sbin/service httpd status"
httpd (pid  4831) is running.

Use the servspec service resource to perform a test to ensure that the httpd service is running and it automatically starts on bootup:

it 'should be running the httpd server' do
  expect(service 'httpd').to be_running
  expect(service 'httpd').to be_enabled
end

Add this statement to your webserver_spec file:

myface/test/integration/default/serverspec/localhost/webserver_spec.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
require 'spec_helper'

describe 'MyFace webserver' do

  it 'should have a myface user' do
    expect(user 'myface').to exist
  end

  it 'should be running the httpd server' do
    expect(service 'httpd').to be_running
    expect(service 'httpd').to be_enabled
  end

end

Run kitchen verify and kitchen list again to run this new test:

$ kitchen verify
-----> Starting Kitchen (v1.1.1)
-----> Verifying <default-centos65>...
...
       MyFace webserver
         should have a myface user
         should be running the httpd server

       Finished in 0.04706 seconds
       2 examples, 0 failures
       Finished verifying <default-centos65> (0m1.61s).
-----> Verifying <default-ubuntu1204>...
       Removing /tmp/busser/suites/serverspec
Uploading /tmp/busser/suites/serverspec/localhost/webserver_spec.rb (mode=0644)
Uploading /tmp/busser/suites/serverspec/spec_helper.rb (mode=0644)
-----> Running serverspec test suite
/opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -S /opt/chef/embedded/bin/rspec /tmp/busser/suites/serverspec/localhost/webserver_spec.rb --color --format documentation

MyFace webserver
  should have a myface user
httpd: unrecognized service
  should be running the httpd server (FAILED - 1)

Failures:

  1) MyFace webserver should be running the httpd server
     Failure/Error: expect(service 'httpd').to be_running
       ps aux | grep -w -- httpd | grep -qv grep
       expected Service "httpd" to be running
     # /tmp/busser/suites/serverspec/localhost/webserver_spec.rb:10:in `block (2 levels) in <top (required)>'

Finished in 0.05906 seconds
2 examples, 1 failure

Failed examples:

rspec /tmp/busser/suites/serverspec/localhost/webserver_spec.rb:9 # MyFace webserver should be running the httpd server
/opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -S /opt/chef/embedded/bin/rspec /tmp/busser/suites/serverspec/localhost/webserver_spec.rb --color --format documentation failed
Ruby Script[/tmp/busser/gems/gems/busser-serverspec-0.2.5/lib/busser/runner_plugin/../serverspec/runner.rb /tmp/busser/suites/serverspec] exit code was 1
>>>>>> Verify failed on instance <default-ubuntu1204>.
>>>>>> Please see .kitchen/logs/default-ubuntu1204.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: SSH exited (1) for command: [sh -c 'BUSSER_ROOT="/tmp/busser" GEM_HOME="/tmp/busser/gems" GEM_PATH="/tmp/busser/gems" GEM_CACHE="/tmp/busser/gems/cache" ; export BUSSER_ROOT GEM_HOME GEM_PATH GEM_CACHE; sudo -E /tmp/busser/bin/busser test']
>>>>>> ----------------------
...
$ kitchen list
Instance            Driver   Provisioner  Last Action
default-centos65    Vagrant  ChefSolo     Verified
default-ubuntu1204  Vagrant  ChefSolo     Verified

Uh oh! That’s not what we expected! The tests failed on our Ubuntu 12.04 instance - and yet it still says that it is Verified, but the tests passed on CentOS 6.5. The Last Action field is literally the last action. It does not report success or failure state, so you’ll want to pay attention to the output of kitchen verify and note whether or not all the tests passed.

In this case, the reason for the failure is that on Ubuntu, the name of the Apache httpd service is apache2 not httpd. Let’s address this by adding a conditional that checks the os custom configuration setting that is set in spec_helper.rb.

I didn’t explain what this did before, but it runs a serverspec helper method to check the os type before each spec/test run. When running under Ubuntu (or Debian), the value of RSpec.configuation.os will be Debian, otherwise the value will be RedHat if it is running under any RHEL variant, including CentOS. So the following conditional should do the trick:

it 'should be running the httpd server' do
  case RSpec.configuration.os
  when "Debian"
    expect(service 'apache2').to be_running
    expect(service 'apache2').to be_enabled
  else
    expect(service 'httpd').to be_running
    expect(service 'httpd').to be_enabled
  end
end 

After this change, your webserver_spec.rb file should resemble the following:

myface/test/integration/default/serverspec/localhost/webserver_spec.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
require 'spec_helper'

describe 'MyFace webserver' do

  it 'should have a myface user' do
    expect(user 'myface').to exist
  end

  it 'should be running the httpd server' do
    case RSpec.configuration.os[:family]
    when "Ubuntu"
      expect(service 'apache2').to be_running
      expect(service 'apache2').to be_enabled
    else
      expect(service 'httpd').to be_running
      expect(service 'httpd').to be_enabled
    end
  end

end

Run kitchen verify and kitchen list again - all the tests should pass:

$ kitchen verify
-----> Starting Kitchen (v1.1.1)
-----> Verifying <default-centos65>...
       Removing /tmp/busser/suites/serverspec
       Uploading /tmp/busser/suites/serverspec/localhost/webserver_spec.rb (mode=0644)
       Uploading /tmp/busser/suites/serverspec/spec_helper.rb (mode=0644)
-----> Running serverspec test suite
       /opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -S /opt/chef/embedded/bin/rspec /tmp/busser/suites/serverspec/localhost/webserver_spec.rb --color --format documentation

       MyFace webserver
         should have a myface user
         should be running the httpd server

       Finished in 0.04858 seconds
       2 examples, 0 failures
       Finished verifying <default-centos65> (0m1.58s).
-----> Verifying <default-ubuntu1204>...
       Removing /tmp/busser/suites/serverspec
Uploading /tmp/busser/suites/serverspec/localhost/webserver_spec.rb (mode=0644)
Uploading /tmp/busser/suites/serverspec/spec_helper.rb (mode=0644)
-----> Running serverspec test suite
/opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -S /opt/chef/embedded/bin/rspec /tmp/busser/suites/serverspec/localhost/webserver_spec.rb --color --format documentation

MyFace webserver
  should have a myface user
  should be running the httpd server

Finished in 0.06551 seconds
2 examples, 0 failures
       Finished verifying <default-ubuntu1204> (0m1.59s).
-----> Kitchen is finished. (0m3.46s)

$ kitchen list
Instance             Driver   Provisioner  Last Action
default-centos65     Vagrant  Chef Solo    Verified
default-ubuntu1204   Vagrant  Chef Solo    Verified

The final test that is used for the rest of the Test Iterations in Part 1 basically amounts to visiting http://33.33.33.10 with a web browser and eyeballing the results. That would be difficult to automate with serverspec, and one would probably want to use a web automation framework like Selenium to do this. However, you can at least use serverspec to verify that the Apache Server is serving up content on port 80 (the default http port).

We can check that the server is listening on port 80 with the port resource:

it 'should be listening on port 80' do
  expect(port 80).to be_listening
end

We’ll resort to using the command resource to check to see if the server accepts an HTTP connections and returns something that looks reasonable, as there doesn’t seem to be an obvious higher-level resource to perform this action:

it 'should respond to an HTTP request' do
  expect(command 'curl localhost').to return_stdout /.*<title>MyFace Users<\/title>.*/
end

After adding these two checks, this is what your webserver_spec.rb file should look like:

myface/test/integration/default/serverspec/localhost/webserver_spec.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
require 'spec_helper'

describe 'MyFace webserver' do

  it 'should have a myface user' do
    expect(user 'myface').to exist
  end

  it 'should be running the httpd server' do
    case RSpec.configuration.os[:family]
    when "Ubuntu"
      expect(service 'apache2').to be_running
      expect(service 'apache2').to be_enabled
    else
      expect(service 'httpd').to be_running
      expect(service 'httpd').to be_enabled
    end
  end

  it 'should respond to an HTTP request' do
    expect(command 'curl localhost').to return_stdout /.*<title>MyFace Users<\/title>.*/
  end

end

Now we have an automated script that performs some basic tests to verify that our cookbook enabled the web server properly. Let the robots do some of the grunge work!

Testing Iteration #17 - Running the suite

Do a final kitchen verify and kitchen list. Everything should look good:

$ kitchen verify
-----> Starting Kitchen (v1.1.1)
-----> Verifying <default-centos65>...
       Removing /tmp/busser/suites/serverspec
       Uploading /tmp/busser/suites/serverspec/localhost/webserver_spec.rb (mode=0644)
       Uploading /tmp/busser/suites/serverspec/spec_helper.rb (mode=0644)
-----> Running serverspec test suite
       /opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -S /opt/chef/embedded/bin/rspec /tmp/busser/suites/serverspec/localhost/webserver_spec.rb --color --format documentation

       MyFace webserver
         should have a myface user
         should be running the httpd server
         should respond to an HTTP request

       Finished in 0.05525 seconds
       3 examples, 0 failures
       Finished verifying <default-centos65> (0m1.53s).
-----> Verifying <default-ubuntu1204>...
       Removing /tmp/busser/suites/serverspec
Uploading /tmp/busser/suites/serverspec/localhost/webserver_spec.rb (mode=0644)
Uploading /tmp/busser/suites/serverspec/spec_helper.rb (mode=0644)
-----> Running serverspec test suite
/opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -S /opt/chef/embedded/bin/rspec /tmp/busser/suites/serverspec/localhost/webserver_spec.rb --color --format documentation

MyFace webserver
  should have a myface user
  should be running the httpd server
  should respond to an HTTP request

Finished in 0.15965 seconds
3 examples, 0 failures
       Finished verifying <default-ubuntu1204> (0m1.68s).
-----> Kitchen is finished. (0m3.53s)

$ kitchen list
Instance             Driver   Provisioner  Last Action
default-centos65     Vagrant  Chef Solo    Verified
default-ubuntu1204   Vagrant  Chef Solo    Verified

Iteration #18 - Completing the database test suite

Let’s wrap this up by finishing off the tests for the database portion in Part 2 Create a new file called myface/test/integration/default/serverspec/localhost/database_spec.rb to contain the database tests.

In Testing Iteration #7 we checked to see if the mysqld service was running with the following command:

$ sudo /sbin/service mysqld status

There is a similar name difference between the Ubuntu and CentOS services as there was with the Apache web server. For Ubuntu, the name of the MySQL service is mysql. For CentOS, the name of the service is mysqld.

This should be a piece of cake to write a serverspec test for now:

it 'should be running the database server' do
  case RSpec.configuration.os[:family]
  when "Ubuntu"
    expect(service 'mysql').to be_running
    expect(service 'mysql').to be_enabled
  else
    expect(service 'mysqld').to be_running
    expect(service 'mysqld').to be_enabled
  end
end

In Testing Iteration #8 we ran the following command to verify that the myface database was created:

$ mysqlshow -uroot -prootpass

That’s a simple command resource regular expression:

it 'should have created the myface database' do
  expect(command 'mysqlshow -uroot -prootpass').to return_stdout /.*myface.*/
end

In Testing Iteration #9 we created a myface-app MySQL database user and to check to see if the myface_app user only has rights to the myface database with the following commands:

$ mysql -uroot -prootpass -e "select user,host from mysql.user;"
$ mysql -uroot -prootpass -e "show grants for 'myface_app'@'localhost';"

Again, these are just more serverspec commands (\s indicates “any whitespace character”):

it 'should have created the myface_app user' do
  expect(command 'mysql -uroot -prootpass -e "select user,host from mysql.user;"').to return_stdout /.*myface_app\s+localhost.*/
end

it 'should have given the myface_app database user rights to myface' do
  expect(command 'mysql -uroot -prootpass -e "show grants for \'myface_app\'@\'localhost\';"').to return_stdout /.*GRANT ALL PRIVILEGES ON `myface`.\* TO \'myface_app\'@\'localhost\'.*/
end

In Testing Itegration #10 we dumped the contents of the users table to verify it got created:

$ mysql -hlocalhost -umyface_app -psupersecret -Dmyface -e "select id,user_name from users;"'

You guessed it, yet another command:

it 'should have created the users table' do
  expect(command 'mysql -hlocalhost -umyface_app -psupersecret -Dmyface -e "select id,user_name from users;"').to return_stdout /.*mbower.*/
end

In Testing Iteration #11 we checked to see if the php5_module was successfully installed:

$ sudo /usr/sbin/httpd -M | grep php5

Note that there is a php_config serverspec resource for checking PHP config settings, but that’s not helpful for checking the existence of PHP, so another command will do (just remember the service name is different between the two different OSes):

it 'should have installed the Apache php5_module' do
  case RSpec.configuration.os[:family]
  when "Ubuntu"
    expect(command 'sudo /usr/sbin/apache2 -M | grep php5').to return_stdout /.*php5_module.*/
  else
    expect(command 'sudo /usr/sbin/httpd -M | grep php5').to return_stdout /.*php5_module.*/
  end
end

And there you have it! Your final database_spec.rb file should resemble the following:

myface/test/integration/default/serverspec/localhost/database_spec.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
require 'spec_helper'

describe 'MyFace database' do

  it 'should be running the database server' do
    case RSpec.configuration.os[:family]
    when "Ubuntu"
      expect(service 'mysql').to be_running
      expect(service 'mysql').to be_enabled
    else
      expect(service 'mysqld').to be_running
      expect(service 'mysqld').to be_enabled
    end
  end

  it 'should have created the myface database' do
    expect(command 'mysqlshow -uroot -prootpass').to return_stdout /.*myface.*/
  end

  it 'should have created the myface_app user' do
    expect(command 'mysql -uroot -prootpass -e "select user,host from mysql.user;"').to return_stdout /.*myface_app\s+localhost.*/
  end

  it 'should have given the myface_app database user rights to myface' do
    expect(command 'mysql -uroot -prootpass -e "show grants for \'myface_app\'@\'localhost\';"').to return_stdout /.*GRANT ALL PRIVILEGES ON `myface`.\* TO \'myface_app\'@\'localhost\'.*/
  end

  it 'should have created the users table' do
    expect(command 'mysql -hlocalhost -umyface_app -psupersecret -Dmyface -e "select id,user_name from users;"').to return_stdout /.*mbower.*/
  end

  it 'should have installed the Apache php5_module' do
    case RSpec.configuration.os[:family]
    when "Ubuntu"
      expect(command 'sudo /usr/sbin/apache2 -M | grep php5').to return_stdout /.*php5_module.*/
    else
      expect(command 'sudo /usr/sbin/httpd -M | grep php5').to return_stdout /.*php5_module.*/
    end
  end

end

Testing Iteration #19 - kitchen test

Perform a final kitchen verify and kitchen list to check that there are no syntax errors. 9 tests succeeded!

In addition to the kitchen commands that you have used so far, there’s one other command that it quite useful - kitchen test. It runs all the commands in the Test Kitchen test lifecycle in order:

kitchen create - Creates a vagrant instance.

kitchen converge - Provision the vagrant instance with Chef, using the run list specified in the .kitchen.yml file.

kitchen setup - Install and configure any necessary Test Kitchen plugins needed to run tests.

kitchen verify - Run tests.

kitchen destroy - Destroy the vagrant instance, removing it from memory & disk.

When you are in the midst of writing tests, using the above commands interactively can save time (like only running kitchen verify after adding a new test). But once the tests are written, normally you will run kitchen test to run everything in one shot, preferably running as a “latch” triggered when your cookbook changes are committed to source control. This will ensure that your tests are run often.

Conclusion

So hopefully now you understand how to use Test Kitchen and what it’s useful for. In the next article in this series, we’ll cover writing tests that can run before deployment, providing feedback more quickly than with Test Kitchen (albeit in more limited circumstances), using Chefspec and Guard. (In the meantime check out Michael Geotz’s excellent article series on ChefSpec with Guard as I’m currently coauthoring upcoming O’Reilly Media book on Chef with Seth Vargo and probably won’t have time to do further installments in this series anytime soon until I help finish the d*mn book!)

Now you should be able to test everything but the kitchen sink! Wait… actually Test Kitchen has got that covered as well:

$ kitchen sink

                     ___
                    ' _ '.
                  / /` `\ \
                 | |   [__]
                 | |    \{\{
                 | |    \}\}
              _  | |  _ \{\{
  ___________<_>_| |_<_>\}\}________
      .=======^=(___)=^=\{\{====.
     / .----------------\}\}---. \
    / /                 \{\{    \ \
   / /                  \}\}     \ \
  (  '========================='  )
   '-----------------------------'

Windows Server 2012 Automated Install Settings

Updated March 19, 2014

  • Updated ADK link from version 8 to 8.1
  • Updated link to Autounattend.xml
  • Added note about using KMS keys

I just recently revised all my automated install XML files for the Windows System Preparation Tool (Sysprep) that I use for my Windows development testbed. For this go around, I’m documenting the XML answer file settings for each version of Microsoft Windows. This article covers the XML answer files settings needed to automate a Windows Server 2012 (64-bit) base OS install.

You’ll need to use the Windows System Image Manager tool to edit Sysprep XML answer files. The Windows System Image Manager is packaged with the Windows Assessment and Deployment Kit tool suite. Download and install the Windows Assessment and Deployment Kit to install the Windows System Image Manager (WSIM).

Link to Autounattend.xml with all the settings in this article. NOTE: Right-click and choose “Download Linked File As…” in your web browser, as many web browsers will try to interpret the Xml.

Disabling the language settings dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-International-Core-WinPE_6.2.9200.16384_neutral, right-click and choose Add Setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • InputLocale = en-US
  • SystemLocale = en-US
  • UILanguage = en-US
  • UserLocale = en-US

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.2.9200.16384_neutral, right-click on UserData/ProductKey and choose Add Setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • Key = YOUR_PRODUCT_KEY
  • WillShowUI = OnError

The official Microsoft KMS keys are listed here and make a good starting point to test installs.

Disabling the Select Operating System dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.2.9200.16384_neutral, right-click on ImageInstall/OSImage/InstallFrom/Metadata and choose Add Setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • Key = /IMAGE/NAME
  • Value = Windows Server 2012 SERVERDATACENTER

NOTE: Make sure the /IMAGE/NAME value matches the Windows Server 2012 Image flavor you selected. Possible values are:

  • Windows Server 2012 SERVERDATACENTER
  • Windows Server 2012 SERVERDATACENTERCORE
  • Windows Server 2012 SERVERSTANDARD
  • Windows Server 2012 SERVERSTANDARDCORE

Disabling the EULA dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.2.9200.16384_neutral, right-click on UserData and choose Add Setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • AcceptEula = true

Disabling the Disk Allocation dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.2.9200.16384_neutral, right-click on DiskConfiguration and choose Add setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • WillShowUI = OnError

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.2.9200.16384_neutral, right-click on DiskConfiguration/Disk and choose Add setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • DiskID = 0
  • WillWipDisk = true

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.2.9200.16384_neutral, right-click on DiskConfiguration/Disk/CreatePartitions/CreatePartition and choose Add setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • Extend = false
  • Order = 1
  • Size = 10000
  • Type = Primary

NOTE: Don’t worry about getting the size exact - just set it to a reasonable minimum. In the next setting, we will extend the partition to fill all remaining disk space on the drive.

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.2.9200.16384_neutral, right-click on DiskConfiguration/Disk/ModifyPartitions/ModifyPartition and choose Add setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • Active = true
  • Extend = true
  • Format = NTFS
  • Letter = C
  • Order = 1
  • PartitionID = 1

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.1.7600.16385_neutral, right-click on ImageInstall/OSImage/InstallTo and choose Add setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • DiskID = 0
  • PartitionID = 1

Disabling the Administrator password prompt

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.2.9200.16384_neutral, right-click on UserAccounts/AdministratorPassword and choose Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • Value = vagrant

Set up vagrant autologin

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.2.9200.16384_neutral, right-click on UserAccounts/LocalAccounts/LocalAccount and choose Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • Description = Vagrant User
  • DisplayName = vagrant
  • Group = Administrators
  • Name = vagrant

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.2.9200.16384_neutral, right-click on UserAccounts/LocalAccounts/LocalAccount/Password and choose Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • Value = vagrant

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.1.9200.16384_neutral, right-click on AutoLogon and choose Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • Enabled = true
  • Username = vagrant

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.2.9200.16384_neutral, right-click on AutoLogon/Password and choose Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • Value = vagrant

Do not show Server Manager at logon

In the Windows Image pane, select the component amd64_Microsoft-Windows-ServerManager-SvrMgrNc_6.2.9200.16384_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • DoNotOpenServerManagerAtLogon = true

Disable User Account Control (UAC)

In the Windows Image pane, select the component amd64_Microsoft-Windows-LUA-Settings_6.2.9200.16384_neutral, right-click and choose Add Setting to Pass 2 offlineServicing. Using the Answer File Properties and Settings panes, configure the following settings:

  • EnableLUA = false

Disable Internet Explorer Enhanced Security Configuration

In the Windows Image pane, select the component amd64_Microsoft-Windows-IE-ESC_10.0.9200.16384_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • IEHardenAdmin = false
  • IEHardenUser = false

Disable Internet Explorer First Run Wizard

In the Windows Image pane, select the component amd64_Microsoft-Windows-IE-InternetExplorer_10.0.9200.16384_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • DisableAccelerators = true
  • DisableFirstRunWizard = true
  • Home_Page = about:blank

In the Windows Image pane, select the component wow64_Microsoft-Windows-IE-InternetExplorer_10.0.9200.16384_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • DisableAccelerators = true
  • DisableFirstRunWizard = true
  • Home_Page = about:blank

Replace Internet Explorer Bing search with Google

In the Windows Image pane, select the component amd64_Microsoft-Windows-IE-InternetExplorer_10.0.9200.16384_neutral, right-click on SearchScopes/Scope and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • ScopeDefault = true
  • ScopeDisplayName = Google
  • ScopeKey = Google
  • ScopeUrl = http://www.google.com/search?q={searchTerms}
  • ShowSearchSuggestions = true

In the Windows Image pane, select the component wow64_Microsoft-Windows-IE-InternetExplorer_10.0.9200.16384_neutral, right-click on SearchScopes/Scope and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • ScopeDefault = true
  • ScopeDisplayName = Google
  • ScopeKey = Google
  • ScopeUrl = http://www.google.com/search?q={searchTerms}
  • ShowSearchSuggestions = **true

Enable Remote Desktop

In the Windows Image pane, select the component amd64_Microsoft-Windows-TerminalServices-LocalSessionManager_6.2.9200.16384_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • fDenyTSConnections = false

In the Windows Image pane, select the component amd64_Networking-MPSSVC-Svc_6.2.9200.16384_neutral, right-click on FirewallGroups/FirewallGroup and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • Active = true
  • Group = Remote Desktop
  • Key = RemoteDesktop
  • Profile = all

In the Windows Image pane, select the component amd64_Microsoft-Windows-TerminalServices-RDP-WinStationExtensions_6.2.9200.16384_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • SecurityLayer = 1
  • UserAuthentication = 0

Turn off computer password

Prevent the image from changing its computer account password, so you can restore old snapshots without being dropped from a domain

REG ADD "HKLM\System\CurrentControlSet\Services\Netlogon\Parameters" /v DisablePasswordChange /t REG_DWORD /d 1 /f

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.2.9200.16384_neutral, right-click FirstLogonCommands/SynchronousCommand” and choose _Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • CommandLine = REG ADD “HKLM\System\CurrentControlSet\Services\Netlogon\Parameters” /v DisablePasswordChange /t REG_DWORD /d 2 /f
  • Description = Disable computer password change
  • Order = 1
  • RequiresUserInput true

Turn off all power saving and timeouts

set-power-config.bat
1
2
3
4
5
REM Set power configuration to High Performance
powercfg -setactive 8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c
REM Monitor timeout
powercfg -Change -monitor-timeout-ac 0
powercfg -Change -monitor-timeout-dc 0

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.2.9200.16384_neutral, right-click FirstLogonCommands/SynchronousCommand” and choose _Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • CommandLine = cmd.exe /c a:set-power-config.bat
  • Description = Turn off all power saving and timeouts
  • Order = 2
  • RequiresUserInput = true

Windows 8 Automated Install Settings

Updated April 03, 2014

  • Updated link to Autounattend.xml

I just recently revised all my automated install XML files for the Windows System Preparation Tool (Sysprep) that I use for my Windows development testbed. For this go around, I’m documenting the XML answer file settings for each version of Microsoft Windows. This article covers the XML answer files settings needed to automate a Windows 8 (64-bit) base OS install.

You’ll need to use the Windows System Image Manager tool to edit Sysprep XML answer files. The Windows System Image Manager is packaged with the Windows Assessment and Deployment Kit tool suite. Download and install the Windows Assessment and Deployment Kit to install the Windows System Image Manager (WSIM).

Link to Autounattend.xml with all the settings in this article. NOTE: Right-click and choose “Download Linked File As…” in your web browser, as many web browsers will try to interpret the Xml.

Disabling the language settings dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-International-Core-WinPE_6.2.9200.16384_neutral, right-click and choose Add Setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • InputLocale = en-US
  • SystemLocale = en-US
  • UILanguage = en-US
  • UserLocale = en-US

Disabling the Select Operating System dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.2.9200.16384_neutral, right-click on ImageInstall/OSImage/InstallFrom/Metadata and choose Add Setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • Key = /IMAGE/NAME
  • Value = Windows 8 Enterprise

Disabling the EULA dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.2.9200.16384_neutral, right-click on UserData and choose Add Setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • AcceptEula = true

Disabling the Disk Allocation dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.2.9200.16384_neutral, right-click on DiskConfiguration and choose Add setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • WillShowUI = OnError

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.2.9200.16384_neutral, right-click on DiskConfiguration/Disk and choose Add setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • DiskID = 0
  • WillWipDisk = true

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.2.9200.16384_neutral, right-click on DiskConfiguration/Disk/CreatePartitions/CreatePartition and choose Add setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • Extend = false
  • Order = 1
  • Size = 10000
  • Type = Primary

NOTE: Don’t worry about getting the size exact - just set it to a reasonable minimum. In the next setting, we will extend the partition to fill all remaining disk space on the drive.

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.2.9200.16384_neutral, right-click on DiskConfiguration/Disk/ModifyPartitions/ModifyPartition and choose Add setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • Active = true
  • Extend = true
  • Format = NTFS
  • Letter = C
  • Order = 1
  • PartitionID = 1

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.1.7600.16385_neutral, right-click on ImageInstall/OSImage/InstallTo and choose Add setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • DiskID = 0
  • PartitionID = 1

Disable PC name dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.2.9200.16384_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • Value - vagrant-win7
  • TimeZone = Pacific Standard Time

Disable Settings dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.2.9200.16384_neutral, right-click on OOBE and choose Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • NetworkLocation = Work
  • ProtectYourPC = 3

Disabling the Sign in to your PC dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.2.9200.16384_neutral, right-click on UserAccounts/LocalAccounts/LocalAccount and choose Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • Description = Vagrant User
  • DisplayName = vagrant
  • Group = Administrators
  • Name = vagrant

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.2.9200.16384_neutral, right-click on UserAccounts/LocalAccounts/LocalAccount/Password and choose Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • Value = vagrant

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.2.9200.16384_neutral, right-click on AutoLogon and choose Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • Enabled = true
  • Username = vagrant

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.1.7601.17514_neutral, right-click on AutoLogon/Password and choose Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • Value = vagrant

Disable User Account Control (UAC)

In the Windows Image pane, select the component amd64_Microsoft-Windows-LUA-Settings_6.2.9200.16384_neutral, right-click and choose Add Setting to Pass 2 offlineServicing. Using the Answer File Properties and Settings panes, configure the following settings:

  • EnableLUA = false

Disable Internet Explorer First Run Wizard

In the Windows Image pane, select the component amd64_Microsoft-Windows-IE-InternetExplorer_10.0.9200.16384_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • DisableAccelerators = true
  • DisableFirstRunWizard = true
  • Home_Page = about:blank

In the Windows Image pane, select the component wow64_Microsoft-Windows-IE-InternetExplorer_10.0.9200.16384_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • DisableAccelerators = true
  • DisableFirstRunWizard = true
  • Home_Page = about:blank

Replace Internet Explorer Bing search with Google

In the Windows Image pane, select the component amd64_Microsoft-Windows-IE-InternetExplorer_10.0.9200.16384_neutral, right-click on SearchScopes/Scope and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • ScopeDefault = true
  • ScopeDisplayName = Google
  • ScopeKey = Google
  • ScopeUrl = http://www.google.com/search?q={searchTerms}
  • ShowSearchSuggestions = true

In the Windows Image pane, select the component wow64_Microsoft-Windows-IE-InternetExplorer_10.0.9200.16384_neutral, right-click on SearchScopes/Scope and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • ScopeDefault = true
  • ScopeDisplayName = Google
  • ScopeKey = Google
  • ScopeUrl = http://www.google.com/search?q={searchTerms}
  • ShowSearchSuggestions = **true

Enable Remote Desktop

In the Windows Image pane, select the component amd64_Microsoft-Windows-TerminalServices-LocalSessionManager_6.2.9200.16384_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • fDenyTSConnections = false

In the Windows Image pane, select the component amd64_Networking-MPSSVC-Svc_6.2.9200.16384_neutral, right-click on FirewallGroups/FirewallGroup and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • Active = true
  • Group = Remote Desktop
  • Key = RemoteDesktop
  • Profile = all

In the Windows Image pane, select the component amd64_Microsoft-Windows-TerminalServices-RDP-WinStationExtensions_6.1.7601.17514_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • SecurityLayer = 1
  • UserAuthentication = 0

Turn off computer password

Prevent the image from changing its computer account password, so you can restore old snapshots without being dropped from a domain

REG ADD "HKLM\System\CurrentControlSet\Services\Netlogon\Parameters" /v DisablePasswordChange /t REG_DWORD /d 1 /f

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.2.9200.16384_neutral, right-click FirstLogonCommands/SynchronousCommand” and choose _Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • CommandLine = REG ADD “HKLM\System\CurrentControlSet\Services\Netlogon\Parameters” /v DisablePasswordChange /t REG_DWORD /d 2 /f
  • Description = Disable computer password change
  • Order = 1
  • RequiresUserInput true

Turn off all power saving and timeouts

set-power-config.bat
1
2
3
4
5
REM Set power configuration to High Performance
powercfg -setactive 8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c
REM Monitor timeout
powercfg -Change -monitor-timeout-ac 0
powercfg -Change -monitor-timeout-dc 0

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.2.9200.16384_neutral, right-click FirstLogonCommands/SynchronousCommand” and choose _Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • CommandLine = cmd.exe /c a:set-power-config.bat
  • Description = Turn off all power saving and timeouts
  • Order = 2
  • RequiresUserInput = true

Windows 7 Automated Install Settings

Updated April 03, 2014

  • Updated link to Autounattend.xml

I just recently revised all my automated install XML files for the Windows System Preparation Tool (Sysprep) that I use for my Windows development testbed. For this go around, I’m documenting the XML answer file settings for each version of Microsoft Windows. This article covers the XML answer files settings needed to automate a Windows 7 (64-bit) base OS install.

You’ll need to use the Windows System Image Manager tool to edit Sysprep XML answer files. The Windows System Image Manager is packaged with the Windows Assessment and Deployment Kit tool suite. Download and install the Windows Assessment and Deployment Kit to install the Windows System Image Manager (WSIM).

Settings to Use for an Unattended Installation

Automate Windows Welcome

Link to Autounattend.xml with all the settings in this article. NOTE: Right-click and choose “Download Linked File As…” in your web browser, as many web browsers will try to interpret the Xml.

Disabling the language settings dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-International-Core-WinPE_6.1.7600.16385_neutral, right-click and choose Add Setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • InputLocale = en-US
  • SystemLocale = en-US
  • UILanguage = en-US
  • UserLocale = en-US

Disabling the Select Operating System dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.1.7600.16385_neutral, right-click on ImageInstall/OSImage/InstallFrom/Metadata and choose Add Setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

For Windows 7 Enterprise:

  • Key = /IMAGE/NAME
  • Value = Windows 7 ENTERPRISE

For Windows 7 Professional:

  • Key = /IMAGE/NAME
  • Value = Windows 7 PROFESSIONAL

Disabling the EULA dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.1.7600.16385_neutral, right-click on UserData and choose Add Setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • AcceptEula = true

Disabling the Disk Allocation dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.1.7600.16385_neutral, right-click on DiskConfiguration and choose Add setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • WillShowUI = OnError

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.1.7600.16385_neutral, right-click on DiskConfiguration/Disk and choose Add setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • DiskID = 0
  • WillWipDisk = true

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.1.7600.16385_neutral, right-click on DiskConfiguration/Disk/CreatePartitions/CreatePartition and choose Add setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • Extend = false
  • Order = 1
  • Size = 10000
  • Type = Primary

NOTE: Don’t worry about getting the size exact - just set it to a reasonable minimum. In the next setting, we will extend the partition to fill all remaining disk space on the drive.

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.1.7600.16385_neutral, right-click on DiskConfiguration/Disk/ModifyPartitions/ModifyPartition and choose Add setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • Active = true
  • Extend = true
  • Format = NTFS
  • Letter = C
  • Order = 1
  • PartitionID = 1

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.1.7600.16385_neutral, right-click on ImageInstall/OSImage/InstallTo and choose Add setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • DiskID = 0
  • PartitionID = 1

Disabling the account and computer name dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.1.7601.17514_neutral, right-click on UserAccounts/LocalAccounts/LocalAccount and choose Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • Description = Vagrant User
  • DisplayName = vagrant
  • Group = Administrators
  • Name = vagrant

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.1.7601.17514_neutral, right-click on UserAccounts/LocalAccounts/LocalAccount/Password and choose Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • Value = vagrant

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.1.7601.17514_neutral, right-click on AutoLogon and choose Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • Enabled = true
  • Username = vagrant

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.1.7601.17514_neutral, right-click on AutoLogon/Password and choose Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • Value = vagrant

Disable Computer Name dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.1.7601.17514_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • Value - vagrant-win7
  • TimeZone = Pacific Standard Time

Disable Protect Computer dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.1.7601.17514_neutral, right-click on OOBE and choose Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • NetworkLocation = Work
  • ProtectYourPC = 3

Disable User Account Control (UAC)

In the Windows Image pane, select the component amd64_Microsoft-Windows-LUA-Settings_6.1.7600.16385_neutral, right-click and choose Add Setting to Pass 2 offlineServicing. Using the Answer File Properties and Settings panes, configure the following settings:

  • EnableLUA = false

Disable Internet Explorer First Run Wizard

In the Windows Image pane, select the component amd64_Microsoft-Windows-IE-InternetExplorer_8.0.7600.16385_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • DisableAccelerators = true
  • DisableFirstRunWizard = true
  • Home_Page = about:blank

In the Windows Image pane, select the component wow64_Microsoft-Windows-IE-InternetExplorer_8.0.7601.17514_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • DisableAccelerators = true
  • DisableFirstRunWizard = true
  • Home_Page = about:blank

Replace Internet Explorer Bing search with Google

In the Windows Image pane, select the component amd64_Microsoft-Windows-IE-InternetExplorer_8.0.7600.16385_neutral, right-click on SearchScopes/Scope and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • ScopeDefault = true
  • ScopeDisplayName = Google
  • ScopeKey = Google
  • ScopeUrl = http://www.google.com/search?q={searchTerms}
  • ShowSearchSuggestions = true

In the Windows Image pane, select the component wow64_Microsoft-Windows-IE-InternetExplorer_8.0.7601.17514_neutral, right-click on SearchScopes/Scope and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • ScopeDefault = true
  • ScopeDisplayName = Google
  • ScopeKey = Google
  • ScopeUrl = http://www.google.com/search?q={searchTerms}
  • ShowSearchSuggestions = true

Enable Remote Desktop

In the Windows Image pane, select the component amd64_Microsoft-Windows-TerminalServices-LocalSessionManager_6.1.7601.17514_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • fDenyTSConnections = false

In the Windows Image pane, select the component amd64_Networking-MPSSVC-Svc_6.1.7601.175414_neutral, right-click on FirewallGroups/FirewallGroup and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • Active = true
  • Group = Remote Desktop
  • Key = RemoteDesktop
  • Profile = all

In the Windows Image pane, select the component amd64_Microsoft-Windows-TerminalServices-RDP-WinStationExtensions_6.1.7601.17514_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • SecurityLayer = 1
  • UserAuthentication = 0

(Really) Disable Set Network Location Prompt

REG ADD "HKLM\System\CurrentControlSet\Control\Network\NewNetworkWindowOff"

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.1.7601.17514_neutral, right-click FirstLogonCommands/SynchronousCommand and choose Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • CommandLine = REG ADD “HKLM\System\CurrentControlSet\Control\Network\NewNetworkWindowOff”
  • Description = Disable Set Network Location Prompt
  • Order = 1
  • RequiresUserInput true

Turn off computer password

Prevent the image from changing its computer account password, so you can restore old snapshots without being dropped from a domain

REG ADD "HKLM\System\CurrentControlSet\Services\Netlogon\Parameters" /v DisablePasswordChange /t REG_DWORD /d 1 /f

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.1.7601.17514_neutral, right-click FirstLogonCommands/SynchronousCommand” and choose _Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • CommandLine = REG ADD “HKLM\System\CurrentControlSet\Services\Netlogon\Parameters” /v DisablePasswordChange /t REG_DWORD /d 2 /f
  • Description = Disable computer password change
  • Order = 2
  • RequiresUserInput true

Turn off all power saving and timeouts

set-power-config.bat
1
2
3
4
5
REM Set power configuration to High Performance
powercfg -setactive 8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c
REM Monitor timeout
powercfg -Change -monitor-timeout-ac 0
powercfg -Change -monitor-timeout-dc 0

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.1.7601.17514_neutral, right-click FirstLogonCommands/SynchronousCommand” and choose _Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • CommandLine = cmd.exe /c a:set-power-config.bat
  • Description = Turn off all power saving and timeouts
  • Order = 3
  • RequiresUserInput = true

Windows Server 2008 R2 Automated Install Settings

Updated April 03, 2014

  • Updated link to Autounattend.xml

I just recently revised all my automated install XML files for the Windows System Preparation Tool (Sysprep) that I use for my Windows development testbed. For this go around, I’m documenting the XML answer file settings for each version of Microsoft Windows. This article covers the XML answer files settings needed to automate a Windows Server 2008 R2 (64-bit) base OS install.

You’ll need to use the Windows System Image Manager tool to edit Sysprep XML answer files. The Windows System Image Manager is packaged with the Windows Assessment and Deployment Kit tool suite. Download and install the Windows Assessment and Deployment Kit to install the Windows System Image Manager (WSIM).

Link to Autounattend.xml with all the settings in this article. NOTE: Right-click and choose “Download Linked File As…” in your web browser, as many web browsers will try to interpret the Xml.

Disabling the language settings dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-International-Core-WinPE_6.1.7600.16385_neutral, right-click and choose Add Setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • InputLocale = en-US
  • SystemLocale = en-US
  • UILanguage = en-US
  • UserLocale = en-US

Disabling the Select Operating System dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.1.7600.16385_neutral, right-click on ImageInstall/OSImage/InstallFrom/Metadata and choose Add Setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • Key = /IMAGE/NAME
  • Value = Windows Server 2008 R2 SERVERDATACENTER

NOTE: Make sure the /IMAGE/NAME value matches the Windows 2008R2 Image flavor you selected. Possible values are:

  • Windows Server 2008 R2 SERVERDATACENTER
  • Windows Server 2008 R2 SERVERDATACENTERCORE
  • Windows Server 2008 R2 SERVERENTERPRISE
  • Windows Server 2008 R2 SERVERENTERPRISECORE
  • Windows Server 2008 R2 SERVERSTANDARD
  • Windows Server 2008 R2 SERVERSTANDARDCORE
  • Windows Server 2008 R2 SERVERWEB
  • Windows Server 2008 R2 SERVERWEBCORE

Disabling the EULA dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.1.7600.16385_neutral, right-click on UserData and choose Add Setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • AcceptEula = true

Disabling the Disk Allocation dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.1.7600.16385_neutral, right-click on DiskConfiguration and choose Add setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • WillShowUI = OnError

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.1.7600.16385_neutral, right-click on DiskConfiguration/Disk and choose Add setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • DiskID = 0
  • WillWipDisk = true

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.1.7600.16385_neutral, right-click on DiskConfiguration/Disk/CreatePartitions/CreatePartition and choose Add setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • Extend = false
  • Order = 1
  • Size = 10000
  • Type = Primary

NOTE: Don’t worry about getting the size exact - just set it to a reasonable minimum. In the next setting, we will extend the partition to fill all remaining disk space on the drive.

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.1.7600.16385_neutral, right-click on DiskConfiguration/Disk/ModifyPartitions/ModifyPartition and choose Add setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • Active = true
  • Extend = true
  • Format = NTFS
  • Letter = C
  • Order = 1
  • PartitionID = 1

In the Windows Image pane, select the component amd64_Microsoft-Windows-Setup_6.1.7600.16385_neutral, right-click on ImageInstall/OSImage/InstallTo and choose Add setting to Pass 1 windowsPE. Using the Answer File Properties and Settings panes, configure the following settings:

  • DiskID = 0
  • PartitionID = 1

Disabling the Administrator password prompt

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.1.7601.17514_neutral, right-click on UserAccounts/AdministratorPassword and choose Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • Value = vagrant

Set up vagrant autologin

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.1.7601.17514_neutral, right-click on UserAccounts/LocalAccounts/LocalAccount and choose Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • Description = Vagrant User
  • DisplayName = vagrant
  • Group = Administrators
  • Name = vagrant

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.1.7601.17514_neutral, right-click on UserAccounts/LocalAccounts/LocalAccount/Password and choose Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • Value = vagrant

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.1.7601.17514_neutral, right-click on AutoLogon and choose Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • Enabled = true
  • Username = vagrant

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.1.7601.17514_neutral, right-click on AutoLogon/Password and choose Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • Value = vagrant

Disable Initial Configuration Dialog

In the Windows Image pane, select the component amd64_Microsoft-Windows-OutOfBoxExperience_6.1.7600.16385_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • DoNotOpenInitialConfigurationTasksAtLogon = true

Do not show Server Manager at logon

In the Windows Image pane, select the component amd64_Microsoft-Windows-ServerManager-SvrMgrNc_6.1.7600.16385_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • DoNotOpenServerManagerAtLogon = true

Disable User Account Control (UAC)

In the Windows Image pane, select the component amd64_Microsoft-Windows-LUA-Settings_6.1.7600.16385_neutral, right-click and choose Add Setting to Pass 2 offlineServicing. Using the Answer File Properties and Settings panes, configure the following settings:

  • EnableLUA = false

Disable Internet Explorer Enhanced Security Configuration

In the Windows Image pane, select the component amd64_Microsoft-Windows-IE-ESC_8.0.7601.17514_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • IEHardenAdmin = false
  • IEHardenUser = false

Disable Internet Explorer First Run Wizard

In the Windows Image pane, select the component amd64_Microsoft-Windows-IE-InternetExplorer_8.0.7600.16385_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • DisableAccelerators = true
  • DisableFirstRunWizard = true
  • Home_Page = about:blank

In the Windows Image pane, select the component wow64_Microsoft-Windows-IE-InternetExplorer_8.0.7601.17514_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • DisableAccelerators = true
  • DisableFirstRunWizard = true
  • Home_Page = about:blank

Replace Internet Explorer Bing search with Google

In the Windows Image pane, select the component amd64_Microsoft-Windows-IE-InternetExplorer_8.0.7600.16385_neutral, right-click on SearchScopes/Scope and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • ScopeDefault = true
  • ScopeDisplayName = Google
  • ScopeKey = Google
  • ScopeUrl = http://www.google.com/search?q={searchTerms}
  • ShowSearchSuggestions = true

In the Windows Image pane, select the component wow64_Microsoft-Windows-IE-InternetExplorer_8.0.7601.17514_neutral, right-click on SearchScopes/Scope and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • ScopeDefault = true
  • ScopeDisplayName = Google
  • ScopeKey = Google
  • ScopeUrl = http://www.google.com/search?q={searchTerms}
  • ShowSearchSuggestions = true

Enable Remote Desktop

In the Windows Image pane, select the component amd64_Microsoft-Windows-TerminalServices-LocalSessionManager_6.1.7601.17514_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • fDenyTSConnections = false

In the Windows Image pane, select the component amd64_Networking-MPSSVC-Svc_6.1.7601.175414_neutral, right-click on FirewallGroups/FirewallGroup and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • Active = true
  • Group = Remote Desktop
  • Key = RemoteDesktop
  • Profile = all

In the Windows Image pane, select the component amd64_Microsoft-Windows-TerminalServices-RDP-WinStationExtensions_6.1.7601.17514_neutral, right-click and choose Add Setting to Pass 4 specialize. Using the Answer File Properties and Settings panes, configure the following settings:

  • SecurityLayer = 1
  • UserAuthentication = 0

Turn off computer password

Prevent the image from changing its computer account password, so you can restore old VM snapshots without being dropped from a domain

REG ADD "HKLM\System\CurrentControlSet\Services\Netlogon\Parameters" /v DisablePasswordChange /t REG_DWORD /d 1 /f

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.1.7601.17514_neutral, right-click FirstLogonCommands/SynchronousCommand” and choose _Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • CommandLine = REG ADD “HKLM\System\CurrentControlSet\Services\Netlogon\Parameters” /v DisablePasswordChange /t REG_DWORD /d 1 /f
  • Description = Disable computer password change
  • Order = 1
  • RequiresUserInput true

Turn off all power saving and timeouts

set-power-config.bat
1
2
3
4
5
REM Set power configuration to High Performance
powercfg -setactive 8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c
REM Monitor timeout
powercfg -Change -monitor-timeout-ac 0
powercfg -Change -monitor-timeout-dc 0

In the Windows Image pane, select the component amd64_Microsoft-Windows-Shell-Setup_6.1.7601.17514_neutral, right-click FirstLogonCommands/SynchronousCommand” and choose _Add Setting to Pass 7 oobeSystem. Using the Answer File Properties and Settings panes, configure the following settings:

  • CommandLine = cmd.exe /c a:set-power-config.bat
  • Description = Turn off all power saving and timeouts
  • Order = 2
  • RequiresUserInput = true

Getting Started Writing Chef Cookbooks the Berkshelf Way, Part 2

Updated December 29, 2013

  • Bumped apache2 cookbook reference from 1.7.x to 1.8.x
  • Bumped database cookbook reference from 1.4.x to 1.6.x
  • Per Nicholas Johns removed php 5.5 deprecated mysql* functions_
  • Replaced symbol references with strings to match part 1 changes
  • Added Windows instructions

Updated September 1, 2013

  • Bumped apache2 cookbook reference from 1.6.x to 1.7.x
  • Bumped database cookbook reference from 1.3.x to 1.4.x

Updated August 7, 2013

  • Fixed error in Iteration #10 test per Jeff Thomas

Updated July 23rd, 2013

  • Referenced Sean OMeara’s & Charles Johnson’s latest myface example app

This is a second article in a series on writing Opscode Chef cookbooks the Berkshelf Way. Here’s a link to Part 1. The source code examples covered in this article can be found on Github: https://github.com/misheska/myface

In this installment, Part 2, we’re going to create a new database recipe. In Part1 MyFace is just a web application serving up a static page. Now we’re going to enhance MyFace so that it stores account information in a persistent MySQL database.

Thanks go out to the Opscode Advanced Chef Cookbook Authoring class and specifically Sean OMeara and Charles Johnson for the database and PHP code used in this article.

Iteration #7 - Install MySQL

Edit metadata.rb and add a reference to the mysql cookbook. Also bump the version to 2.0.0 because we know that there will be incompatible API changes, moving to MySQL, per Semantic Versioning:

myface/metadata.rb
1
2
3
4
5
6
7
8
9
10
name             'myface'
maintainer       'Mischa Taylor'
maintainer_email 'mischa@misheska.com'
license          'Apache 2.0'
description      'Installs/Configures myface'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version          '2.0.0'

depends 'apache2', '~> 1.8.0'
depends 'mysql', '~> 4.0.0'

Create a new recipe called recipes/database.rb which includes the MySQL cookbook’s server recipe (this is a similar abstraction to what you created in Part 1 with recipes/webserver.rb):

myface/recipes/database.rb
1
2
3
4
5
6
7
8
9
10
#
# Cookbook Name:: myface
# Recipe:: database
#
# Copyright (C) 2012 YOUR_NAME
# 
# All rights reserved - Do Not Redistribute
#

include_recipe 'mysql::server'

Wire the database recipe into the MyFace cookbook by adding an include_recipe reference to recipes/default.rb:

myface/recipes/default.rb
1
2
3
4
5
6
7
8
9
10
# Cookbook Name:: myface
# Recipe:: default
#
# Copyright (C) 2013 YOUR_NAME
#
# All rights reserved - Do Not Redistribute
#

include_recipe 'myface::database'
include_recipe 'myface::webserver'

Run vagrant provision to converge your changes.

$ vagrant provision
[Berkshelf] This version of the Berkshelf plugin has not been fully tested on this version of Vagrant.
[Berkshelf] You should check for a newer version of vagrant-berkshelf.
[Berkshelf] If you encounter any errors with this version, please report them at https://github.com/RiotGames/vagrant-berkshelf/issues
[Berkshelf] You can also join the discussion in #berkshelf on Freenode.
[Berkshelf] Updating Vagrant's berkshelf: '/Users/misheska/.berkshelf/default/vagrant/berkshelf-20131228-44581-4bhc9d-default'
[Berkshelf] Using myface (2.0.0)
[Berkshelf] Using apache2 (1.8.14)
[Berkshelf] Installing mysql (4.0.14) from site: 'http://cookbooks.opscode.com/api/v1/cookbooks'
[Berkshelf] Installing openssl (1.1.0) from site: 'http://cookbooks.opscode.com/api/v1/cookbooks'
[Berkshelf] Installing build-essential (1.4.2) from site: 'http://cookbooks.opscode.com/api/v1/cookbooks'
[default] Chef 11.8.2 Omnibus package is already installed.
[default] Running provisioner: chef_solo...
Generating chef JSON and uploading...
Running chef-solo...
[2013-12-28T23:45:15-08:00] INFO: Forking chef instance to converge...
[2013-12-28T23:45:15-08:00] INFO: *** Chef 11.8.2 ***
[2013-12-28T23:45:15-08:00] INFO: Chef-client pid: 28502
[2013-12-28T23:45:16-08:00] INFO: Setting the run_list to ["recipe[myface::default]"] from JSON
[2013-12-28T23:45:16-08:00] INFO: Run List is [recipe[myface::default]]
[2013-12-28T23:45:16-08:00] INFO: Run List expands to [myface::default]
[2013-12-28T23:45:16-08:00] INFO: Starting Chef Run for myface-berkshelf
[2013-12-28T23:45:16-08:00] INFO: Running start handlers
[2013-12-28T23:45:16-08:00] INFO: Start handlers complete.
[2013-12-28T23:45:16-08:00] WARN: Cloning resource attributes for directory[/var/lib/mysql] from prior resource (CHEF-3694)
[2013-12-28T23:45:16-08:00] WARN: Previous directory[/var/lib/mysql]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/mysql/recipes/_server_rhel.rb:11:in `block in from_file'
[2013-12-28T23:45:16-08:00] WARN: Current  directory[/var/lib/mysql]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/mysql/recipes/_server_rhel.rb:20:in `from_file'
[2013-12-28T23:45:16-08:00] WARN: Cloning resource attributes for service[apache2] from prior resource (CHEF-3694)
[2013-12-28T23:45:16-08:00] WARN: Previous service[apache2]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/apache2/recipes/default.rb:24:in `from_file'
[2013-12-28T23:45:16-08:00] WARN: Current  service[apache2]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/apache2/recipes/default.rb:210:in `from_file'
[2013-12-28T23:45:33-08:00] INFO: package[mysql-server] installing mysql-server-5.1.71-1.el6 from base repository
[2013-12-28T23:45:43-08:00] INFO: directory[/var/log/mysql] created directory /var/log/mysql
[2013-12-28T23:45:43-08:00] INFO: directory[/var/log/mysql] owner changed to 27
[2013-12-28T23:45:43-08:00] INFO: directory[/var/log/mysql] group changed to 27
[2013-12-28T23:45:43-08:00] INFO: directory[/var/log/mysql] mode changed to 755
[2013-12-28T23:45:43-08:00] INFO: directory[/etc/mysql/conf.d] created directory /etc/mysql/conf.d
[2013-12-28T23:45:43-08:00] INFO: directory[/etc/mysql/conf.d] owner changed to 27
[2013-12-28T23:45:43-08:00] INFO: directory[/etc/mysql/conf.d] group changed to 27
[2013-12-28T23:45:43-08:00] INFO: directory[/etc/mysql/conf.d] mode changed to 755
[2013-12-28T23:45:43-08:00] INFO: template[initial-my.cnf] backed up to /var/chef/backup/etc/my.cnf.chef-20131228234543.651750
[2013-12-28T23:45:43-08:00] INFO: template[initial-my.cnf] updated file contents /etc/my.cnf
[2013-12-28T23:45:43-08:00] INFO: template[initial-my.cnf] sending start action to service[mysql-start] (immediate)
[2013-12-28T23:45:45-08:00] INFO: service[mysql-start] started
[2013-12-28T23:45:45-08:00] INFO: execute[assign-root-password] ran successfully
[2013-12-28T23:45:45-08:00] INFO: template[/etc/mysql_grants.sql] created file /etc/mysql_grants.sql
[2013-12-28T23:45:45-08:00] INFO: template[/etc/mysql_grants.sql] updated file contents /etc/mysql_grants.sql
[2013-12-28T23:45:45-08:00] INFO: template[/etc/mysql_grants.sql] owner changed to 0
[2013-12-28T23:45:45-08:00] INFO: template[/etc/mysql_grants.sql] group changed to 0
[2013-12-28T23:45:45-08:00] INFO: template[/etc/mysql_grants.sql] mode changed to 600
[2013-12-28T23:45:45-08:00] INFO: template[/etc/mysql_grants.sql] sending run action to execute[install-grants] (immediate)
[2013-12-28T23:45:45-08:00] INFO: execute[install-grants] ran successfully
[2013-12-28T23:45:45-08:00] INFO: execute[install-grants] sending restart action to service[mysql] (immediate)
[2013-12-28T23:45:50-08:00] INFO: service[mysql] restarted
[2013-12-28T23:45:50-08:00] INFO: service[mysql] enabled
[2013-12-28T23:45:52-08:00] INFO: Chef Run complete in 35.813790999 seconds
[2013-12-28T23:45:52-08:00] INFO: Running report handlers
[2013-12-28T23:45:52-08:00] INFO: Report handlers complete
[2013-12-28T23:45:15-08:00] INFO: Forking chef instance to converge...

Testing Iteration #7

Verify that the mysqld service is running on your vagrant guest by running the following command on Mac OS X/Linux:

$ vagrant ssh -c "sudo /sbin/service mysqld status"
mysqld (pid  8525) is running...

And on Windows:

> vagrant ssh -c "sudo /sbin/service mysqld status" -- -n -T
mysqld (pid  8525) is running...

Also check that MySQL is enabled to start on boot on Mac OS X/Linux:

$ vagrant ssh -c "sudo /sbin/chkconfig --list | grep mysqld"
mysqld         	0:off	1:off	2:on	3:on	4:on	5:on	6:off

And on Windows:

> vagrant ssh -c "sudo /sbin/chkconfig --list | grep mysqld" -- -n -T
mysqld         	0:off	1:off	2:on	3:on	4:on	5:on	6:off

If the service is set to be activated at runlevels 3 and 5, then MySQL is enabled to run under full multi-user text mode and full multi-user graphical mode, which is exactly the desired behavior.

Iteration #8 - Create the MyFace Database

We’ve installed MySQL, but we don’t have a database yet. Now we’re going to create a database to store information about our users with another cookbook, the database cookbook.

Add the database cookbook as a dependency in the metadata.rb file:

myface/metadata.rb
1
2
3
4
5
6
7
8
9
10
11
name             'myface'
maintainer       'Mischa Taylor'
maintainer_email 'mischa@misheska.com'
license          'Apache 2.0'
description      'Installs/Configures myface'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version          '2.0.0'

depends 'apache2', '~> 1.8.0'
depends 'mysql', '~> 4.0.0'
depends 'database', '~> 1.6.0'

Berkshelf automatically populates MySQL passwords for you. They were configured in the Vagrantfile when you ran berks cookbook in Part 1:

...
config.vm.provision :chef_solo do |chef|
  chef.json = {
    :mysql => {
      :server_root_password => 'rootpass',
      :server_debian_password => 'debpass',
      :server_repl_password => "replpass'
    }
  }
  ...
end
...

You can reference these passwords as variables in your Chef recipes, which we will do when we add some data attributes. Add the following attributes to attributes/default.rb so it looks like so:

myface/attributes/default.rb
1
2
3
4
5
6
7
8
9
10
default['myface']['user'] = 'myface'
default['myface']['group'] = 'myface'
default['myface']['name'] = 'myface'
default['myface']['config'] = 'myface.conf'
default['myface']['document_root'] = '/srv/apache/myface'

default['myface']['database']['host'] = 'localhost'
default['myface']['database']['username'] = 'root'
default['myface']['database']['password'] = node['mysql']['server_root_password']
default['myface']['database']['dbname'] = 'myface'

Describe the database to be created for MyFace in recipes/database.rb:

myface/recipes/database.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
#
# Cookbook Name:: myface
# Recipe:: database
#
# Copyright (C) 2012 YOUR_NAME
#
# All rights reserved - Do Not Redistribute
#

include_recipe 'mysql::server'
include_recipe 'database::mysql'

mysql_database node['myface']['database']['dbname'] do
  connection(
    :host => node['myface']['database']['host'],
    :username => node['myface']['database']['username'],
    :password => node['myface']['database']['password']
  )
  action :create
end

Converge the changes with vagrant provision:

$ vagrant provision
[Berkshelf] This version of the Berkshelf plugin has not been fully tested on this version of Vagrant.
[Berkshelf] You should check for a newer version of vagrant-berkshelf.
[Berkshelf] If you encounter any errors with this version, please report them at https://github.com/RiotGames/vagrant-berkshelf/issues
[Berkshelf] You can also join the discussion in #berkshelf on Freenode.
[Berkshelf] Updating Vagrant's berkshelf: '/Users/misheska/.berkshelf/default/vagrant/berkshelf-20131229-47158-1y8wp6-default'
[Berkshelf] Using myface (2.0.0)
[Berkshelf] Using apache2 (1.8.14)
[Berkshelf] Using mysql (4.0.14)
[Berkshelf] Using openssl (1.1.0)
[Berkshelf] Using build-essential (1.4.2)
[Berkshelf] Using database (1.6.0)
[Berkshelf] Using postgresql (3.3.4)
[Berkshelf] Using apt (2.3.4)
[Berkshelf] Using aws (1.0.0)
[Berkshelf] Using xfs (1.1.0)
[default] Chef 11.8.2 Omnibus package is already installed.
[default] Running provisioner: chef_solo...
Generating chef JSON and uploading...
Running chef-solo...
[2013-12-29T00:40:09-08:00] INFO: Forking chef instance to converge...
[2013-12-29T00:40:09-08:00] INFO: *** Chef 11.8.2 ***
[2013-12-29T00:40:09-08:00] INFO: Chef-client pid: 5081
[2013-12-29T00:40:10-08:00] INFO: Setting the run_list to ["recipe[myface::default]"] from JSON
[2013-12-29T00:40:10-08:00] INFO: Run List is [recipe[myface::default]]
[2013-12-29T00:40:10-08:00] INFO: Run List expands to [myface::default]
[2013-12-29T00:40:10-08:00] INFO: Starting Chef Run for myface-berkshelf
[2013-12-29T00:40:10-08:00] INFO: Running start handlers
[2013-12-29T00:40:10-08:00] INFO: Start handlers complete.
[2013-12-29T00:40:10-08:00] WARN: Cloning resource attributes for directory[/var/lib/mysql] from prior resource (CHEF-3694)
[2013-12-29T00:40:10-08:00] WARN: Previous directory[/var/lib/mysql]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/mysql/recipes/_server_rhel.rb:11:in `block in from_file'
[2013-12-29T00:40:10-08:00] WARN: Current  directory[/var/lib/mysql]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/mysql/recipes/_server_rhel.rb:20:in `from_file'
[2013-12-29T00:40:11-08:00] INFO: package[autoconf] installing autoconf-2.63-5.1.el6 from base repository
[2013-12-29T00:40:18-08:00] INFO: package[bison] installing bison-2.4.1-5.el6 from base repository
[2013-12-29T00:40:25-08:00] INFO: package[flex] installing flex-2.5.35-8.el6 from base repository
[2013-12-29T00:40:31-08:00] INFO: package[gcc-c++] installing gcc-c++-4.4.7-4.el6 from base repository
[2013-12-29T00:40:41-08:00] INFO: package[mysql-devel] installing mysql-devel-5.1.71-1.el6 from base repository
[2013-12-29T00:41:17-08:00] WARN: Cloning resource attributes for service[apache2] from prior resource (CHEF-3694)
[2013-12-29T00:41:17-08:00] WARN: Previous service[apache2]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/apache2/recipes/default.rb:24:in `from_file'
[2013-12-29T00:41:17-08:00] WARN: Current  service[apache2]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/apache2/recipes/default.rb:210:in `from_file'
[2013-12-29T00:41:19-08:00] INFO: Chef Run complete in 69.220896743 seconds
[2013-12-29T00:41:19-08:00] INFO: Running report handlers
[2013-12-29T00:41:19-08:00] INFO: Report handlers complete
[2013-12-29T00:40:09-08:00] INFO: Forking chef instance to converge...

Testing Iteration #8

Run mysqlshow on your vagrant guest to display database information, verifying that the myface database was created on Mac OS X/Linux:

$ vagrant ssh -c "mysqlshow -uroot -prootpass"
+--------------------+
|     Databases      |
+--------------------+
| information_schema |
| myface             |
| mysql              |
| test               |
+--------------------+

And on Windows:

> vagrant ssh -c "mysqlshow -uroot -prootpass" -- -n -T

Note that myface is listed as a database name - success!

Iteration #9 - Create a MySQL user

It’s a good idea to create a user in MySQL for each one of your applications that has the ability to only manipulate the application’s database and has no MySQL administrative privileges.

Add some attributes to attributes/default.rb for your app user:

default['myface']['database']['app']['username'] = 'myface_app'
default['myface']['database']['app']['password'] = 'supersecret'

attributes/default.rb should look like so:

myface/attributes/default.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
default['myface']['user'] = 'myface'
default['myface']['group'] = 'myface'
default['myface']['name'] = 'myface'
default['myface']['config'] = 'myface.conf'
default['myface']['document_root'] = '/srv/apache/myface'

default['myface']['database']['host'] = 'localhost'
default['myface']['database']['username'] = 'root'
default['myface']['database']['password'] = node['mysql']['server_root_password']
default['myface']['database']['dbname'] = 'myface'

default['myface']['database']['app']['username'] = 'myface_app'
default['myface']['database']['app']['password'] = 'supersecret'

Edit recipes/database.rb and describe the MySQL database user:

...
  )
  action :create
end

mysql_database_user node['myface']['database']['app']['username'] do
  connection(
    :host => node['myface']['database']['host'],
    :username => node['myface']['database']['username'],
    :password => node['myface']['database']['password']
  )
  password node['myface']['database']['app']['password']
  database_name node['myface']['database']['dbname']
  host node['myface']['database']['host']
  action [:create, :grant]
end

After editing recipes/database.rb should look like the following:

myface/recipes/database.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
#
# Cookbook Name:: myface
# Recipe:: database
#
# Copyright (C) 2012 YOUR_NAME
#
# All rights reserved - Do Not Redistribute
#

include_recipe 'mysql::server'
include_recipe 'database::mysql'

mysql_database node['myface']['database']['dbname'] do
  connection(
    :host => node['myface']['database']['host'],
    :username => node['myface']['database']['username'],
    :password => node['myface']['database']['password']
  )
  action :create
end

mysql_database_user node['myface']['database']['app']['username'] do
  connection(
    :host => node['myface']['database']['host'],
    :username => node['myface']['database']['username'],
    :password => node['myface']['database']['password']
  )
  password node['myface']['database']['app']['password']
  database_name node['myface']['database']['dbname']
  host node['myface']['database']['host']
  action [:create, :grant]
end

Converge the node to apply the changes:

$ vagrant provision

Testing Iteration #9

Check to see if the myface-app user is enabled as a local user by running the following mysql command on Mac OS X/Linux:

$ vagrant ssh -c 'mysql -uroot -prootpass -e "select user,host from mysql.user;"'
+------------+------------------+
| user       | host             |
+------------+------------------+
| repl       | %                |
| root       | 127.0.0.1        |
|            | localhost        |
| myface_app | localhost        |
| root       | localhost        |
|            | myface-berkshelf |
+------------+------------------+
Connection to 127.0.0.1 closed.

And on Windows:

> vagrant ssh -c "mysql -uroot -prootpass -e 'select user,host from mysql.user;'" -- -n -T

As you can see above, the myface_app@localhost user exists, so our cookbook did what was expected.

Also check to see that the myface_app user only has rights on the myface databse on Mac OS X/Linux:

$ vagrant ssh -c 'mysql -uroot -prootpass -e "show grants for 'myface_app'@'localhost';"'
Grants for myface_app@localhost
GRANT USAGE ON *.* TO 'myface_app'@'localhost' IDENTIFIED BY PASSWORD '*90BA3AC0BFDE07AE334CA523CB27167AE33825B9'
GRANT ALL PRIVILEGES ON `myface`.* TO 'myface_app'@'localhost'

And on Windows:

> vagrant ssh -c "mysql -uroot -prootpass -e 'show grants for "myface_app"@"localhost";'" -- -n -T

Iteration #10 - Create a table for users

Let’s create a SQL script to create a table modeling MyFace users and populate it with some initial data. Create a file files/default/myface-create.sql with the following content:

myface/files/default/myface-create.sql
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
/* A table for myface users */

CREATE TABLE users(
 id CHAR (32) NOT NULL,
 PRIMARY KEY(id),
 user_name VARCHAR(64),
 url VARCHAR(256),
 email VARCHAR(128),
 neck_beard INTEGER
);

/* Initial records */
INSERT INTO users ( id, user_name, url, email, neck_beard ) VALUES ( uuid(), 'jtimberman', 'http://jtimberman.housepub.org', 'joshua@opscode.com', 4 );
INSERT INTO users ( id, user_name, url, email, neck_beard ) VALUES ( uuid(), 'someara', 'http://blog.afistfulofservers.net/', 'someara@opscode.com', 5 );
INSERT INTO users ( id, user_name, url, email, neck_beard ) VALUES ( uuid(), 'jwinsor', 'http://vialstudios.com', 'jamie@vialstudios.com', 4 );
INSERT INTO users ( id, user_name, url, email, neck_beard ) VALUES ( uuid(), 'cjohnson', 'http://www.chipadeedoodah.com/', 'charles@opscode.com', 3 );
INSERT INTO users ( id, user_name, url, email, neck_beard ) VALUES ( uuid(), 'mbower', 'http://www.webbower.com/', 'matt@webbower.com', 4 );

Add an attribute for the temporary location used for the SQL script we just created:

default['myface']['database']['seed_file'] = '/tmp/myface-create.sql'

The resultant attributes/default.rb should resemble the following:

myface/attributes/default.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
default['myface']['user'] = 'myface'
default['myface']['group'] = 'myface'
default['myface']['name'] = 'myface'
default['myface']['config'] = 'myface.conf'
default['myface']['document_root'] = '/srv/apache/myface'

default['myface']['database']['host'] = 'localhost'
default['myface']['database']['username'] = 'root'
default['myface']['database']['password'] = node['mysql']['server_root_password']
default['myface']['database']['dbname'] = 'myface'

default['myface']['database']['app']['username'] = 'myface_app'
default['myface']['database']['app']['password'] = 'supersecret'

default['myface']['database']['seed_file'] = '/tmp/myface-create.sql'

Modify recipes/database.rb so that the cookbook transfers the SQL script to the guest node and so that the SQL script executes. As you learned in Part 1, recipes should be idempotent, so you will need to add a not_if statement which ensures that the command is only executed when necessary.

...
  host 'localhost'
  action [:create, :grant]
end

# Write schema seed file to filesystem
cookbook_file node['myface']['database']['seed_file'] do
  source 'myface-create.sql'
  owner 'root'
  group 'root'
  mode '0600'
end

# Seed database with test data
execute 'initialize myface database' do
  command "mysql -h #{node['myface']['database']['host']} -u #{node['myface']['database']['app']['username']} -p#{node['myface']['database']['app']['password']} -D #{node['myface']['database']['dbname']} < #{node['myface']['database']['seed_file']}"
  not_if  "mysql -h #{node['myface']['database']['host']} -u #{node['myface']['database']['app']['username']} -p#{node['myface']['database']['app']['password']} -D #{node['myface']['database']['dbname']} -e 'describe users;'"
end

Once you have made these changes, recipes/database.rb should look like so:

myface/recipes/database.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
#
# Cookbook Name:: myface
# Recipe:: database
#
# Copyright (C) 2012 YOUR_NAME
#
# All rights reserved - Do Not Redistribute
#

include_recipe 'mysql::server'
include_recipe 'database::mysql'

mysql_database node['myface']['database']['dbname'] do
  connection(
    :host => node['myface']['database']['host'],
    :username => node['myface']['database']['username'],
    :password => node['myface']['database']['password']
  )
  action :create
end

mysql_database_user node['myface']['database']['app']['username'] do
  connection(
    :host => node['myface']['database']['host'],
    :username => node['myface']['database']['username'],
    :password => node['myface']['database']['password']
  )
  password node['myface']['database']['app']['password']
  database_name node['myface']['database']['dbname']
  host node['myface']['database']['host']
  action [:create, :grant]
end

# Write schema seed file to filesystem
cookbook_file node['myface']['database']['seed_file'] do
  source 'myface-create.sql'
  owner 'root'
  group 'root'
  mode '0600'
end

# Seed database with test data
execute 'initialize myface database' do
  command "mysql -h #{node['myface']['database']['host']} -u #{node['myface']['database']['app']['username']} -p#{node['myface']['database']['app']['password']} -D #{node['myface']['database']['dbname']} < #{node['myface']['database']['seed_file']}"
  not_if  "mysql -h #{node['myface']['database']['host']} -u #{node['myface']['database']['app']['username']} -p#{node['myface']['database']['app']['password']} -D #{node['myface']['database']['dbname']} -e 'describe users;'"
end

Run vagrant provision to converge your changes:

$ vagrant provision

Testing Iteration #10

Run the following mysql command to dump the contents of the users table on Mac OS X/Linux:

$ vagrant ssh -c 'mysql -hlocalhost -umyface_app -psupersecret -Dmyface -e "select id,user_name from users;"'
id                                  user_name
216e03c2-ffe4-11e2-b1ad-080027c8	jtimberman
216e0890-ffe4-11e2-b1ad-080027c8	someara
216e0bce-ffe4-11e2-b1ad-080027c8	jwinsor
216e0eda-ffe4-11e2-b1ad-080027c8	cjohnson
216e11e6-ffe4-11e2-b1ad-080027c8	mbower

And on Windows:

> vagrant ssh -c "mysql -hlocalhost -umyface_app -psupersecret -Dmyface -e 'select id,user_name from users;'" -- -n -T

The output should look similar to what you see above - the data from the INSERT INTO statemens in the SQL script.

Iteration #11 - Install PHP

Let’s add some PHP scripting sizzle to sell the steak of the database we just created. We’re going to install Apache 2 mod_php5 module and the php-mysql package to support our PHP script.

Edit recipes/webserver.rb and add the following:

...
include_recipe 'apache2'
include_recipe 'apache2::mod_php5'

package 'php-mysql' do
  action :install
  notifies :restart, 'service[apache2]'
end

This will use the apache2 cookbook’s mod_php5 module to install PHP5 and install the php-mysql support package. After editing, recipes/webserver.rb should look like this:

myface/recipes/webserver.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
#
# Cookbook Name:: myface
# Recipe:: webserver
#
# Copyright (C) 2013 YOUR_NAME
#
# All rights reserved - Do Not Redistribute
#

group node['myface']['group']

user node['myface']['user'] do
  group node['myface']['group']
  system true
  shell '/bin/bash'
end

include_recipe 'apache2'
include_recipe 'apache2::mod_php5'

package 'php-mysql' do
  action :install
  notifies :restart, 'service[apache2]'
end

# disable default site
apache_site '000-default' do
  enable false
end

# create apache config
template "#{node['apache']['dir']}/sites-available/#{node['myface']['config']}" do
  source 'apache2.conf.erb'
  notifies :restart, 'service[apache2]'
end

# create document root
directory "#{node['myface']['document_root']}" do
  action :create
  recursive true
end

# write site
cookbook_file "#{node['myface']['document_root']}/index.html" do
  mode "0644"
end

# enable myface
apache_site "#{node['myface']['config']}" do
  enable true
end

Run vagrant provision to converge your changes:

$ vagrant provision

Test Iteration #11

Run the following command to verify that the php5_module was successfully installed on Mac OS X/Linux:

$ vagrant ssh -c "sudo /usr/sbin/httpd -M | grep php5"
httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName
[Wed Aug 07 21:42:03 2013] [warn] NameVirtualHost *:80 has no VirtualHosts
Syntax OK
 php5_module (shared)

And on Windows:

> vagrant ssh -c "sudo /usr/sbin/httpd -M | grep php5" -- -n -T

Iteration #12 - Add PHP Sizzle

It’s the last iteration, get ready to see the PHP sizzle! First modify templates/default/apache2.conf.erb as follows:

myface/templates/default/apache2.conf.erb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# Managed by Chef for <%= node['hostname'] %>
<VirtualHost *:80>
        ServerAdmin <%= node['apache']['contact'] %>

        DocumentRoot <%= node['myface']['document_root'] %>
        <Directory />
                Options FollowSymLinks
                AllowOverride None
        </Directory>
        <Directory <%= node['myface']['document_root'] %>>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>

        ErrorLog <%= node['apache']['log_dir'] %>/error.log

        LogLevel warn

        CustomLog <%= node['apache']['log_dir'] %>/access.log combined
        ServerSignature Off
</VirtualHost>

Next delete files/default/index.html with the following command:

$ rm files/default/index.html

You’ll be replacing it with the following parametized PHP script as templates/default/index.php.erb:

myface/templates/default/index.php.erb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
<!--
Copyright 2013, Opscode, Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

-->

<?php
$db_host = 'localhost';
$db_user = 'root';
$db_pwd = '<%= node['mysql']['server_root_password'] %>';

$database = 'myface';
$table = 'users';

// UTILITY FUNCTIONS
function create_gravatar_hash($email) {
    return md5( strtolower( trim( $email ) ) );
}

function gravatar_img($email=null, $name=null, $size=null) {
    if(!$email) {
        return '';
    }

    $url =  'http://www.gravatar.com/avatar/';
    $url .= create_gravatar_hash($email);
    if($size) {
        $url .= "?s={$size}";
    }

    return sprintf('<img src="%s" alt="%s" />', $url, $name ? $name : '');
}

function neckbeard($rating) {
    $ratings = array(
        'Puberty awaits!',
        'Peach fuzz',
        'Solid week&#39;s growth',
        'Lumberjacks would be proud',
        'Makes dwarves weep',
    );

    return $ratings[(int) $rating - 1];
}

function fetch($host, $dbname, $dbuser, $dbpass, $table) {
    try {
        $connection = new PDO("mysql:host={$host};dbname={$dbname}", $dbuser, $dbpass);
        $connection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);

        $rc = $connection->query("SELECT COUNT(*) FROM {$table}");
        $count = $rc->fetchColumn();

        $result = $connection->query("SELECT * FROM {$table}");

        return array($result, $count);
    } catch(Exception $e) {
        die($e->getMessage());
    }
}

list($result, $fields_num) = fetch($db_host, $database, $db_user, $db_pwd, $table);

?>
<!DOCTYPE html>
<html lang="en">
<head>
    <title>MyFace Users</title>
    <style>
    * {
        -webkit-box-sizing: border-box;
        -moz-box-sizing: border-box;
        box-sizing: border-box;
    }
    
    html, body {
        margin: 0;
        padding: 0;
    }
    
    html {
        background: #999;
    }
    
    body {
        max-width: 480px;
        margin: 0 auto;
        font-family: Arial, Helvetica, sans-serif;
        color: #222;
        padding: 20px;
        border: 1px solid #666;
        -webkit-box-shadow: 0 0 5px rgba(0,0,0,0.3);
        -moz-box-shadow: 0 0 5px rgba(0,0,0,0.3);
        box-shadow: 0 0 5px rgba(0,0,0,0.3);
        background: #FFF;
    }
        
    a:link {
        text-decoration: none;
        color: #777;
    }
    
    a:hover,
    a:focus {
        text-decoration: underline;
    }
    
    h1 {
        text-align: center;
        margin-top: 0;
    }
        
    h1 span {
        color: #00C;
    }
    
    h2 {
        font-size: 24px;
        line-height: 1.0;
        margin: 0 0 10px;
    }
    
    p {
        font-size: 14px;
        line-height: 18px;
        margin: 10px 0;
    }
    
    p:last-child {
        margin-bottom: 0;
    }
    
    .email {
        display: block;
        overflow: hidden;
        text-overflow: ellipsis;
        white-space: nowrap;
    }
    
    /* Adapted from OOCSS
        * mod object (https://github.com/stubbornella/oocss/blob/master/core/module/mod.css)
        * media object (https://github.com/stubbornella/oocss/blob/master/core/media/media.css)
    */
    article {
        display: block;
        overflow: hidden;
        margin-bottom: 20px;
        border: 1px solid #CCC;
        background: #EEE;
        -webkit-border-radius: 4px;
        -moz-border-radius: 4px;
        border-radius: 4px;
        -webkit-box-shadow: 1px 1px 1px rgba(0,0,0,0.3) inset;
        -moz-box-shadow: 1px 1px 1px rgba(0,0,0,0.3) inset;
        box-shadow: 1px 1px 1px rgba(0,0,0,0.3) inset;
    }

    article .img {
        float: left;
        margin-right: 10px;
    }

    article .img img {
        display: block;
    }

    article .imgExt {
        float: right;
        margin-left: 10px;
    }

    article .bd {
        overflow: hidden;
        padding: 10px 0;
    }
    </style>
</head>
<body>
    <h1>Welcome to My<span>Face</span>!</h1>

    <?php while($row = $result->fetch(PDO::FETCH_OBJ)): ?>
    <article>
        <a href="<?php echo $row->url ?>" class="img" target="_blank">
            <?php echo gravatar_img($row->email, $row->user_name, 150) ?>
        </a>
        <div class="bd">
            <h2><?php echo $row->user_name ?></h2>
            <p><a href="<?php echo $row->url ?>" target="_blank" class="email"><?php echo $row->url ?></a></p>
            <p>Neckbeard rating: <?php echo neckbeard($row->neck_beard) ?></p>
        </div>
    </article>
    <?php endwhile; ?>
</body>
</html>
<?php mysql_free_result($result); ?>

Finally modify recipes/webserver.rb to use index.php.erb template to generate a new index.php document root:

myface/recipes/webserver.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
#
# Cookbook Name:: myface
# Recipe:: webserver
#
# Copyright (C) 2013 YOUR_NAME
#
# All rights reserved - Do Not Redistribute
#

group node['myface']['group']

user node['myface']['user'] do
  group node['myface']['group']
  system true
  shell '/bin/bash'
end

include_recipe 'apache2'
include_recipe 'apache2::mod_php5'

package 'php-mysql' do
  action :install
  notifies :restart, 'service[apache2]'
end

# disable default site
apache_site '000-default' do
  enable false
end

# create apache config
template "#{node['apache']['dir']}/sites-available/#{node['myface']['config']}" do
  source 'apache2.conf.erb'
  notifies :restart, 'service[apache2]'
end

# create document root
directory "#{node['myface']['document_root']}" do
  action :create
  mode '0755'
  recursive true
end

# write site
template "#{node['myface']['document_root']}/index.php" do
  source 'index.php.erb'
  mode '0644'
end

# enable myface
apache_site "#{node['myface']['config']}" do
  enable true
end

Since we changed the document root and our recipe contains no statements to remove the old index.html document root, we’ll need to destroy our vagrant test node and do a full vagrant up again, otherwise if we visit http://33.33.33.10 again, we’ll just see the old document root:

$ vagrant destroy -f
$ vagrant up

Testing Iteration #12

Visit http://33.33.33.10 Now you should see the lovely new PHP version of Myface.

myfacephp

More to Come!

In Part 3, we’ll introduce a new tool test-kitchen and show you how to automate all the tests you’ve been doing manually to test each iteration.

If you want to see the full source for MyFace, check out the following GitHub link: https://github.com/misheska/myface

Getting Started Writing Chef Cookbooks the Berkshelf Way, Part 1

Updated Jan 29th, 2014

  • Per Seth Vargo, Note about the future of vagrant-berkshelf

Updated December 27th, 2013

  • Being more prescriptive about the necessary Ruby 1.9.x environment
  • Bumped VirtualBox from version 4.3.4 to 4.3.6
  • Bumped vagrant-berkshelf plugin from version 1.3.6 to 1.3.7
  • Bumped vagrant-omnibus plugin from version 1.1.2 to 1.2.1
  • Added alternate command lines for Windows, as necessary
  • Debate on symbols vs strings is an unnecessary distraction, removed this section
  • Per Dan Patrick introducing the concept of cookbook_file before template, as this was confusing

Updated December 15th, 2013

  • Bumped CentOS basebox version from 6.4 to 6.5
  • Added note about issue with Vagrant 1.4.0
  • Bumped VirtualBox from version 4.2.18 to 4.3.4
  • Bumped Vagrant from version 1.3.1 to 1.3.5
  • Bumped vagrant-berkshelf plugin from version 1.3.3 to 1.3.6
  • Bumped vagrant-omnibus plugin from version 1.1.1 to 1.1.2

Updated September 9th, 2013

  • Bumped VirtualBox from version 4.2.16 to 4.2.18
  • Bumped berkshelf from version 2.0.9 to 2.0.10
  • Bumped vagrant from version 1.2.7 to 1.3.1
  • Bumped vagrant-omnibus plugin from version 1.1.0 to 1.1.1

NOTE: As of Tuesday, January 28th the Berkshelf core team announced the future deprecation and retirement of the vagrant-berkshelf plugin It is recommended that new users get started with the Berkshelf Way by using Test Kitchen and its .kitchen.yml covered in Part 3 of this series.


Jamie Winsor hasn’t yet updated his guide to authoring cookbooks the Berkshelf way to match recent changes related to Vagrant 1.x and Chef 11 This post is an attempt to update these instructions, walking through Jamie’s and Sean O’Meara’s example app - MyFace. For more information on Berkshelf, check out his recent talk and slides from ChefConf 2013.

NOTE: The source code examples covered in this article can be found on GitHub: https://github.com/misheska/myface

Getting Started

You can write Chef Cookbooks with Berkshelf on Mac OS X, Linux or Windows. To set up your cookbook-writing environment, make sure you have the following installed:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
$ gem install berkshelf --no-ri --no-rdoc
Fetching: berkshelf-2.0.10.gem (100%)
Successfully installed berkshelf-2.0.10
1 gem installed

$ berks -v
Berkshelf (2.0.10)

Copyright 2012-2013 Riot Games

    Jamie Winsor (<jamie@vialstudios.com>)
    Josiah Kiehl (<jkiehl@riotgames.com>)
    Michael Ivey (<michael.ivey@riotgames.com>)
    Justin Campbell (<justin.campbell@riotgames.com>)
    Seth Vargo (<sethvargo@gmail.com>)
  • Install the vagrant-berkshelf Plugin (1.3.3 or higher)
1
2
3
$ vagrant plugin install vagrant-berkshelf
Installing the 'vagrant-berkshelf' plugin. This can take a few minutes...
Installed the plugin 'vagrant-berkshelf (1.3.7)'!
  • Install the vagrant-omnibus plugin (1.1.0 or higher)
1
2
3
$ vagrant plugin install vagrant-omnibus
Installing the 'vagrant-omnibus' plugin.  This can take a few minutes...
Installed the plugin 'vagrant-omnibus (1.2.1)'!

Upgrade from Berkshelf 1.x

NOTE: If you had a previous 1.x version of the berkshelf plugin installed, when it was named berkshelf-vagrant, which you can verify by running the following command:

$ vagrant plugin list
berkshelf-vagrant (1.1.3)

Make sure you fully uninstall the old berkshelf-vagrant plugin before installing the new vagrant-berkshelf plugin, as vagrant will get confused by the name change:

$ vagrant plugin uninstall berkshelf-vagrant
Uninstalling the 'berkshelf-vagrant' plugin...
$ vagrant plugin install vagrant-berkshelf
Installing the 'vagrant-berkshelf' plugin.  This can take a few minutes...

Create the MyFace Application Cookbook

Key to the Berkshelf way is the use of the Application Cookbook Pattern. An application cookbook contains the list of recipes needed to build your application or service. As an example, this blog post will walk you through the creation of an example service - MyFace - the next killer social web app.

First create a new cookbook for the MyFace application using the berks cookbook myface command:

$ berks cookbook myface
      create  myface/files/default
      create  myface/templates/default
      create  myface/attributes
      create  myface/definitions
      create  myface/libraries
      create  myface/providers
      create  myface/recipes
      create  myface/resources
      create  myface/recipes/default.rb
      create  myface/metadata.rb
      create  myface/LICENSE
      create  myface/README.md
      create  myface/Berksfile
      create  myface/Thorfile
      create  myface/chefignore
      create  myface/.gitignore
         run  git init from "./myface"
      create  myface/Gemfile
      create  .kitchen.yml
      append  Thorfile
      create  test/integration/default
      append  .gitignore
      append  .gitignore
      append  Gemfile
      append  Gemfile
You must run `bundle install' to fetch any new gems.
      create  myface/Vagrantfile

Run bundle install in the newly created cookbook directory to install the necessary Gem dependencies:

$ cd myface
$ bundle install
Fetching gem metadata from https://rubygems.org/.......
Fetching additional metadata from https://rubygems.org/..
Resolving dependencies...
Using i18n (0.6.9)
Using multi_json (1.8.2)
Using activesupport (3.2.16)
. . .
Using berkshelf (2.0.10)
Using mixlib-shellout (1.3.0)
Using net-scp (1.1.2)
Using safe_yaml (0.9.7)
Using test-kitchen (1.1.1)
Using kitchen-vagrant (0.14.0)
Using bundler (1.5.0)
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.

Prepare a virtual machine for testing

It’s a good idea to develop your cookbook incrementally, testing in short iterations. Berkshelf integrates with Vagrant to deploy your cookbook changes to a virtual machine for testing.

Ensure that the vagrant-omnibus plugin is installed correctly.

$ vagrant plugin list
...
vagrant-omnibus (1.2.1)
...

The vagrant-omnibus plugin hooks into Vagrant and allows you to specify the version of the Chef Omnibus package you want installed using the omnibus.chef_version key

Edit the Vagrantfile generated by the berks cookbook command to use a VirtualBox template that does not have a version of Chef provisioned. Then, specify that you want your image to always use the latest version of Chef via the config.omnibus.chef_version config option and replace the legacy config.ssh.max_tries and config.ssh.timeout settings with config.vm.boot_timeout.

myface/Vagrantfile
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Vagrant.configure("2") do |config|
  config.vm.hostname = "myface-berkshelf"
  config.vm.box = "centos65"
  config.vm.box_url = "https://s3-us-west-2.amazonaws.com/misheska/vagrant/virtualbox4.3.6/centos65.box"
  config.omnibus.chef_version = :latest
  config.vm.network :private_network, ip: "33.33.33.10"
  config.vm.boot_timeout = 120
  config.berkshelf.enabled = true

  config.vm.provision :chef_solo do |chef|
    chef.json = {
      :mysql => {
        :server_root_password => 'rootpass',
        :server_debian_password => 'debpass',
        :server_repl_password => 'replpass'
      }
    }
    chef.run_list = [
      "recipe[myface::default]"
    ]
  end
end

NOTE: Vagrant 1.3.0 deprecated the config.ssh.max_tries and config.ssh.timeout settings that are inserted in the Vagrantfile by Berkshelf. Until Berkshelf is updated for these changes to vagrant, you’ll need to remove these settings from the Vagrantfile and replace them with config.vm.boot_timeout as above. If you are using vagrant 1.2.x, keep the config.ssh.max_tries and config.ssh.timeout settings, as the new config.vm.boot_timeout setting is not valid in the older version of vagrant.

Run vagrant up to start up the virtual machine and to test the stub MyFace cookbook you just created:

$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
[default] Box 'centos65' was not found. Fetching box from specified URL for
the provider 'virtualbox'. Note that if the URL does not have
a box for this provider, you should interrupt Vagrant now and add
the box yourself. Otherwise Vagrant will attempt to download the
full box prior to discovering this error.
Downloading box from URL: https://s3-us-west-2.amazonaws.com/misheska/vagrant/virtualbox4.3.6/centos65.box
Extracting box...te: 6881k/s, Estimated time remaining: 0:00:01)
Successfully added box 'centos65' with provider 'virtualbox'!
[default] Importing base box 'centos65'...
[default] Matching MAC address for NAT networking...
[default] Setting the name of the VM...
[default] Clearing any previously set forwarded ports...
[Berkshelf] This version of the Berkshelf plugin has not been fully tested on this version of Vagrant.
[Berkshelf] You should check for a newer version of vagrant-berkshelf.
[Berkshelf] If you encounter any errors with this version, please report them at https://github.com/RiotGames/vagrant-berkshelf/issues
[Berkshelf] You can also join the discussion in #berkshelf on Freenode.
[Berkshelf] Updating Vagrant's berkshelf: '/Users/misheska/.berkshelf/default/vagrant/berkshelf-20131228-44316-176rdqc-default'
[Berkshelf] Using myface (0.1.0)
[default] Clearing any previously set network interfaces...
[default] Preparing network interfaces based on configuration...
[default] Forwarding ports...
[default] -- 22 => 2222 (adapter 1)
[default] Booting VM...
[default] Waiting for machine to boot. This may take a few minutes...
[default] Machine booted and ready!
[default] Setting hostname...
[default] Configuring and enabling network interfaces...
[default] Mounting shared folders...
[default] -- /vagrant
[default] -- /tmp/vagrant-chef-1/chef-solo-1/cookbooks
[default] Installing Chef 11.8.2 Omnibus package...
[default] Downloading Chef 11.8.2 for el...
[default] downloading https://www.opscode.com/chef/metadata?v=11.8.2&prerelease=false&p=el&pv=6&m=x86_64
  to file /tmp/install.sh.2993/metadata.txt
trying wget...
[default] url	https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-11.8.2-1.el6.x86_64.rpm
md5	10f3d0da82efa973fe91cc24a6a74549
sha256	044558f38d25bbf75dbd5790ccce892a38e5e9f2a091ed55367ab914fbd1cfed
[default] downloaded metadata file looks valid...
[default] downloading https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-11.8.2-1.el6.x86_64.rpm
  to file /tmp/install.sh.2993/chef-11.8.2.x86_64.rpm
trying wget...
[default] Checksum compare with sha256sum succeeded.
[default] Installing Chef 11.8.2
installing with rpm...
[default] warning:
[default] /tmp/install.sh.2993/chef-11.8.2.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 83ef826a: NOKEY
[default] Preparing...
[default] ##################################################
[default]
[default] chef
[default] #
[default]
[default] Thank you for installing Chef!
[default] Running provisioner: chef_solo...
Generating chef JSON and uploading...
Running chef-solo...
[2013-12-28T13:42:49-08:00] INFO: Forking chef instance to converge...
[2013-12-28T13:42:49-08:00] INFO: *** Chef 11.8.2 ***
[2013-12-28T13:42:49-08:00] INFO: Chef-client pid: 3368
[2013-12-28T13:42:50-08:00] INFO: Setting the run_list to ["recipe[myface::default]"] from JSON
[2013-12-28T13:42:50-08:00] INFO: Run List is [recipe[myface::default]]
[2013-12-28T13:42:50-08:00] INFO: Run List expands to [myface::default]
[2013-12-28T13:42:50-08:00] INFO: Starting Chef Run for myface-berkshelf
[2013-12-28T13:42:50-08:00] INFO: Running start handlers
[2013-12-28T13:42:50-08:00] INFO: Start handlers complete.
[2013-12-28T13:42:50-08:00] INFO: Chef Run complete in 0.023677599 seconds
[2013-12-28T13:42:50-08:00] INFO: Running report handlers
[2013-12-28T13:42:50-08:00] INFO: Report handlers complete
[2013-12-28T13:42:49-08:00] INFO: Forking chef instance to converge...

If all goes well, you should see Chef Run complete with no errors.

NOTE: The basebox URL comes from my current collection of baseboxes. The following link points to a README file which provides links to all the vagrant baseboxes I use (which I normally update frequently): https://github.com/misheska/basebox-packer

If you would ever like to delete your test virtual machine and start over, you can destroy your test virtual machine with the vagrant destroy command:

$ vagrant destroy
Are you sure you want to destroy the 'default' VM? [y/N] y
[default] Forcing shutdown of VM...
[default] Destroying VM and associated drives...
[Berkshelf] Cleaning Vagrant's berkshelf
[default] Running cleanup tasks for 'chef_solo' provisioner...

Run vagrant up to recreate the test virtual machine.

NOTE: If you just ran vagrant destroy make sure you run vagrant up before proceeding to the next section.

Iteration #1: Create an application user

For our first short iteration, let’s create a myface user under which we’ll run our application. One best practice is to avoid running applications as root and create a user/group under which the application runs instead who has just enough rights that the app needs.

Edit myface/recipes/default.rb defining a new Group Resource and User Resource for myface, so it looks like the following:

myface/recipes/default.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
#
# Cookbook Name:: myface
# Recipe:: default
#
# Copyright (C) 2013 YOUR_NAME
#
# All rights reserved - Do Not Redistribute
#

group 'myface'

user 'myface' do
  group 'myface'
  system true
  shell '/bin/bash'
end

Save recipes/default.rb and re-run vagrant provision to create the myface user on your test virtual machine:

$ vagrant provision
[Berkshelf] This version of the Berkshelf plugin has not been fully tested on this version of Vagrant.
[Berkshelf] You should check for a newer version of vagrant-berkshelf.
[Berkshelf] If you encounter any errors with this version, please report them at https://github.com/RiotGames/vagrant-berkshelf/issues
[Berkshelf] You can also join the discussion in #berkshelf on Freenode.
[Berkshelf] Updating Vagrant's berkshelf: '/Users/misheska/.berkshelf/default/vagrant/berkshelf-20131228-44581-4bhc9d-default'
[Berkshelf] Using myface (0.1.0)
[default] Chef 11.8.2 Omnibus package is already installed.
[default] Running provisioner: chef_solo...
Generating chef JSON and uploading...
Running chef-solo...
[2013-12-28T13:57:59-08:00] INFO: Forking chef instance to converge...
[2013-12-28T13:57:59-08:00] INFO: *** Chef 11.8.2 ***
[2013-12-28T13:57:59-08:00] INFO: Chef-client pid: 3845
[2013-12-28T13:57:59-08:00] INFO: Setting the run_list to ["recipe[myface::default]"] from JSON
[2013-12-28T13:57:59-08:00] INFO: Run List is [recipe[myface::default]]
[2013-12-28T13:57:59-08:00] INFO: Run List expands to [myface::default]
[2013-12-28T13:57:59-08:00] INFO: Starting Chef Run for myface-berkshelf
[2013-12-28T13:57:59-08:00] INFO: Running start handlers
[2013-12-28T13:57:59-08:00] INFO: Start handlers complete.
[2013-12-28T13:57:59-08:00] INFO: group[myface] created
[2013-12-28T13:57:59-08:00] INFO: user[myface] created
[2013-12-28T13:57:59-08:00] INFO: Chef Run complete in 0.157055758 seconds
[2013-12-28T13:57:59-08:00] INFO: Running report handlers
[2013-12-28T13:57:59-08:00] INFO: Report handlers complete
[2013-12-28T13:57:59-08:00] INFO: Forking chef instance to converge...

You should expect to see the Chef run complete with no errors. Notice that it also creates group[myface] and user[myface].

Testing Iteration #1

Verify that Chef actually created the myface user on our test virtual machine by running the following on Mac OS X/Linux:

$ vagrant ssh -c "getent passwd myface"
myface:x:497:501::/home/myface:/bin/bash

On Windows, add two extra parameters to the ssh invocation to squelch some error messages:

> vagrant ssh -c "getent passwd myface" -- -n -T
myface:x:497:501::/home/myface:/bin/bash

The extra -n and -T parameters for Windows are additional ssh parameters to prevent trying to read from stdin and to suppress an error message that a pseudo terminal can’t be allocated, respectively. On Windows, vagrant runs as a service, and windows services do not have console sessions attached which ssh assumes, so you’ll see some errors on Windows if you don’t use these extra parameters.

We use vagrant ssh -c to run a command on our test virtual machine. The getent command can be used to query all user databases. In this case we’re looking for myface, and it exists!

Because we are using well-defined resources that are completely idempotent, you should notice that if you run vagrant provision again, the Chef run executes more quickly and it does not try to re-create the user/group it already created.

$ vagrant provision
[Berkshelf] This version of the Berkshelf plugin has not been fully tested on this version of Vagrant.
[Berkshelf] You should check for a newer version of vagrant-berkshelf.
[Berkshelf] If you encounter any errors with this version, please report them at https://github.com/RiotGames/vagrant-berkshelf/issues
[Berkshelf] You can also join the discussion in #berkshelf on Freenode.
[Berkshelf] Updating Vagrant's berkshelf: '/Users/misheska/.berkshelf/default/vagrant/berkshelf-20131228-44581-4bhc9d-default'
[Berkshelf] Using myface (0.1.0)
[default] Chef 11.8.2 Omnibus package is already installed.
[default] Running provisioner: chef_solo...
Generating chef JSON and uploading...
Running chef-solo...
[2013-12-28T14:01:18-08:00] INFO: Forking chef instance to converge...
[2013-12-28T14:01:18-08:00] INFO: *** Chef 11.8.2 ***
[2013-12-28T14:01:18-08:00] INFO: Chef-client pid: 4378
[2013-12-28T14:01:18-08:00] INFO: Setting the run_list to ["recipe[myface::default]"] from JSON
[2013-12-28T14:01:18-08:00] INFO: Run List is [recipe[myface::default]]
[2013-12-28T14:01:18-08:00] INFO: Run List expands to [myface::default]
[2013-12-28T14:01:18-08:00] INFO: Starting Chef Run for myface-berkshelf
[2013-12-28T14:01:18-08:00] INFO: Running start handlers
[2013-12-28T14:01:18-08:00] INFO: Start handlers complete.
[2013-12-28T14:01:18-08:00] INFO: Chef Run complete in 0.023349135 seconds
[2013-12-28T14:01:18-08:00] INFO: Running report handlers
[2013-12-28T14:01:18-08:00] INFO: Report handlers complete
[2013-12-28T14:01:18-08:00] INFO: Forking chef instance to converge...

Iteration #2 - Refactor to attributes

What if at some point you wanted to change the name of the myface user/group you just created to something else? At the moment, you would need to edit myface/recipes/default.rb in three places.

Let’s create a new file called myface/attributes/default.rb which initializes Chef attributes defining the user name and group name under which our application will run so that you don’t repeat yourself.

myface/attributes/default.rb
1
2
default['myface']['user'] = 'myface'
default['myface']['group'] = 'myface'

In Chef, attributes are a hash of a hash used to override the default settings on a node. The first hash is the cookbook name - in our case we’ve named our cookbook 'myface'. The second hash is the name of our attribute - in this case, we’re defining two new attributes: 'user' and 'group'.

default implies the use of the node object node.default and is a Chef attribute file shorthand. The following are equivalent definitions to the ones above:

node.default['myface']['user'] = 'myface'
node.default['myface']['group'] = 'myface'

Note that the hash names are defined as strings enclosed by quotes. In this case, it doesn’t matter if you use single or double quotes. In Chef source files, double quotes indicate that string interpolation should be performed, replacing special tokens in a string with their values. If single quotes are used, no string interpolation is performed. We’ll see more examples of when string interpolation is necessary later in this artile.

Now that you’ve created your attribute definitions, edit myface/recipes/default.rb and replace all references to the ‘myface’ user name with node['myface']['user'] and all references to the ‘myface’ group with node['myface']['group']. myface/recipes/default.rb should now look like this:

myface/recipes/default.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
#
# Cookbook Name:: myface
# Recipe:: default
#
# Copyright (C) 2013 YOUR_NAME
#
# All rights reserved - Do Not Redistribute
#

group node['myface']['group']

user node['myface']['user'] do
  group node['myface']['group']
  system true
  shell '/bin/bash'
end

Re-provision with vagrant provision to see how the refactor went:

$ vagrant provision

As long as you didn’t create any syntax errors in your refactoring file edits, there should be no net change on the virtual machine test node (as you’ve only just moved some strings into a node attribute).

Testing Iteration #2

Running getent on the test virtual machine should also produce the same result as when you validated Iteration #1 on Mac OS X/Linux:

$ vagrant ssh -c "getent passwd myface"
myface:x:497:501::/home/myface:/bin/bash

And on Windows:

> vagrant ssh -c "getent passwd myface" -- -n -T
myface:x:497:501::/home/myface:/bin/bash

Iteration #3 - Install the Apache2 Web Server

Our hot new social networking application, myface, is a web app, so we need to install a web server. Let’s install the Apache2 web server.

Modify myface/recipes/default.rb to include the apache2 cookbook’s default recipe:

include_recipe 'apache2'

The resultant myface/recipes/default.rb file should look like so:

myface/recipes/default.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
#
# Cookbook Name:: myface
# Recipe:: default
#
# Copyright (C) 2013 YOUR_NAME
#
# All rights reserved - Do Not Redistribute
#

group node['myface']['group']

user node['myface']['user'] do
  group node['myface']['group']
  system true
  shell '/bin/bash'
end

include_recipe 'apache2'

Since you are loading Apache2 from another cookbook, you need to configure the dependency in your metadata. Edit myface/metadata.rb and add the apache2 dependency at the bottom:

depends 'apache2', '~> 1.8.0'

This tells Chef that the myface cookbook depends on the apache2 cookbook. We’ve also specified a version constraint using the optimistic operator ~> to tell our Chef that we want the latest version of the apache2 cookbook that is greater than 1.8.0 but not 1.9.0 or higher.

It is recommended that Chef cookbooks follow a Semantic Versioning scheme. So if you write your cookbook to use the latest apache2 1.8.x cookbook, if the apache2 cookbook is ever bumped to a 1.9.x version (or 2.x), it has some public API functionality that has at least been deprecated. So you’ll want to review the changes and test before automatically using an apache2 cookbook version 1.9.x or higher. However, automatic use of any new 1.8.x is perfectly fine, because no only backwards-compatible bug fixes has been introduced. Semantic Versioning guarantees there are no changes in the public APIs.

Your myface/metadata.rb file should look like this:

myface/metadata.rb
1
2
3
4
5
6
7
8
9
name             'myface'
maintainer       'YOUR_NAME'
maintainer_email 'YOUR_EMAIL'
license          'All rights reserved'
description      'Installs/Configures myface'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version          '0.1.0'

depends 'apache2', '~> 1.8.0'

Now when you re-run vagrant provision it will install apache2 on your test virtual machine:

$ vagrant provision
[Berkshelf] This version of the Berkshelf plugin has not been fully tested on this version of Vagrant.
[Berkshelf] You should check for a newer version of vagrant-berkshelf.
[Berkshelf] If you encounter any errors with this version, please report them at https://github.com/RiotGames/vagrant-berkshelf/issues
[Berkshelf] You can also join the discussion in #berkshelf on Freenode.
[Berkshelf] Updating Vagrant's berkshelf: '/Users/misheska/.berkshelf/default/vagrant/berkshelf-20131228-44581-4bhc9d-default'
[Berkshelf] Using myface (0.1.0)
[Berkshelf] Installing apache2 (1.8.14) from site: 'http://cookbooks.opscode.com/api/v1/cookbooks'
[default] Chef 11.8.2 Omnibus package is already installed.
[default] Running provisioner: chef_solo...
Generating chef JSON and uploading...
Running chef-solo...
...
[2013-12-28T14:10:49-08:00] INFO: service[apache2] started
[2013-12-28T14:10:49-08:00] INFO: template[/etc/httpd/mods-available/deflate.conf] sending restart action to service[apache2] (delayed)
[2013-12-28T14:10:51-08:00] INFO: service[apache2] restarted
[2013-12-28T14:10:51-08:00] INFO: Chef Run complete in 38.225601754 seconds
[2013-12-28T14:10:51-08:00] INFO: Running report handlers
[2013-12-28T14:10:51-08:00] INFO: Report handlers complete
[2013-12-28T14:10:12-08:00] INFO: Forking chef instance to converge...

Testing Iteration #3

You can verify that the apache2 httpd service is running on your berkshelf virtual machine with the following command on Mac OS X/Linux:

$ vagrant ssh -c "sudo /sbin/service httpd status"
httpd (pid  5758) is running.

And on Windows:

> vagrant ssh -c "sudo /sbin/service httpd status" -- -n -T
httpd (pid  5758) is running.

Since this is a web server, so you can also check it out in your favorite web browser. The host-only private network address for the virtual machine that Berkshelf created is in the Vagrantfile. Display the IP address with the following command:

$ grep ip: Vagrantfile
config.vm.network :private_network, ip: "33.33.33.10"

Check it out with your favorite web browser:

http://33.33.33.10

While you will get a 404 Not Found error because we haven’t added any content to our web site yet, the important part is that Apache Server at 33.33.33.10 Port 80 sent the response:

Apache Server Response

Wait a second, though. You never downloaded the apache2 cookbook! That’s the magic of the Berkshelf Vagrant plugin you installed earlier. The Berkshelf Vagrant plugin will make sure that any changes you make to your cookbook and all of your cookbook’s dependencies are made available to your virtual machine. Berkshelf automatically loads all your cookbook dependencies much like Bundler automatically loads all your gem dependencies.

Where does the Berkshelf put the cookbooks it downloads? You can find them in ~/.berkshelf/default/cookbooks (or %USERPROFILE%\.berkshelf\default\cookbooks on Windows)

Users/misheska/.berkshelf/default/cookbooks
└── apache2-1.8.14
    ├── attributes
    ├── definitions
    ├── files
    │   └── default
    │       └── tests
    │           └── minitest
    │               └── support
    ├── recipes
    └── templates
        └── default
            └── mods

~/.berkshelf (or %USERPROFILE%\.berkshelf on Windows) is just the default location where Berkshelf stores data on your local disk. This location can be altered by setting the environment variable BERKSHELF_PATH.

Iteration #4 - Add Content

Let’s add some content to make that 404 go away. Edit myface/recipes/default.rb as follows:

myface/recipes/default.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
#
# Cookbook Name:: myface
# Recipe:: default
#
# Copyright (C) 2013 YOUR_NAME
#
# All rights reserved - Do Not Redistribute
#

group node['myface']['group']

user node['myface']['user'] do
  group node['myface']['group']
  system true
  shell '/bin/bash'
end

include_recipe 'apache2'

# disable default site
apache_site '000-default' do
  enable false
end

# create apache config
template "#{node['apache']['dir']}/sites-available/myface.conf" do
  source 'apache2.conf.erb'
  notifies :restart, 'service[apache2]'
end

# create document root
directory '/srv/apache/myface' do
  action :create
  recursive true
end

# write site
cookbook_file '/srv/apache/myface/index.html' do
  mode '0644' # forget me to create debugging exercise
end

# enable myface
apache_site 'myface.conf' do
  enable true
end

If you’re familiar with Chef and configuring a web app via apache2, nothing here should be too surprising. But if not, spend some time reading up on the resource references at http://docs.opscode.com

Note our first use of string interpolation on line 26:

template "#{node['apache']['dir']}/sites-available/myface.conf" do

We need to enclose the template string in double-quotes because we’re using the Ruby #{} operator to indicate that we want to perform string interpolation and evaluate the value of node['apache']['dir'] inserting the evaluated value in the string.

Create a file to contain our web site content as myface/files/default/index.html.
Chef looks for files in the files subtree by default. A subdirectory level underneath files is used to specify platform-specific files. Files in files/default are used with every platform. During the Chef run this file is copied to the target node in the location specified in the cookbook_file resource (see line 38 of myface/recipes/default.rb).

myface/files/default/index.html
1
Welcome to MyFace!

With Chef, you can also create config files from templates using ERB, a Ruby templating system. When the template .erb file is processed by Chef, it will replace any token bracketed by a <%= %> tag with the value of the Ruby expression inside the token (like node[:hostname]). (Unlike with a cookbook_file whose content is static and is never manipulated by Chef).

Chef expects ERB template files to be in the templates subirectory of a cookbook. A subdirectory level underneath templates is used to specify platorm-specific files. Files in the templates/default subdirectory are used with every platform.

Create a new template file called myface/templates/default/apache2.conf.erb which will become the file .../sites-available/myface.conf on our test virtual machine (refer to myface/recipes/default.rb above):

myface/templates/default/apache2.conf.erb
1
2
3
4
5
6
7
8
# Managed by Chef for <%= node['hostname'] %>

Alias / /srv/apache/myface/

<Directory /srv/apache/myface>
	Options FollowSymLinks +Indexes
	Allow from All
</Directory>

After you have created these three files, run vagrant provision to deploy your changes:

$ vagrant provision

Testing Iteration #4

If the Chef run completed successfully, if you point your web browser at your myface web site again:

http://33.33.33.10

You’ll see some lovely content!

Welcome to MyFace

Also, note the difference between a cookbook_file and a template. Run this command on Mac OS X/Linux to print out index.html that was copied to the node during the Chef run.

$ vagrant ssh -c "cat /srv/apache/myface/index.html"
Welcome to MyFace!
Connection to 127.0.0.1 closed.

And on Windows:

> vagrant ssh -c "cat /srv/apache/myface/index.html" -- -n -T
Welcome to MyFace!
Connection to 127.0.0.1 closed.

Note that index.html is copied verbatim with no changes to the file.

By comparison, in a template any ERB tokens are replaced when they are copied to the node. Look at the resultant myface.conf on the node with the following command on Mac OS X/Linux:

$ vagrant ssh -c "cat /etc/httpd/sites-available/myface.conf"
# Managed by Chef for myface-berkshelf

Alias / /srv/apache/myface/

<Directory /srv/apache/myface>
 	Options FollowSymLinks +Indexes
 	Allow from All
</Directory>
Connection to 127.0.0.1 closed.

and with the following command on Windows:

> vagrant ssh -c "cat /etc/httpd/sites-available/myface.conf" -- -n -T

The ERB token <%= node['hostname'] %> was replaced by the evaluated string myface-berkshelf during the Chef run.

Iteration #5 - Refactoring webserver

myface/recipes/default.rb is getting rather large and we’ve got a lot more to add to our cookbook. Let’s go through another refactoring pass.

Let’s move all the webserver-related resources to their own file myface/recipes/webserver.rb. Rename myface/recipes/default.rb to myface/recipes/webserver.rb.

Now myface/recipes/webserver.rb should look like this:

myface/recipes/webserver.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
#
# Cookbook Name:: myface
# Recipe:: webserver
#
# Copyright (C) 2013 YOUR_NAME
#
# All rights reserved - Do Not Redistribute
#

group node['myface']['group']

user node['myface']['user'] do
  group node['myface']['group']
  system true
  shell '/bin/bash'
end

include_recipe 'apache2'

# disable default site
apache_site '000-default' do
  enable false
end

# create apache config
template "#{node['apache']['dir']}/sites-available/myface.conf" do
  source 'apache2.conf.erb'
  notifies :restart, 'service[apache2]'
end

# create document root
directory '/srv/apache/myface' do
  action :create
  recursive true
end

# write site
cookbook_file '/srv/apache/myface/index.html' do
  mode '0644'
end

# enable myface
apache_site 'myface.conf' do
  enable true
end

Create a new myface/recipes/default.rb file which references webserver.rb.

myface/recipes/default.rb
1
2
3
4
5
6
7
8
9
# Cookbook Name:: myface
# Recipe:: default
#
# Copyright (C) 2013 YOUR_NAME
#
# All rights reserved - Do Not Redistribute
#

include_recipe 'myface::webserver'

Converge your node again to make sure there are no syntax errors:

$ vagrant provision

Let’s eliminate some more of the duplication that crept in while we were working on things. Edit myface/attributes/default.rb

myface/attributes/default.rb
1
2
3
4
5
default['myface']['user'] = 'myface'
default['myface']['group'] = 'myface'
default['myface']['name'] = 'myface'
default['myface']['config'] = 'myface.conf'
default['myface']['document_root'] = '/srv/apache/myface'

NOTE: With Chef 11, it is now possible to nest attributes, like so:

node.default['app']['name'] = 'my_app'
node.default['app']['document_root'] = "/srv/apache/#{node['app']['name']}"

This approach is overkill for MyFace (and is frankly overkill for most Chef recipes). Even though nesting is an option now with Chef 11, you should try to keep your attribute files as simple and straightforward to follow as possible.

In myface/recipes/webserver.rb replace the corresponding hardcoded references to attribute references:

myface/recipes/webserver.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
#
# Cookbook Name:: myface
# Recipe:: webserver
#
# Copyright (C) 2013 YOUR_NAME
#
# All rights reserved - Do Not Redistribute
#

group node['myface']['group']

user node['myface']['user'] do
  group node['myface']['group']
  system true
  shell '/bin/bash'
end

include_recipe 'apache2'

# disable default site
apache_site '000-default' do
  enable false
end

# create apache config
template "#{node['apache']['dir']}/sites-available/#{node['myface']['config']}" do
  source 'apache2.conf.erb'
  notifies :restart, 'service[apache2]'
end

# create document root
directory "#{node['myface']['document_root']}" do
  action :create
  recursive true
end

# write site
cookbook_file "#{node['myface']['document_root']}/index.html" do
  mode '0644'
end

# enable myface
apache_site "#{node['myface']['config']}" do
  enable true
end

Converge your node to make sure there are no syntax errors:

$ vagrant provision

Add tokens to the templates/default/apache2.conf.erb ERB template file so they will be replaced with the value of the node['myface']['document_root'] attribute during the Chef run.

myface/templates/default/apache2.conf.erb
1
2
3
4
5
6
7
8
# Managed by Chef for <%= node['hostname'] %>

Alias / <%= node['myface']['document_root'] %>/

<Directory <%= node['myface']['document_root'] %>>
    Options FollowSymLinks +Indexes
    Allow from All
</Directory>

Converge your node one last time to make sure there are no syntax errors:

$ vagrant provision

Testing Iteration #5

Visiting http://33.33.33.10 should produce the same result as before as you have made no net changes, just shuffled things around a bit.

Iteration #6 - Version Bump and Production Deploy

Now that we have tested our cookbook locally and everything seems to work, we’re ready to finalize the cookbook and deploy it to production.

Version Bump to 1.0.0

First you need to “bump” the cookbook version in the metadata.rb file to its final version 1.0.0. As mentioned in Iteration #3, cookbooks (even the ones you write), should follow the Semantic Versioning scheme. Since this is the first public version of our cookbook, it’s version 1.0.0.

myface/metadata.rb should look like this:

myface/metadata.rb
1
2
3
4
5
6
7
8
9
name             'myface'
maintainer       'Mischa Taylor'
maintainer_email 'mischa@misheska.com'
license          'Apache 2.0'
description      'Installs/Configures myface'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version          '1.0.0'

depends 'apache2', '~> 1.8.0'

Configure Berkshelf

In order to deploy to your production server (instead of just locally with vagrant), you’ll need to add a section to your Berkshelf config file with your production Chef settings. Run the following command to create a default Berkshelf configuration file:

$ berks configure

It will prompt you for a bunch of server settings. Since I’m using hosted Chef to manage my servers, my settings are as follows (yours will be slightly different):

$ berks configure
Enter value for chef.chef_server_url (default: 'http://localhost:4000'):  https://api.opscode.com/organizations/misheska
Enter value for chef.node_name (default: 'Ruby.local'):  misheska
Enter value for chef.client_key (default: '/etc/chef/client.pem'):  /Users/misheska/.chef/misheska.pem
Enter value for chef.validation_client_name (default: 'chef-validator'):  misheska-validator
Enter value for chef.validation_key_path (default: '/etc/chef/validation.pem'):  /Users/misheska/.chef/misheska-validator.pem
Enter value for vagrant.vm.box (default: 'Berkshelf-CentOS-6.3-x86_64-minimal'):  centos65
Enter value for vagrant.vm.box_url (default: 'https://dl.dropbox.com/u/31081437/Berkshelf-CentOS-6.3-x86_64-minimal.box'):  https://s3-us-west-2.amazonaws.com/misheska/vagrant/virtualbox4.3.6/centos65.box
Config written to: '/Users/misheska/.berkshelf/config.json'

The config file is located in $HOME/.berkshelf/config.json (or inx %USERPROFILE%/.berkshelf/config.json on Windows).

Here’s what my $HOME/.berkshelf/config.json file looks like:

{
  "chef":{
    "chef_server_url":"https://api.opscode.com/organizations/misheska",
    "validation_client_name":"misheska-validator",
    "validation_key_path":"/Users/misheska/.chef/misheska-validator.pem",
    "client_key":"/Users/misheska/.chef/misheska.pem",
    "node_name":"misheska"
  },
  "cookbook":{
    "copyright":"Mischa Taylor",
    "email":"mischa@misheska.com",
    "license":"Apache 2.0"
  },
  "allowed_licenses":[],
  "raise_license_exception":false,
  "vagrant":{
    "vm":{
      "box":"centos65",
      "box_url":"https://s3-us-west-2.amazonaws.com/misheska/vagrant/virtualbox4.3.6/centos65.box",
      "forward_port":{},
      "network":{"bridged":false,"hostonly":"33.33.33.10"},
      "provision":"chef_solo"}
  },
  "ssl":{
    "verify":false
  }
}

{
  "chef":{
    "chef_server_url": "https://api.opscode.com/organizations/misheska",
    "validation_client_name": "misheska-validator",
    "validation_key_path": "/Users/mischa/.chef/misheska-validator.pem",
    "client_key": "/Users/mischa/.chef/misheska.pem",
    "node_name":"misheska"
  },
  "vagrant":{
    "vm":{
      "box": "centos65",
      "box_url":"https://s3-us-west-2.amazonaws.com/misheska/vagrant/virtualbox4.3.6/centos65.box",
      "forward_port": {
      },
      "network":{
        "bridged": false,
        "hostonly": "33.33.33.10"
      },
      "provision": "chef_solo"
    }
  },
  "ssl": {
    "verify":false
  }
}

I assume you have your own production Chef setup either running Hosted Chef, Private Chef, or Open Source Chef Server. If you need more help getting your .pem file values, refer to the QuickStart Guide on LearnChef.

Upload cookbooks

Edit your $HOME/.berkshelf/config.json file similarly with your .pem file values. Then run berks upload to upload your cookbooks to the chef server:

$ berks upload
Using myface (1.0.0)
Using apache2 (1.8.14)
Uploading myface (1.0.0) to: 'https://api.opscode.com:443/organizations/misheska'
Uploading apache2 (1.8.14) to: 'https://api.opscode.com:443/organizations/misheska'

Similarly to when you ran vagrant up, Berkshelf automatically uploaded all the cookbook dependencies.

Converge node

Add the default myface cookbook recipe to your node’s run list. For example, I used the following command to add myface to mine:

$ knife node run_list add mischa-ubuntu 'recipe[myface]'
mischa-ubuntu:
  run_list: recipe[myface]

Converge the node by running chef-client (if you don’t already have it setup to run chef-client automatically). For example, here’s the command I used to run chef-client on my production node:

$ knife ssh name:mischa-ubuntu "sudo chef-client" -x misheska
Starting Chef Client, version 11.8.2
resolving cookbooks for run list: ["myface"]
Synchronizing Cookbooks:
  - myface
  - apache2
...
Chef Client finished, 20 resources updated 

Testing Iteration #6

Browsing your production node’s URL should produce the same result as when you tested Iteration #4. For example, I visited http://mischa-ubuntu

More to Come!

This is just part one of a multi-part series. So far you’ve gone through several short iteration loops as you evolve the myface cookbook web application. In Part 2, we’ll wire up a database to the myface application and explore the use of the mysql and database cookbooks.

If you want to see the full source for myface, check out the following GitHub link: https://github.com/misheska/myface