Skip to content

John Bennett's blog

Getting started with Opscode Chef (on Windows), part 1

Wednesday, December 12, 2012

Warning: I’m a newbie when it comes to both Chef and Ruby.  Anything I say here may be completely wrong or counter to standard chef idioms.  I’m describing my beginner’s understanding, and what I’ve found useful so far.  YMMV.

I’m working on a project that is using Chef — a tool for automating server configuration.  Chef is an awesome way to automate creation and maintenance of machines and environments.

One downside of Chef (for me) is that Windows tends to be an afterthought, especially in the documentation. Windows examples are few and far between.  The project I’m working on is Windows-based, so it’s been a bit of a slog to figure out what I’m doing.  That said, it is definitely worth learning about Chef for configuring and deploying to Windows machines.

This is the first of n posts about my experiences learning Chef and using it on Windows.

An aside: Try searching for “chef tutorial”, “chef cookbooks and recipes”, “chef knife help”, etc. and you won’t find anything remotely related to the chef we’re talking about.  (I’m excited to see all the new traffic from foodies on my blog.)  I’ve taken to adding “opscode” to every search I do, but half the results are still related to cooking.  Metaphors are great, but when naming a product or project, please make it easily bingoogleable!

A quick overview of Chef

The philosophy of Chef — which I whole-heartedly agree with — is that server and environment configuration should be treated like source code and managed in source control.  You want it versioned, you want to be able to rollback to previous configurations, you want to branch and do experiments, etc.  Just like code.  For that reason, chef is primarily file-driven.  Specifically, Ruby and json files.  You don’t have to use source control to manage these files, but I’ll go ahead and call you insane if you don’t.

At its heart, chef is just a convention-based approach to organizing these files, along with a Ruby DSL to make the code they contain more declarative.  In those files, you answer the questions: What actions should I take on this particular machine?  And what machine- or environment-specific settings should I use when taking those actions?

Chef executables

There are several chef executables.  In a non-trivial environment, you will use:

  • Chef client - the agent that runs on the machines being managed. It is often set up to run as a service/daemon, periodically asking a chef server for updates and then taking action accordingly.

  • Chef server – a centralized server app that stores all the various files and distributes them to the machines being managed.

  • Knife – a command line tool used to control chef server. You normally run it from your workstation and connect remotely to a chef server.

However, when you are just getting started, for simple one-off scenarios, or for developing/testing your chef code, you will likely use Chef Solo –  a standalone version of chef.  You simply point Chef Solo at a local directory and run it.

Chef client and chef solo each run locally on the machine being managed or deployed to.  (To keep things simple, from here on I’ll say “chef” when I mean “either chef client or chef solo”.  If I mean server, I’ll say “chef server”.)  The machine chef runs on is called a node.

Chef loads up the Ruby files, reads in the json files, and based on their contents creates a single ordered list of things to run on the node. The list of things to run is called, surprisingly enough, the run list.  The run list might be different on each node, depending on what apps you want to install, what environment it’s in, etc.

Chef also uses data in those files to determine the right attributes to use when executing the run list.  Items in the run list may need file paths, logging levels, connection strings, etc. — and those are going to differ across machines and environments.  Attributes hold those values.  As with the run list, you can set default values and then override them by node, environment, etc.

Rephrasing what I said above using chef terminology:  Chef is a set of tools and conventions for generating the right run list and attributes for a specific node, and then executing that run list.

Chef Roles

role is just what you’d think: something like SqlServer, MyApp or whatever other roles you have in your system/application/environment.

A role is just data, which you can declare either in chef’s Ruby DSL or in json.  (If you use the Ruby DSL, it is converted to the equivalent json when chef runs.)  Here’s the json for a “MyApp” role:

{
  "name": "MyApp",
  "chef_type": "role",
  "json_class": "Chef::Role",
  "description": "My awesome app. It's web-scale!",
  "default_attributes": {
    "iis_log_root": "d:\logs"
  },
  "override_attributes": {
  },
  "run_list": [
    "role[BaseWindows2008R2Server]",
    "role[WebServer]",
    "recipe[MyCookbook::install_myapp_latest_version]"
  ],
  "env_run_lists": {
  }
}

A few interesting things to note:

  • A role’s run list can reference other roles.  In this case, we have some things we need to do on every Windows server — e.g., install ops tools, turn off the Windows firewall, disable UAC.  We have other things we need to do on every web server — e.g., install IIS with the modules we need, change the default logging directory.

  • There’s a default attribute for iis_log_root.  I just made that up, but it shows how you can set default values that are used across all machines in a role.  In this case, the WebServer role presumably makes use of the attribute to modify the IIS config.  We’re saying that all machines with the MyApp role should default to storing IIS logs in d:\Logs.

  • There is a recipe in the run list. Run lists always boil down (no pun intended) to recipes.  When chef uses the MyApp role, it will expand the run list so that it first includes the recipes in the BaseWindows2008R2Server role, then the recipes in the WebServer role, then the install_myapp_latest_version recipe.

Chef recipes, resources and providers

Recipes are Ruby files.  However, a recipe is still not the thing that does most of the work.  That is left up to Resources and Providers.  A typical recipe uses chef’s DSL to declare resources and an action to take on each one.

A resource is something you’re going to create, delete, modify or execute during a chef run.  It could be a file, a directory, a registry key, a package (msi or exe installer), a scheduled task, a Windows service, an IIS site/app/apppool, a user account, etc.  It could also be a custom resource type you have created.  For each resource, you declare the “action” you want to take: install, uninstall, create, delete, modify, etc.

The imperative code that actually does the work is in a Provider.  A resource is the abstraction — the simple declaration of what you want to do; a provider is the platform-specific implementation, written in Ruby, where (finally!) real things happen.

The separation between resources and providers is how the same recipes can be run on different platforms.  You use the same resource declaration to create a user account on any flavor of Linux, OSX, or Windows.  Chef detects the platform and runs the appropriate provider for that platform. Of course, not all resources have providers for all platforms — alas, you can’t create an IIS site on OSX…  Much more often it’s the other way around — some providers don’t have Windows implementations.

Here is part of a recipe I use to install RavenDB, which requires .NET 4.0:

windows_package ".NET 4.0" do
  package_name "Microsoft .NET Framework 4 Extended"
  source "http://download.microsoft.com/download/9/5/A/95A9616B-7A37-4AF6-BC36-D6EA96C8DAAE/dotNetFx40_Full_x86_x64.exe"
  options "/q"
  installer_type :inno
  action :install
end

This declaration says that we want to install a “windows_package” resource.  The underlying provider (the implementation) will download the source file and execute it with the “/q” option so there are no user prompts.

Here’s part of another recipe:

powershell "Start all IIS sites and app pools" do
  code <<-EOH
    Import-Module WebAdministration
    Restart-Service W3SVC -Force -WarningAction SilentlyContinue
    Get-Service SMTPSVC -ErrorAction SilentlyContinue | Restart-Service
    Get-ChildItem "iis:\\apppools" | Start-WebAppPool;
    Get-ChildItem "iis:\\sites" | Start-Website;
  EOH
  action :run
  not_if { buildIsCurrent }
end

Wait, Powershell?  I thought Chef was all Ruby?  Well, yes and no.  As with most build and deployment tools, you can shell out to any other executable you want.  In this case, we declare a powershell resource that we want to run.  If you already have Powershell scripts, you can reuse them by wrapping them in a resource like this.

Idempotence is important

It’s possible — indeed likely — that a recipe will appear in the run list of multiple roles.  If one of those roles references another, the recipe will appear multiple times in the expanded run list.  The recipe may well be run multiple times during the same chef run.

In addition, chef client is typically scheduled to run regularly on all the machines managed by a chef server.  You don’t want to re-install everything every time.  Even if it doesn’t harm anything, it’s a waste of time, CPU, etc.

Therefore, resource actions should be idempotent.  That is, no matter how many times they execute, the end result should be the same.

In the first example above, package_name is not just for user-friendliness.  It is the name used by msiexec to record the package installation in the registry.  If it already exists, the resource doesn’t do anything.

For Powershell and other resources that execute custom code and for custom resource types you create, you need to ensure idempotency yourself.  The powershell example uses chef’s not_if property that all resources share.  If that returns false, chef won’t take the action.

A better way would be to ensure idempotency inside the Powershell itself.  Whether or not to perform the action (not_if) is a different thing than ensuring multiple runs have the same end result (idempotence in the Powershell).  You may need to do one or the other or both, depending on the situation.

Chef cookbooks

Those resources look nice.  You probably want to use them yourself.  Like me, you may not know much Ruby, and you certainly don’t want to have to create dozens of resource types that someone else has already created.  On the other hand, you might create a super-cool resource type that you want to share.

cookbook is the unit of reuse in chef.  Think: module, gem, or NuGet package. A cookbook can contain resources and providers, default attribute values and files that those resources need to do their jobs, recipes that compose resources in useful ways, and a few other things.

The windows_package resource above is contained in Opscode’s standard “windows” cookbook.  The powershell resource is in the powershell cookbook.

Roles are not part of cookbooks.  Roles compose recipes and other roles.  To get value out of chef, you will almost certainly want to create a cookbook containing recipes that you can compose to create your server roles.

Cookbooks are available for download from the Opscode site — manually or directly in the knife command line tool.  Many community cookbooks are also available, usually on github.com.  Check out http://community.opscode.com/cookbooks or https://github.com/opscode-cookbooks for the long list.

That’s an awfully long post already.  I’ll stop there.  In future posts, I’ll cover getting set up with chef solo to play around, learn and start developing some roles/recipes; and then how to set up a real environment using chef server and client.