Hitchhiker's Guide to Chef

Knife

knife is a command-line tool that provides an interface between a local chef-repo and the Chef server. knife helps users to manage [1]:

• Nodes
• Cookbooks and recipes
• Roles, Environments, and Data Bags
• Resources within various cloud environments
• The installation of the chef-client onto nodes
• Searching of indexed data on the Chef server

Chef.io makes available an all-in-one knife quick reference commands, you can either get the source files or get a ready to print PDF.

Cookbooks

In Chef, a cookbook is the fundamental unit of configuration and policy distribution. A cookbook defines a scenario and contains everything that is required to support that scenario [2]:

• Recipes that specify the resources to use and the order in which they are to be applied
• Attribute values
• File distributions
• Templates
• Extensions to Chef, such as custom resources and libraries

The reference language used to create cookbooks and defining recipes is Ruby. There is probably as much love for it as there is hate, in this post I will refrain about giving my personal opinion about it.

In this section the Cookbook should be seen as the development of the scenario itself and all properties that are a part of it.

default['myapp']['dir'] = '/srv/myapp'

node.default['myapp']['dir'] = '/srv/myapp'

cookbook_file '/etc/sshd_config' do
  source 'sshd_config'
  owner 'root'
  group 'root'
  mode '0640'
  action :create
end

file '/etc/resolv.conf' do
  content 'nameserver 1.1.1.1
nameserver 1.0.0.1'
  mode '0644'
  owner 'root'
  group 'root'
end

remote_file '/tmp/wordpress-latest.tar.gz' do
  source 'https://wordpress.org/latest.tar.gz'
  owner 'root'
  group 'root'
  mode '0755'
  action :create
end

template '/etc/resolv.conf' do
  source 'resolv.conf.erb'
  owner 'root'
  group 'root'
  mode '0644'
  variables(nameservers: node['nameservers']) # optional
end

1
2
3
4
5

# DNS resolver configuration managed by Chef
# do not edit, changes will be overridden!
<% @nameservers.each  do |ns| %>
nameserver <%= ns %>
<% end  %>

default['nameservers'] = ['1.1.1.1', '1.0.0.1', '8.8.8.8', '8.8.4.4']

1
2
3
4
5
6

# DNS resolver configuration managed by Chef
# do not edit, changes will be overridden!
nameserver 1.1.1.1
nameserver 1.0.0.1
nameserver 8.8.8.8
nameserver 8.8.4.4

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

name '<cookbook name>'

license '<license type>'

maintainer '<first and last name>'
maintainer_email '<your e-mail>'

issues_url '<github url>/issues'
source_url '<github_url>'

description 'small description'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))

version '0.0.1'
chef_version '~> 14.0'

Linting

Linters are very important in a development workflow, a linter verifies a program or program segments against standard libraries. It checks the code for common portability errors. It tests the programming against some tried and true guidelines. linting your code is a necessary (though not sufficient) step in writing clean, portable, effective programs. But lint is not perfect. It will not magically salvage bad code. It will not find all your bugs. [7] I certainly do not have anything to add to this sentence besides the tools we should use for our cookbooks:

• cookstyle
• foodcritic (deprecated since 25/09/2020)

Tests

According to Chef's documentation, we can use ChefSpec to simulate the convergence of resources on a node:

• Is an extension of RSpec, a behavior-driven development (BDD) framework for Ruby
• Is the fastest way to test resources and recipes

ChefSpec is a framework that tests resources and recipes as part of a simulated chef-client run. ChefSpec tests execute very quickly. When used as part of the cookbook authoring workflow, ChefSpec tests are often the first indicator of problems that may exist within a cookbook. [8]

It does not make sense to make tests to ensure that, i.e: a package was really installed or the contents of a template file are up to date as the recipes that apply those things already do them. It is however sane to write tests reflecting a set of operations that should be applied to the node according to the current stable state of the recipes, this assures that if some regression, i.e: a wrong file is changed or a package dependency is no longer available, the recipe(s) will fail some test and you will notice that something is wrong before converging a node.

Compliance

InSpec is a free and open-source framework for testing and auditing your applications and infrastructure. InSpec works by comparing the actual state of your system with the desired state that you express in easy-to-read and easy-to-write InSpec code. InSpec detects violations and displays findings in the form of a report, but puts you in control of remediation. [9]

It can be integrated with Chef, but that is out of the scope of this post. Maybe another day.

Kitchen

Kitchen is a tool that allows us to test infrastructure code on one or more platforms in isolation.

"A driver plugin architecture is used to run code on various cloud providers and virtualization technologies such as Vagrant, Amazon EC2, and Docker.

For Chef workflows, cookbook dependency resolution via Berkshelf or Policyfiles is supported or include a cookbooks/ directory and Kitchen will know what to do.

Kitchen is used by all Chef-managed community cookbooks and is the integration testing tool of choice for cookbooks."

We specify every requirement for kitchen in .kitchen.yml file usually located in the root directory of our coookbook, but really kitchen only cares about having that file present in whatever directory you are calling it.

An example file for a .kitchen.yml configuration file running on Openstack would be:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

driver:
  name: openstack
  openstack_auth_url: <%= ENV['OS_AUTH_URL'] %>
  openstack_domain_name: <%= ENV['OS_USER_DOMAIN_NAME'] %>
  openstack_region: <%= ENV['OS_REGION_NAME'] %>
  openstack_project_name: <%= ENV['OS_PROJECT_NAME'] %>
  openstack_project_id: <%= ENV['OS_PROJECT_ID'] %>
  openstack_username: <%= ENV['OS_USERNAME'] %>
  openstack_api_key: <%= ENV['OS_API_KEY'] %>
  flavor_ref: <%= ENV['OS_FLAVOR'] %>
  key_name: <%= ENV['OS_KEYPAIR_NAME'] %>

provisioner:
  name: chef_zero
  data_bags_path: "data_bags"
  encrypted_data_bag_secret_key_path: "secrets/encrypted_data_bag_secret"


transport:
  ssh_key: .<%= ENV['OS_KEYPAIR_NAME'] %>.pem

verifier:
  name: inspec

platforms:
  - name: ubuntu-18
    transport:
      username: ubuntu
    driver:
      image_ref: Ubuntu-18.04-Latest
      server_name_prefix: ubuntu-bionic
  - name: debian-9
    transport:
      username: debian
    driver:
      image_ref: Debian-9-Latest
      server_name_prefix: debian-stretch

suites:
  - name: defaults
    run_list:
      - recipe[mycookbook::predefault]
      - recipe[mycookbook::default]
      - recipe[mycookbook::postdefault]
    verifier:
      inspec_tests:
        - test/integration/predefault
        - test/integration/default
        - test/integration/postdefault
    attributes:
  - name: verytastyrun
    run_list:
      - recipe[mycookbook::saltandpepper]
    verifier:
      inspec_tests:
        - test/integration/salt
        - test/integration/pepper
        - test/integration/spicy
    attributes:

In this configuration we have a 2 suites:

• defaults
• verytastyrun

We use the Openstack driver, and so we have to configure our environment:

We also configure the environment variable OS_KEYPAIR_NAME which should correspond to the name of the keypair created in Openstack with our OS_USERNAME account, and the private key should be located in .OS_KEYPAIR_NAME.pem in the root directory of our cookbook, as defined by the transport['ssh_key'] yaml key.

Next we specify which provisioner to use, in our case we are testing a chef cookbook so we chose chef_zero. Here we can specify configuration options to be passed to chef, in this example we define options for data bags.

We then define which platforms should be converged, which should be available in Openstack. We can define a prefix for the instance name with the server_name_prefix property.

Finally, in the suites section we define which convergence run lists should be made available to kitchen. In this example we have two:

• defaults
• verytastyrun

In the run_list property we should specify from which cookbook we want to get each recipe.

A usual command sequence to converge a suite and destroy the converged instances would be:

1. localhost:~$ kitchen converge <platform name>
2. localhost:~$ kitchen destroy <platform name>

If, however, we would like to just create one of the machines, i.e: debian, login into it, do some manual testing and then converge the machine we would:

1. localhost:~$ kitchen create debian
2. localhost:~$ kitchen login debian
3. debian-stretch-asdfgh:~$ logout
4. localhost:~$ kitchen converge debian
5. localhost:~$ kitchen destroy debian

Users

Like any architecture which has an authentication component, we have users. What can we do with it?

Organizations

An organization is the top-level entity for role-based access control in the Chef server. Each organization contains the default groups (admins, clients, and users, plus billing_admins for the hosted Chef server), at least one user and at least one node (on which the chef-client is installed). The Chef server supports multiple organizations. The Chef server includes a single default organization that is defined during setup. Additional organizations can be created after the initial setup and configuration of the Chef server. [10]

Groups

A group is used to define access to object types and objects in the Chef server and also to assign permissions that determine what types of tasks are available to members of that group who are authorized to perform them. Groups are configured per-organization. [10]

Individual users who are members of a group will inherit the permissions assigned to the group. The Chef server includes the following default groups: admins, clients, and users. For users of the hosted Chef server, an additional default group is provided: billing_admins. [10]

There is another group named server-admins, it is a global group that grants its members permission to create, read, update, and delete user accounts, with the exception of superuser accounts. The server-admins group is useful for users who are responsible for day-to-day administration of the Chef server, especially user management via the knife user subcommand. Before members can be added to the server-admins group, they must already have a user account on the Chef server. [10]

Group administration is done via chef-web, chef-automate or the hosted chef management interface.

Cookbooks

We have already covered what a Cookbook are. In this section the Cookbook should be seen not as the development of the scenario itself but as one or more versions of a already developed scenario which each version should be frozen. Meaning that if any changes should occur to a certain version of the cookbook, the cookbook is no longer on that version. Although not mandatory, this practice makes it harder to break something that depends on a cookbook's version.

To manage cookbooks present on the Chef Server we can perform the following operations, using the knife and chef commands:

Nodes

To manage nodes that exist on a Chef Server we either use the knife command or the web management interface.

Run-lists

A run-list defines all of the information necessary for Chef to configure a node into the desired state. A run-list is [11]:

• An ordered list of roles and/or recipes that are run in the exact order defined in the run-list; if a recipe appears more than once in the run-list, the chef-client will not run it twice
• Always specific to the node on which it runs; nodes may have a run-list that is identical to the run-list used by other nodes
• Stored as part of the node object on the Chef server
• Maintained using knife and then uploaded from the workstation to the Chef server, or maintained using Chef Automate

Policies

Policy maps business and operational requirements, process, and workflow to settings and objects stored on the Chef server [12]:

• Roles define server types, such as “web server” or “database server”
• Environments define process, such as “dev”, “staging”, or “production”
• Certain types of data - passwords, user account data, and other sensitive items - can be placed in data bags, which are located in a secure sub-area on the Chef server that can only be accessed by nodes that authenticate to the Chef server with the correct SSL certificates
• The cookbooks (and cookbook versions) in which organization-specific configuration policies are maintained

Environments

An environment is a way to map an organization’s real-life workflow to what can be configured and managed when using Chef server. Every organization begins with a single environment called the _default environment, which cannot be modified (or deleted). Additional environments can be created to reflect each organization’s patterns and workflow. For example, creating production, staging, testing, and development environments. Generally, an environment is also associated with one (or more) cookbook versions. [13]

Roles

A role is a way to define certain patterns and processes that exist across nodes in an organization as belonging to a single job function. Each role consists of zero (or more) attributes and a run-list. Each node can have zero (or more) roles assigned to it. When a role is run against a node, the configuration details of that node are compared against the attributes of the role, and then the contents of that role’s run-list are applied to the node’s configuration details. When a chef-client runs, it merges its own attributes and run-lists with those contained within each assigned role. [14]

Attributes

Previously we saw what are attributes and how to define them in a cookbook. However, attributes can also be defined in:

• The state of the Node itself
• Roles
• Environments

We also saw which types of precedence exist, but taking this new information that Roles, Environments, and Node state play a part on this we can view the attribute precedence as a table, as shown in {F#fig-chef-preced}:

Chef attributes overview

For more information on attributes take a look at [4].

Data Bags

Data bags store global variables as JSON data. Data bags are indexed for searching and can be loaded by a cookbook or accessed during a search. [15]

We will usually use the knife tool to manage data bags, although creating data bags manually can be useful to test cookbooks with Kitchen.

In order to use manually created data bags in our recipes we can create a directory named data_bags that follows a simple directory/file structure, i.e:

data_bags/
├── db_passwords
│   ├── mysqlroot.json
│   └── slurm.json
├── secrets
│   └── munge.json  
├── service_passwords
│   └── myservice.json
└── user_passwords
    ├── alice.json
    └── bob.json

This way every subdirectory within the data_bags directory corresponds to a data bag, and every .json file corresponds to an item to each of those data bags.

Each data bag item should also follow a specific structure: [15]

1
2
3
4
5
6

{
  /* This is a supported comment style */
  // This style is also supported
  "id": "ITEM_NAME",
  "key": "value"
}

$ openssl rand -base64 512 | tr -d '\r\n' > encrypted_data_bag_secret

Chef Client

A chef-client is an agent that runs locally on every node. When a chef-client is run, it will perform all the steps that are required to bring the node into an expected state, including: [16]

• Registering and authenticating the node with the Chef server
• Building the node object
• Synchronizing cookbooks
• Compiling the resource collection by loading each of the required cookbooks, including recipes, attributes, and all other dependencies
• Taking the appropriate and required actions to configure the node
• Looking for exceptions and notifications, handling each as required

Ohai

Ohai is a tool that is used to collect system configuration data and is run at the beginning of every Chef run to determine the system's state.

The types of attributes Ohai collects include but are not limited to:

• Operating System
• Network
• Memory
• Disk
• CPU
• Kernel
• Host names
• Fully qualified domain names
• Virtualization
• Cloud provider metadata

Attributes that are collected by Ohai are automatic level attributes, in that these attributes are used by the chef-client to ensure that these attributes remain unchanged after the chef-client is done configuring the node. [16]

Berkshelf

Berkshelf can include multiple Chef Supermarket instances for dependency resolution. Cookbook dependency resolution is performed from the top down. The first source defined in the Berksfile will be searched for the cookbook before the second source.

The Berksfile first looks for the cookbook on the private Chef Supermarket and, if not discovered there, then looks on the public Chef Supermarket. [17]

1
2

source 'https://your_private_supermarket_url'
source 'https://supermarket.chef.io'