In this series of three posts we will show a configuration management framework, build of Puppet, Hiera and Jenkins that supports multiple environments:
- In this post we will review concepts and Puppet's configuration files
- Second post will be devoted to custom modules and interaction with Hiera
- Final, third post will focus on Jenkins integration
 |
| Fig 1: Deployment schema for two environments |
We will build on top of practices [1], and I strongly advice to read that article before proceeding further.
Conceptually, the workflow consist of the following steps:
1. Platform setup: during this step system interfaces and services of the nodes are configured (networking, file-system permissions, users and groups creations, rdbms, http servers, etc)
 |
| Fig 2: Puppet workflow |
2. Application setup: during this step business applications are installed and configured on the nodes
 |
| Fig 3: Jenkins workflow |
Our skeleton's file structure will look as follow:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| . | |
| ├── biz | |
| │ └── puppet | |
| │ ├── hieradata | |
| │ │ ├── ci | |
| │ │ │ ├── node | |
| │ │ │ │ ├── ip-10-0-8-11.sampup.com.json | |
| │ │ │ │ └── ip-10-0-8-12.sampup.com.json | |
| │ │ │ ├── common.json | |
| │ │ │ └── site.pp | |
| │ │ └── qa | |
| │ │ ├── node | |
| │ │ │ ├── ip-10-0-8-11.sampup.com.json | |
| │ │ │ └── ip-10-0-8-12.sampup.com.json | |
| │ │ ├── common.json | |
| │ │ └── site.pp | |
| │ └── modules | |
| │ ├── configuration | |
| │ │ ├── files | |
| │ │ │ ├── ci | |
| │ │ │ │ ├── enterprise_app_1 | |
| │ │ │ │ │ ├── dir.1 | |
| │ │ │ │ │ │ └── file.1 | |
| │ │ │ │ │ ├── file.1 | |
| │ │ │ │ │ └── file.2 | |
| │ │ │ │ └── web_app_1 | |
| │ │ │ │ ├── dir.1 | |
| │ │ │ │ │ └── file.1 | |
| │ │ │ │ ├── file.1 | |
| │ │ │ │ └── file.2 | |
| │ │ │ └── qa | |
| │ │ │ ├── enterprise_app_1 | |
| │ │ │ │ ├── dir.1 | |
| │ │ │ │ │ └── file.1 | |
| │ │ │ │ ├── file.1 | |
| │ │ │ │ └── file.2 | |
| │ │ │ └── web_app_1 | |
| │ │ │ ├── dir.1 | |
| │ │ │ │ └── file.1 | |
| │ │ │ ├── file.1 | |
| │ │ │ └── file.2 | |
| │ │ └── manifests | |
| │ │ └── init.pp | |
| │ ├── networking | |
| │ │ └── manifests | |
| │ │ └── init.pp | |
| │ ├── profile | |
| │ │ └── manifests | |
| │ │ ├── base.pp | |
| │ │ ├── init.pp | |
| │ │ └── web_server.pp | |
| │ ├── role | |
| │ │ └── manifests | |
| │ │ ├── enterprise_server.pp | |
| │ │ ├── init.pp | |
| │ │ └── web_portal.pp | |
| │ └── timezone | |
| │ └── manifests | |
| │ ├── init.pp | |
| │ └── utc.pp | |
| ├── etc | |
| │ └── puppet | |
| │ ├── hiera.yaml | |
| │ └── puppet.conf | |
| ├── agent_deploy.sh | |
| ├── deploy.sh | |
| └── install_modules.sh |
Three folders form a top-level hierarchy:
- /etc/puppet soon after puppet package is installed this folder contains standard puppet modules
In addition, we will place there two configuration files:
- puppet.conf describing puppet search paths and config
- hiera.yaml describing Hiera search paths and config
- /biz/puppet/modules contains all of the custom modules
- /biz/puppet/hieradata holds Hiera data files grouped by environment
puppet.conf
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [main] | |
| server = ip-10-0-8-10.sampup.com | |
| certname = ip-10-0-8-10.sampup.com | |
| #user = puppet | |
| #group = puppet | |
| vardir = /var/lib/puppet | |
| factpath = $vardir/lib/facter | |
| templatedir = $confdir/templates | |
| # The Puppet log directory. | |
| # The default value is '$vardir/log'. | |
| logdir = /var/log/puppet | |
| # Where Puppet PID files are kept. | |
| # The default value is '$vardir/run'. | |
| rundir = /var/run/puppet | |
| # Where SSL certificates are kept. | |
| # The default value is '$confdir/ssl'. | |
| ssldir = $vardir/ssl | |
| [master] | |
| dns_alt_names = ip-10-0-8-10.sampup.com, puppet | |
| # overriding location of the hierra.yaml | |
| hiera_config = /etc/puppet/hiera.yaml | |
| # site.pp path | |
| manifest = /biz/puppet/hieradata/$environment/site.pp | |
| # a multi-directory modulepath: | |
| modulepath = /etc/puppet/modules:/usr/share/puppet/modules:/biz/puppet/modules | |
| [agent] | |
| pluginsync = true | |
| # The file in which puppetd stores a list of the classes | |
| # associated with the retrieved configuratiion. Can be loaded in | |
| # the separate ``puppet`` executable using the ``--loadclasses`` | |
| # option. | |
| # The default value is '$confdir/classes.txt'. | |
| classfile = $vardir/classes.txt | |
| # Where puppetd caches the local configuration. An | |
| # extension indicating the cache format is added automatically. | |
| # The default value is '$confdir/localconfig'. | |
| localconfig = $vardir/localconfig | |
| certname = ip-10-0-8-11.sampup.com | |
| dns_alt_names = ip-10-0-8-11.sampup.com | |
| report = true | |
| archive_files = true | |
| environment = qa | |
This file should be normally divided into two: one with [main] and [master] sections deployed at Puppet Master node and another with [main] and [agent] sections deployed to Puppet Agent nodes.
puppet.conf provides wide spectrum of settings [3], but here we define only most critical ones, such as:
- modulepath defines path to the custom modules
- manifest defines where site.pp files are located
Note that in our case site.pp files are environment-specific. For instance site.pp for QA environment is stored in /biz/puppet/hieradata/qa/ folder - server identifies the Puppet Master domain name
- environment defines environment of the Agent node. For instance: qa, ci, etc
Details on the environment setting could be found at [2]
hiera.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- | |
| :backends: | |
| - json | |
| :json: | |
| :datadir: /biz/puppet/hieradata/%{::environment} | |
| :hierarchy: | |
| # A single node/ directory will contain any number of files named after some node’s fqdn (fully qualified domain name) fact. | |
| # (E.g. /etc/puppet/hiera/node/grover.example.com.json) This lets us specifically configure any given node with Hiera. | |
| # Not every node needs to have a file in node/ — if it’s not there, Hiera will just move onto the next hierarchy level. | |
| - node/%{::fqdn} | |
| - common | |
| :logger: | |
| - puppet |
Note that the attribute -common under :hierarchy: denotes common settings for the environment and in our case refers to the /biz/puppet/hieradata/${environment}/common.json file.
Continue with custom modules.
[1] Craig Dunn: Designing Puppet – Roles and Profiles
http://www.craigdunn.org/2012/05/239/
[2] Defining environment in Puppet/Hiera
http://docs.puppetlabs.com/guides/environment.html
[3] Puppet.conf in details
http://docs.puppetlabs.com/references/latest/configuration.html
