Introduction and goals
This blog series is about using Docker to run a complete and fully functional Puppet stack consisting of a Puppetmaster, The Foreman and PuppetDB.
Until now I've been using Vagrant to recreate my multi-node production Puppet setup. This has allowed me to develop and test my Puppet code locally. Anyone with a non-trivial Puppet deployment has probably done this by now.
However, lately Docker has been generating a lot of buzz and I wanted to find out if I could benefit from recreating my Puppet setup using containers only. This blog series is a result of that effort and I hope it is useful to others who are looking to Dockerize a fairly complex application stack.
From the beginning I set the following goals for this project:
- PuppetDB, Foreman and the Puppetmaster should each run in their own container
- The setup and configuration of the containers should be fully automated, no manual steps required
- The result should be described and open-sourced so that others may be benefit from it
This will be a really 'hands-on' blog series that does not describe each tool in full detail but rather how to use each one to achieve the above goals. So let's get started!
This tutorials contains six parts in total (including this introduction). The other parts are:
- Building container images with Packer and Puppet
- Puppetmaster, PuppetDB and Foreman installation with Puppet
- Lifting Docker containers with Crane
- Service discovery with Docker: introducing Docker-spy
- Provisioning Docker Containers with Puppet and The Foreman
Each post describes a different aspect of working with Docker containers and images. Please jump to a particular blog post if you are interested in a certain subject only. However, if you want to build the whole dockerized Puppet development stack you will have to read each one in order.
I've created a github repository that contains all the necessary configuration files and scripts to recreate the setup described in this blog series. It should provide you with a one-step procedure to setup the whole stack and enjoy a fully functional Puppet environment. Please check the included documentation for further instructions.
To achieve the goal of a fully automated setup we need a couple of tools. The most important one is Docker. This tool allows for easy management of Linux containers and provides a public infrastructure for sharing images. I've used Crane to manage the containers like you would manage virtual machines with a Vagrantfile. Ofcourse running empty containers is not very interesting so we need something to provision the containers.
I've chosen Packer from Hashicorp for this purpose. It's a really nice tool that allows you to create many different (cloud) images from a single definition. Basically it orchestrates the commands that need to be run during image creation. It includes a Docker builder so it suits my requirements well.
For the actual provisioning it seemed only fitting that I would use Puppet to setup the whole Puppet stack. This means writing some manifests and using (community) modules to generate fully provisioned containers.
After image creation we need to run the containers and have some kind of service discovery mechanism in place to make the inter-container communication happen (more on this in a later part of this blog series). For this purpose I have created my own little Docker service discovery tool called Docker-spy. I will go into full detail about this tool in an upcoming blog post in the series.
With the introductions out of the way let's get started and take a look at the first step of our process: building container images with Packer and Puppet!