Better infrastructure management a.k.a. IAC (Infrastructure as Code)
Last year we faced the problem of renewing the hardware of the machines that we are using at agile42 and Agilo Software. This was a good opportunity to find some better solution for the management of the configuration and the installed software.
We had already some experience with provisioning systems in order to automate the configuration settings of a machine, but we took this as an opportunity to have a closer look at the whole idea of Infrastructure as Code.
We can say that the concept of the Infrastructure as Code (IaC) is an evolution of the old system administration approach that consisted in creating tons of scripts in order to automate the basic configurations that were shared among the machines.
In fact not only it’s a way to automate a series of steps in order to "restore" or "set" the state of a machine, but it gives a developer/system administrator an entire framework for orchestrating and automating your infrastructure in the easiest way possible, decoupling the architecture and the OS of your machine from the configuration that needs to be set. Moreover it gives an easy way to test your code for (almost) complete reliability.
There are several IaC solutions, but we decided to focus on the two most used: Chef and Puppet.
Chef is a pure configuration management software written in Ruby. It uses a cookbook-recipe paradigm to create a configuration that can be easily replicated. It doesn't take care of the instantiation of the virtual machine, but thanks to an Operating System image repository (called Opscode/Bento) provided by the chef community, and a virtualization software like Vagrant, it is possible to test your configuration on development Virtual Machines (VMs).
The structure can be summarized in 3 macro areas:
The Development, where we have the development tools for the creation of a recipe, a test driven infrastructure based on Rspec and Test Kitchen in order to test respectively the syntax of your recipes and the actual outcome configuration on a development VM, a Development Kit (DK) for the communication with the Chef Server and for pushing the new/updated recipes.
The Chef Server, which is in charge of storing the recipes and orchestrating the Nodes, managing the recipes for the Nodes and the users access and collecting the reports from the Nodes.
The Distributed Nodes, that have access to the chef server in order to pull their configuration and provision it.
Puppet follows the same concept of Chef, but it uses its own language for the configuration declaration. The main difference between the two is on how the order of the written commands is defined. In Puppet the code describes a list of dependencies to have a better control; in chef on the other hand there is no explicit dependency among commands, but it mostly follows the order of the commands adding some predefined priority.
Puppet structure is way simpler than the one that Chef provides. All the configurations are stored in a versioned repository (like Git or SVN) accessible by a Master (called puppetmaster). This one is in charge of orchestrating the nodes pushing regularly the configurations based on the node and collecting the reports of the provisioning.
Both solution are really awesome and can cover all the needs of a small/medium/large company and choosing between them is just a matter of point of view. In fact, the two communities created around these tools cannot find a real reason for which one is better than the other, is just a matter of "is more suitable for me because...".
For us the "most suitable" was Chef and the main reasons are two:
- The dependency approach for the Puppet commands can give an higher control over the provisioning, but for complex systems can also lead to a very hard to read (and consequently, hard to maintain) code. In Chef, instead, thanks to the higher abstraction language and the cookbook-recipe paradigm, it is easier to define "portions of code" with a general purpose and then combine them to your needs.
- With Chef comes also a series of tools for testing the syntax of your code (like rspec) and the real outcome of the provisioning (like test kitchen and bats). These two tools really helped us in the learning process.
- Django-treebeard and Wagtail page creation
- The Charity Sport Tournament in Lublin
- New Release of Agilo for Trac (0.9.15/1.3.15)
- Incontro DevOps Italia 2016
- Configuring Test Kitchen output for Jenkins
- Configuring Test Kitchen on Jenkins
- Better infrastructure management a.k.a. IAC (Infrastructure as Code)
- Our approach to automated visual regression testing
- Test parallelization with Lettuce, take 2
- New Release of Agilo for Trac (0.9.14/1.3.14)