The Tools
Last updated
Last updated
Now that we’ve established what IaC is, we can start thinking about some of the tools used for infrastructure as code. Many tools fall under the IaC umbrella, including Terraform, AWS CloudFormation, Google Cloud Deployment Manager, Ansible, Puppet, Chef, SaltStack and Pulumi. It’s important to understand that some of these tools have different use cases, each with its own benefits. Due to the varied use cases of these IaC tools, having infrastructure provisioned, configured and deployed end-to-end will require using a combination of IaC tools rather than a single solution. To better understand these tools, this task will cover some of the key characteristics which distinguish these tools from each other so you know what tool to use and when. Bear in mind that the behaviour of these tools can often depend on exactly how they are used, often leading to debate over some of their characteristics; with this in mind, we will define those characteristics in question and how each tool is generally used.
First off, let’s establish the difference between declarative and imperative (also known as functional and procedural) IaC tools:
This involves declaring an explicit desired state for your infrastructure, min/max resources, x components, etc.; the IaC tool will perform actions based on what is defined. A state file is kept to ensure your infrastructure’s current state matches the desired state. In other words, it focuses on the what rather than the how. A declarative IaC tool is usually idempotent, meaning the same result will be achieved when run repeatedly. This is because that check is done against the state file, and if the infrastructure is already provisioned, it will make no changes if rerun. Examples of declarative IaC tools are Terraform, AWS CloudFormation, Pulumi and Puppet (Ansible also supports declarative).
This approach involves defining specific commands to be run to achieve the desired state; these commands need to be executed in a particular order. Imperative IaC is usually not idempotent in that if you run these tools multiple times, the same result won’t be replicated, or the tool may run into some configuration issues. This is because these commands will be run regardless of the state of your infrastructure, so running these commands on an already provisioned infrastructure may cause additional unwanted infrastructure to be added or may break the configuration of some existing components (depending on the set-up of the infrastructure and the commands being run). Examples of imperative IaC tools: Chef is the best example of an imperative tool; however, SaltStack and Ansible both support imperative programming as well.
Imagine it like directions to an X on a map, where imperative will give you a list of directions which will bring you to X if you are at the known start point (e.g., an unconfigured server). However, if you are not at this known start point (e.g., a partially configured server), you will not arrive at X but somewhere slightly different. Declarative takes into account where you are on the map and works out which directions need to be taken to get to X from that point. Deciding which to use can depend on what your infrastructure needs are. Declarative is considered a more straightforward approach that is easier to manage, especially for long-term infrastructure. In contrast, imperative IaC is more flexible, giving the user more control and allowing them to specify exactly how the infrastructure is provisioned/managed rather than the tool itself taking care of it.
Agent-based vs Agentless is a term that extends beyond infrastructure as code and is also used in the context of monitoring and security tools. In the context of IaC:
An “agent” will need to be installed on the server that is to be managed. It runs in the background and acts as a communication channel between the IaC tool and the resources that need managing. This agent is responsible for executing and reporting on the state of the infrastructure/managed resources. Note that this agent must be installed on every target system/node that needs to be managed. An agent-based IaC tool can perform tasks even when the system has limited connectivity or is offline, making it a robust choice for automation; however, you need to take care with maintenance when using these tools, monitoring the agent software closely so it can be restarted in the event of a crash.
Another thing to consider with agent-based IaC tools is that some of these tools will require opening ports on the server for inbound and outbound traffic so that the agent can push/pull configuration information; from a security perspective, more open ports are negative. Despite this, they offer a level of granular control over managed resources and can collect more detailed monitoring data. Because of this, they might be ideal if control is an important factor in your setup. Some agent-based IaC tools: Puppet, Chef, and Saltstack.
Agentless IaC tools, as you may have guessed, do not require agents to be installed on target systems. Instead, these tools leverage existing communication protocols like SSH, WinRM or Cloud APIs to interact with and provision resources on the target system. Agentless IaC tools benefit from being easier to use; their simplicity during set-up can mean faster deployment time, and their agentless nature can also mean the tool can be deployed across multiple environments without custom agent changes needing to be made based on each environment’s needs, making it a flexible choice. Also, compared to agent-based IaC tools, there is less maintenance and no risks surrounding the securing of an agent. However, agentless IaC tools generally have less control over target systems than agent-based tools. Agentless tools are useful in environments which have to adapt to fluctuating workloads, with resources being created/destroyed to meet these workloads. These newly created resources can be managed without having to rely on pre-installed agents. Agentless IaC tools: Terraform, AWS CloudFormation, Pulumi and Ansible.
Once again, both options have their advantages and disadvantages. Infrastructure provisioning and configuration can present developers with a very particular set of problems, and choosing the correct tool for the job, depending on what needs to be done, involves taking these into account.
Immutable vs mutable infrastructure refers to whether or not changes can be made to an infrastructure. To define both, let’s imagine we have a simple infrastructure, a virtual machine with a web server and application. If you want to change this infrastructure, for example, deploy some application code to your web application, this code will mean your web application will go from version 1 to version 2. How might that look for each?
Mutable infrastructure means you can make changes to that infrastructure in place. So, in our current example, we can upgrade the application on the server it’s currently running on. The advantage here is that no additional resources need to be provisioned for this update, with the resources already being used just needing to be changed. Where mutable infrastructure starts presenting issues is, imagine your web application update contains three changes, and whilst these updates are being made, one of the changes fails to apply. This is a common occurrence in development and can occur for several reasons, such as application misconfiguration or a network dropout. Now running on your server will be a web application version with two of the three changes required to be version 2. In other words, it’s no longer version 1 any more, but not quite version 2 either. When an application is updated in a production environment, it will have been tested first in development and staging environments. What won’t have been tested is this halfway version, leading to all kinds of potential issues. Imagine now that this update was not just rolled out on a single server, but many, maybe some updates succeeded, and some failed with errors in different changes. You can see how this can become an issue very quickly with different versions of the web application existing across many servers.
Immutable infrastructure means you cannot make changes to it. Once an infrastructure has been provisioned, that’s how it will be until it’s destroyed. Now, let’s apply that to our example. When this new version 2 of the web application is deployed, an entirely new infrastructure will be deployed with this new version. Now, you have two infrastructures stood up: one with version 1 of the web application and one with version 2. However, if all of the changes are implemented successfully (and only if they ALL are; otherwise, if it fails, the infrastructure will be torn down, and the process will be retried), the version 1 infrastructure can be torn down. This approach has some drawbacks, as having multiple infrastructures stood up side by side or retrying on failed attempts is more resource-intensive than simply updating in place, but it allows for consistency across servers. You know with certainty that x server is running version 2 of the web application.
Immutable IaC tools: Terraform, AWS CloudFormation, Google Cloud Deployment Manager, Pulumi (some of these tools can be configured to be used with a mutable infrastructure but work better with an immutable approach)
In the example given, it makes more sense to use an immutable infrastructure. However, to demonstrate further that each can have its own use case, imagine the server instead is a critical database system that requires regular maintenance updates and patching. If you were to use an immutable infrastructure for this, the critical database would have to be rebuilt from scratch every time, which can be a risky process when dealing with business-critical data. So, in this instance, it might make sense to use a mutable infrastructure.
All the characteristics discussed in this task so far are behavioural; in other words, how does this tool work? This way or that way. Considering what a tool's purpose is and when it is used in the development of infrastructure will tell us whether it is a provisioning tool or a configuration management tool. You might see these primary functions defined differently elsewhere, for example, infrastructure as code vs. configuration as code, but to determine which tool falls into which category, you simply need to ask, "What can it do?". If we think about four key tasks: Infrastructure provisioning (the set-up of the infrastructure), infrastructure management (changes made to infrastructure), software installation (initial installation and configuration of software/applications), and software management (updates made to software or config changes).
There is no single tool that can perform all 4 of these tasks. Instead, combining two tools, a provisioning tool and a configuration management tool, is common practice to cover all these. There will be a practical example of exactly what tools can cover which tasks in the static site coming up. Some examples of each type of tool:
Provisioning tools: Terraform, AWS CloudFormation, Google Cloud Deployment Manager, Pulumi
Configuration management tools: Ansible, Chef, Puppet, Saltstack
Let’s consider a real-life industry example of how these two types of tools might be used. Say you work for a company which needs to set up a new infrastructure for a web application. A similar one has been deployed before and fell prey to some malicious activity, so this time, you will install an agent to monitor for this activity. The infrastructure would be defined and provisioned using a provisioning tool like Terraform, and then a configuration management tool like Ansible would be used to install an agent within the provisioned infrastructure to monitor for malicious behaviour.
Now, you’ve been introduced to the world of infrastructure as code tools and their various uses. Each has its characteristics which best suit different use cases and scenarios. In that way, they are no different to physical tools; you have to assess the problem and choose the best tool to address it. Here's a little summary of what we've discussed, as well as a little bit more information about some of the tools mentioned:
A declarative, agentless infrastructure provisioning tool that works with immutable infrastructure. Terraform is one of the most popular infrastructure provisioning tools out there, allowing for the management of infrastructure across multiple cloud providers like AWS, GCP and Azure.
A hybrid, typically agentless configuration management tool that works with mutable infrastructure. Another example of a massively popular tool in the DevSecOps space. It's slightly harder to categorise than some of the other tools, because of how this tool functions and what it does can depend on how you employ it. For example, there is much discussion over whether Ansible is an imperative or declarative tool. In reality, Ansible is sort of a hybrid in a sense.
A declarative, agentless infrastructure provisioning tool that works with immutable infrastructure. Pulumi is similar in nature to Terraform but has become increasingly popular in recent years due to its ability to let users define their desired infrastructure using familiar general-purpose languages like Python, JavaScript, Go, Java and markup languages like YAML.
A declarative, agentless infrastructure provisioning tool which can be used to implement an immutable infrastructure on AWS. This is an AWS-managed service and provisions AWS cloud resources using JSON / YAML templates.
An imperative, agent-based configuration management tool that works with mutable infrastructure. Chef gets an infrastructure to a desired state by running a series of instructions contained in a file referred to as a 'Recipe'; a collection of these files is referred to as a 'Cookbook'.
A declarative, agent-based configuration management tool that works with mutable infrastructure. In Puppet, you define the desired configuration state using their own domain-specific language (DSL), "Puppet Code"; Puppet then automates this configuration through a Puppet primary server and agent.
