Google LOVES This Cloud Automation Tool (And Your Infrastructure Will Too!)

7 min readMar 2, 2024

Managing stacks of infrastructure can grind work to a halt quicker than a server rack toppling (trust me here!).

Clicking around consoles and stumbling through manual configuration after configuration piles on the long hours, errors, risks, and tech debt.

Yet ever-complex systems demand orderly change at cloud velocity no mere mortal can sustain.

But what if you could compose robust cloud networks, instances, and services safely through code the way you’d craft an application?

Enter Terraform is one of the hottest and fastest-growing infrastructure-as-code tools for infrastructure provisioning, management, and automation.

In today’s newsletter issue, I will cover

What is Terraform, its key benefits, and use cases
Terraform Basics
A simple example using Terraform’s Hashicorp Configuration Language (HCL) to provision infrastructure
Getting Set up
Infrastructure Inspecting and Planning
Learning Resources

We have a lot to cover. Let’s jump right in!

What is Terraform?

Terraform is an open-source infrastructure as code tool from HashiCorp that allows you to define, provision, and update cloud, container, networking, or storage resources in a declarative configuration file.

Think of it like an infrastructure manager speaking all major cloud APIs natively.

The key value Terraform provides is enabling infrastructure automation and orchestration for faster, safer changes at scale.

Some prime benefits and use cases:

Safely provision infrastructure faster through code rather than manual clicks and configs
Enforce security, compliance, and governance by policy as code reviewable in version control
Implement modular, reusable infrastructure components
Gain efficiency managing multi-cloud or hybrid environments from one tool
Simplify complex changes like zero-downtime migrations
Accelerate onboarding through automated sandbox environments

Key Benefits

A few key benefits that make Terraform well-suited for the complexity of current infrastructure:

Declarative Syntax — Terraform uses a declarative language called HCL to define the desired end-state of infrastructure. This is more intuitive than imperative scripts and helps provide a plan/preview of changes.
Execution Plans — terraform plan allows previewing exactly what add, change, or destroy actions will occur before applying. This prevents unexpected changes. (We will dive deeper into this in a later section)
Resource Graph Model — Terraform understands all dependencies between resources, allowing safe orchestration of complex infrastructure actions in the proper order.
Change Automation — Sophisticated logic, like creating a new instance before destroying the old one to prevent downtime, becomes simple in Terraform.
Tool Ecosystem — Providers exist for all major cloud platforms and services, with abundant modules in the public registry to incorporate common patterns.
Version Control — Terraform code can be treated like application code, with the same peer reviews, testing, and versioning best practices in tools like Git.

Some examples I’ve come across where Terraform is extensively used to improve cloud management are:

Multi-region Kubernetes cluster deployments while handling network topology complexity
Safely migrating legacy VMs to dockerized containers on ECS in phases
Regularly applying security hardening baseline policies to auto-scaling group AMIs
Configuring zero-downtime blue/green deployments for web applications

Terraform Basics

Now, let’s look at the core components and workflow basics of Terraform:

Providers: Terraform relies on plugins called providers to interact with cloud platforms like AWS, Azure, or GCP and services like Kubernetes. Providers handle authentication, expose resources, and execute actions. Hundreds exist.

Resources: Resources represent infrastructure objects — VM instances, subnets, load balancers. Syntax aws_instance defines component types. Resource blocks contain configurations.

State Files: Terraform tracks resource state data locally in state files indicating id, attributes, and relationships. This allows Terraform to plan incremental changes by comparing the desired end state code to the current state snapshot.

Terraform CLI: Terraform covers major commands like init, plan, apply, and destroy, driving the core workflow. Init handles provider installs and backend connectivity, plan previews changes, apply makes them, destroy deleted infrastructure not just deprovisioning it.

Lifecycle Overview: Typical workflow involves authoring HCL config files with resources > init > plan > apply cycles to reach desired infrastructure, punctuated by runs of plan to preview subsequent changes before updating live environments again via apply.

HCL: The Hashicorp Configuration Language (HCL) used defines blocks, arguments, and expressions for resources.

A simple example using Terraform’s Hashicorp Configuration Language (HCL) to provision infrastructure

At a high level, HCL represents the declarative language Terraform utilizes to define infrastructure resources and configuration.

It allows describing desired end-state componentry and relationships in a structured manner that both humans and systems can parse.

Some key elements of HCL syntax:

Blocks — Resources like aws_instance represent components. Blocks contain meta-arguments for providers and nested configuration keys/values for that resource, such as compute sizing.
Arguments — Configuration variables set resource-specific attributes like AMI ID or provision additional options like multiple network interfaces.
Expressions — Dynamic expressions set arguments based on functions, math operations, or external variables to enable reusability. For example, dynamically sizing volume storage by referencing a num_users input variable provided at runtime.

Here is a simple example of Terraform configuration with HCL syntax that provisions an AWS S3 bucket and EC2 instance:

# Configure AWS provider
provider "aws" {
  region = "us-east-1"
}

# Create S3 bucket 
resource "aws_s3_bucket" "data" {
  bucket = "my-data-bucket"
  acl    = "private"  tags = {
    Name        = "My data bucket"
    Environment = "Dev"
  }
}# Create EC2 instance
resource "aws_instance" "app" {
  ami           = "ami-2757f631" // Amazon Linux
  instance_type = "t2.micro"
  
  tags = {
    Name = "App Server"
  }
}

This example showcases a few HCL fundamentals:

Declaring the AWS provider with region set to “us-east-1”.
Defining an S3 bucket resource with ID “data”
Setting attributes like bucket name, access control
Using tags for metadata
Defining an EC2 instance resource “app”
Referencing AMI ID and size args
Again using tags for EC2 instances

On using the “terraform apply” command, Terraform will create both the S3 bucket and EC2 instance based on this configuration.

Getting Set up

Getting set up with Terraform involves a few key steps — installing the tool, configuring cloud credentials, initialization, and remote state setup best practices:

Installing Terraform

Terraform supports all major operating systems. On Linux/macOS, use standard package managers like apt-get, homebrew, or manually extract and add the terraform binary to your PATH. Windows users can install Chocolatey.

Configuring Credentials

Providers require valid credentials to manage infrastructure. For AWS, ensure your environment is configured with access and secret access keys, allowing necessary permissions. Or utilize an IAM role attached to an EC2 instance.

Initializing Providers

Within a Terraform working directory, run terraform init. This installs providers defined in the configuration, like aws. Init also sets up the backend based on the configuration.

Using Remote State Storage

Terraform state should be stored remotely in a backend like S3 to enable collaboration and protect local file loss. The backend block configures this location, telling Terraform where to store and retrieve state data with locking to prevent corruption across concurrent access.

A typical S3 backend example:

terraform {
  backend "s3" {
    bucket = "mybucket"   
    key    = "path/to/my/key"
    region = "us-east-1" 
  } 
}

Infrastructure Inspecting and Planning

Terraform provides robust capabilities for inspecting the current state, planning changes, and debugging issues before ever touching live infrastructure:

Terraform Plan

The terraform plan command previews proposed changes in a “changeset” showing all actions that will occur when you run terraform apply. Create, update, and destroy events highlighted along with resource attributes to be modified. Plans should always be inspected carefully as a safety net.

Terraform State

terraform state list summarizes all known resources.

terraform state show <resource> displays details on a specific component’s attributes and metadata useful when diagnosing issues.

Terraform Graph

terraform graph renders a visual dependency graph of resources emphasizing ordering and relationships — invaluable for complex changes.

Terraform Console

terraform console opens an interactive console to experiment directly with state data, variables, and outputs, which is helpful for dynamically inspecting the current environment state while planning/debugging.

Debugging Issues

Beyond the tools above, warning messages, logging flags, trace-level verbosity, and community advice help resolve problems by applying changes or mismatches between the desired and current states. Linters also catch HCL formatting errors.

Additional Key Features

Terraform offers additional important features beyond just resource provisioning to enable encapsulation, reuse, and parameterization:

Input Variables

Defining variables allows the customization of infrastructure without changing code. Assign APIs keys set regional endpoints or component sizes dynamically at runtime:

variable "instance_type" {
  default = "t2.micro"  
}

resource "aws_instance" "app" {
  instance_type = var.instance_type 
}

Outputs

Exposing select resource attributes from the state gives access to other scripts (e.g., outputting site URLs after provisioning).

Workspaces

Workspaces partition distinct Terraform state files to allow managing separate environments like prod vs. staging resources from the same configurations.

Modules

Modules package reusable groups of resources neatly as compartmentalized black boxes. They simplify configuration reuse and abstraction. Teams can compose reliable components quickly:

module "vpc" {
  source = "./modules/aws-base-netwoking" 
  
  cidr = "10.0.0.0/16"
}

Key takeaways

And that wraps our beginner’s guide to Terraform!

By now, you should feel equipped with a basic yet comprehensive understanding of how this invaluable infrastructure provisioning tool works its magic — from HCL syntax to configurable cloud resources to workspaces and modules unlocking reusable infrastructure components.

We covered the Terraform workflow and key basics — providers exposing infrastructure APIs you can command with simple declarative code to build, update, and scale systems safely through code rather than risky manual clicks. Architect your infrastructure environment as neatly as you would any application.

We also covered additional key features Terraform provides, like Input Variables, Outputs, Modules, and Workspaces.

Always double-check plans and rely on Terraform’s ability to determine creation order and dependencies when wrangling resources.

Store remote state for durability and teamwork.

Take baby steps and let example projects guide the journey, whether deploying Docker containers or spinning up EC2 server fleets.

Our infrastructure challenges accelerate daily. But with Terraform in hand, you now have a powerful tool to keep up and adapt gracefully at cloud speed, no matter tomorrow’s demands!

Learning Resources

This is not an exhaustive list but a guide to get you started:

Terraform explained in 15 mins: https://youtu.be/l5k1ai_GBDE
Terraform Tutorials: https://developer.hashicorp.com/terraform/tutorials?product_intent=terraform
Terraform Up & Running book by Yevgeniy Brikman: https://www.terraformupandrunning.com/
HashiCorp Terraform Associate Certification Course: https://youtu.be/V4waklkBC38

About Me

Amrut is a Cloud Software Architect with 10+ years of experience in the software industry. He is also 4x AWS Certified and a Udemy instructor.

He is the author of The Tech Pulse Newsletter, a free weekly newsletter. He shares his insights on AWS Cloud, DevOps, MLOps, System Design, Software Architecture and Engineering.

You can also follow him on LinkedIn and X to get a daily dose of these topics.

Originally published at https://thetechpulse.substack.com/ on January 13, 2024.