I'm following the DevOps Zero to Hero playlist by Abhishek Veeramalla sir, and his explanations are incredibly crystal clear. Using his insights, I’m preparing this blog to share key concepts that i have learned from his videos. If you're diving into DevOps, I highly recommend following his content! 🚀
Problem: Managing Infrastructure
Scenario: Flipkart's Infrastructure Management Challenge
Let's say we are DevOps engineers working at Flipkart. As a company, Flipkart can create its compute resources anywhere—on any cloud platform (AWS, Azure, GCP) or on premises.
Flipkart has around 300 applications to deploy, and for that, they need servers, because servers are where applications are deployed and exposed to customers.
Flipkart’s Infrastructure Management Approach
To host these applications, Flipkart can choose:
AWS (Amazon Web Services)
Azure (Microsoft Cloud)
GCP (Google Cloud)
On-premises servers
As DevOps engineers, we evaluated and analyzed all options and decided to use AWS to host the entire infrastructure. We then started creating the necessary resources on AWS, such as
EC2 instances
S3 Buckets
RDS
Automating Infrastructure Provisioning with AWS CloudFormation
Manual vs Automated Infrastructure Provisioning:
Manually creating infrastructure via the AWS console is time-consuming. So, as good DevOps engineers, we automated the entire process using
AWS CLI
AWS CloudFormation Templates (CFT)
Now, whenever a developer asks for resources (Eg:"I need 10 EC2 instances"), we simply run a pre-written CloudFormation script, and AWS provisions the resources instantly. Similarly, we have scripts ready for S3, RDS, and more. Everything was running smoothly!
The Challenges of Cloud Migration & Vendor Lock-In
Cloud Migration: Flipkart Moves from AWS to Azure
After some time, management decides to switch from AWS to Microsoft Azure due to reasons like cost, support, or compliance. Now, all the 100+ AWS CloudFormation scripts we wrote are useless because CFT(CloudFormation Templates) is specific to AWS.
If a developer now requests 10 VMs on Azure, we cannot use AWS CloudFormation. We must rewrite everything using Azure Resource Manager (ARM) templates to automate Azure resources
Cloud Migration: Flipkart Moves to On-Premises Infrastructure
Later, our company decides to move from Azure to on-premises (OpenStack) due to further cost concerns. Again, we must rewrite all automation scripts—this time using Heat Templates for OpenStack.
The Rise of Hybrid Cloud
Nowadays, many organizations adopt a hybrid cloud model, where different parts of their infrastructure are hosted on different platforms.
For example:
AWS may offer better storage services, so we use AWS for storage.
Azure may provide better project management tools, so we use Azure for DevOps-related tasks.
This approach increases complexity because DevOps engineers must now learn these multiple things:
AWS CloudFormation (for AWS automation)
Azure ARM Templates (for Azure automation)
OpenStack Heat Templates (for on-premises automation)
Here Comes the Ultimate Solution to solve these problems!
The Ultimate Solution: Terraform
Instead of learning multiple platform-specific tools, we need a single tool that can work with any cloud provider. This is where Terraform comes in!
What is Terraform?
Terraform is an Infrastructure-as-Code tool developed by HashiCorp that allows DevOps engineers to automate infrastructure across any cloud provider using a single, unified tool.
Instead of learning separate tools like AWS CloudFormation, Azure ARM, and OpenStack Heat Templates, we only need to learn Terraform.
How Terraform Works?
If we use AWS, Terraform uses an AWS module to automate resources.If we migrate to Azure, we only update the provider details in the Terraform script—no need to rewrite everything from scratch!
Why Terraform?
✅ Simplifies cloud automation (no need to learn multiple tools)
✅ Makes cloud migration easy (minimal changes needed)
✅ Supports all major cloud providers (AWS, Azure, GCP, OpenStack, etc.)
✅ Saves time and effort (one script instead of hundreds of cloud-specific scripts
Terraform : API as Code
How API's Enable Automation with Terraform
Terraform follows the API as Code concept, which means it interacts with cloud provider APIs (AWS API, Azure API, GCP API, etc.) to automate infrastructure.
What is an API?
API (Application Programming Interface) allows us to communicate with an application and get a response back.
For example : To access Google
Manually: Open a browser, type www.google.com , and hit enter. This sends a request to Google's servers, and we receive a response in the form of a web page.
Programmatically: Instead of opening a browser, we can write a script that interacts with Google’s API to fetch data automatically. Many applications, like GitHub, Google etc.. expose their APIs
Example: GitHub API
GitHub provides an API that allows us to retrieve information without logging in manually.
Instead of opening GitHub in a browser and authenticating: We can use the terminal and send a request using curl or an HTTP GET request. GitHub will respond with the requested information in a structured format (like JSON). This way, we can interact with applications programmatically using APIs.
Think of an API as a waiter in a restaurant:
You place an order (Request).The waiter (API) takes your request to the kitchen (Server).The kitchen prepares the food and gives it to the waiter (Response).The waiter delivers the food to you (Data returned from API).
How Terraform Uses API
Each cloud provider has its own API. Normally, we would need to write complex code in Python, Java, or other languages to interact with these APIs.
But Terraform simplifies this!
Terraform provides pre-built modules that simplify the process of interacting with various cloud resources.
Instead of making API calls manually, we define our desired infrastructure in a Terraform script.
Terraform then interacts with the cloud provider’s API and provisions the resources automatically.
Example: Creating an EC2 Instance on AWS using Terraform
Define Infrastructure – Write a Terraform configuration file specifying the EC2 instance details.
Terraform Processes the Configuration – It interprets the file and prepares the necessary API requests for AWS.
Provisioning the Instance – Terraform sends requests to AWS, creating the EC2 instance and returning the details.
This is code for creating a EC2 instances.
Conclusion :
Terraform is a powerful game-changer for cloud automation. It eliminates the need for multiple tools across cloud providers, streamlining infrastructure management and making it more efficient.