Setting Up a Kubernetes Cluster on GKE Using Terraform

This guide describes how to configure and deploy a Terraform module to set up a GCP network and an autopilot private GKE cluster with a structured and reusable design.

Repository

Terraform GKE Cluster Repository

git clone git@github.com:thomasscothamilton/terraform-google-kubernetes-engine.git

cd terraform-google-kubernetes-engine

Prerequisites

  1. Install Terraform:
# Download and install Terraform
brew tap hashicorp/tap
brew install hashicorp/tap/terraform

# Verify installation
terraform -v
  1. Install and Configure Google Cloud SDK:
# Download and install Google Cloud SDK
curl https://sdk.cloud.google.com | bash

# Restart your shell
exec -l $SHELL

# Initialize the SDK
gcloud init

# Verify installation
gcloud version
  1. Service Account: Create a service account with the necessary permissions for GKE, Networking, and Storage APIs.
# Set your project ID
PROJECT_ID="your-gcp-project-id"

# Create the service account
gcloud iam service-accounts create terraform-sa --display-name "Terraform Service Account"

# Assign roles to the service account
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member "serviceAccount:terraform-sa@$PROJECT_ID.iam.gserviceaccount.com" \
  --role "roles/container.admin"
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member "serviceAccount:terraform-sa@$PROJECT_ID.iam.gserviceaccount.com" \
  --role "roles/compute.admin"
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member "serviceAccount:terraform-sa@$PROJECT_ID.iam.gserviceaccount.com" \
  --role "roles/storage.admin"

# Create a key for the service account
gcloud iam service-accounts keys create ~/terraform-key.json \
  --iam-account terraform-sa@$PROJECT_ID.iam.gserviceaccount.com
  1. Enable APIs: Ensure the following APIs are enabled:
 
# Enable Kubernetes Engine API
gcloud services enable container.googleapis.com

# Enable Compute Engine API
gcloud services enable compute.googleapis.com

# Enable Cloud Storage API
gcloud services enable storage.googleapis.com
  1. Credentials: Export Google Cloud credentials:
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/google-application-credentials.json"

Module Structure

Files

  1. backend.tf: Configures the backend to store Terraform state in a GCS bucket.

  2. versions.tf: Defines Terraform and provider version constraints.

  3. provider.tf: Configures the Google Cloud provider.

  4. variables.tf: Defines input variables for the module.

  5. main.tf: Deploys the GKE cluster using the network module.

  6. network.tf: Sets up the VPC and subnet configurations.

  7. outputs.tf: Exports outputs like cluster endpoint, CA certificate, and network details.

Step 1: Initialize Terraform

  1. Clone the repository containing the Terraform module.

  2. Navigate to the module directory:

cd environments/dev
  1. Initialize Terraform:
export TF_VAR_project_id="my-gcp-project"
export TF_VAR_region="us-central1"
export TF_VAR_network="my-vpc-network"
export TF_VAR_subnetwork="my-subnet"

# Copy this command:
terraform init

Step 2: Define Input Variables

Create a terraform.tfvars file to provide values for the required variables (project_id and region).

Example terraform.tfvars

project_id = "your-gcp-project-id"
region     = "your-region"

Step 3: Plan the Deployment

Generate a plan to review the changes Terraform will apply:

terraform plan -out main.tfplan

Step 4: Apply the Plan

Apply the configuration to deploy the resources:

terraform apply main.tfplan

Step 5: Review Outputs

After deployment, Terraform will display outputs defined in outputs.tf.

Example Outputs * Cluster Endpoint: Used to connect to the Kubernetes cluster. * CA Certificate: Used to authenticate with the cluster. * Network Name: The name of the created VPC.

Key Features

GCP Network Configuration (network.tf)

VPC Creation:

  • Creates a custom VPC network.
  • Configures subnets with secondary IP ranges for pods and services.

Subnets:

  • A primary subnet for cluster nodes.
  • A master subnet for GKE private endpoint authentication.

Autopilot Private GKE Cluster (main.tf)

  • Creates a GKE cluster with the following features:
    • Autopilot mode.
    • Private nodes and private endpoints.
    • Regional clusters for high availability.
    • IP ranges for pods and services.