GKE - Create and Import Cluster
Create GKE cluster using Kosmos
Prerequisites
Required tools:
gcloud components install gke-gcloud-auth-pluginAdd to your shell profile (
~/.bashrc,~/.zshrc, or equivalent):export USE_GKE_GCLOUD_AUTH_PLUGIN=True # Homebrew (macOS): source "$(brew --prefix)/share/google-cloud-sdk/path.zsh.inc" # Linux/manual: source "$HOME/google-cloud-sdk/path.bash.inc"
Required access:
- Kosmos Access Key – Authenticate with Kosmos console
- GCP IAM Account with permissions to create Workload Identity pools, service accounts, and GKE clusters
Environment variables:
Set these before starting:
export PROJECT_ID="your-gcp-project"
export PROJECT_NUMBER=$(gcloud projects describe $PROJECT_ID --format="value(projectNumber)")
export FLEET_ID="your-fleet-id"
export CLUSTER_NAME="your-cluster-name"
export REGION="us-west1"
export WI_POOL_ID="kosmos-wi-pool"
export WI_PROVIDER_ID="kosmos-oidc-provider"
export SA_NAME="kosmos-gke-sa"
export SA_EMAIL="${SA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com"
export VPC_NAME="your-vpc"
export SUBNET_NAME="your-subnet"
export USER="your-kosmos-username" # Your Kosmos console username for adminUsers
Network configuration reference
GKE clusters require several network components. Understanding these prerequisites helps avoid common configuration issues.
Required GCP resources
Before creating a cluster, these resources must exist (or be created in step 6):
| Resource | Description | Create vs Import |
|---|---|---|
| VPC Network | Virtual network for cluster nodes | Create: can auto-create or use existing. Import: must pre-exist |
| Subnet | IP range for node primary IPs | Create: can auto-create or use existing. Import: must pre-exist |
IP address ranges
GKE uses three distinct IP ranges. Plan these carefully as Service CIDR cannot be changed after cluster creation.
| Field | Purpose | Default if empty | Notes |
|---|---|---|---|
clusterIpv4Cidr | Pod IP addresses | Auto: /14 from 10.0.0.0/8 (~262k IPs) | Can conflict with existing routes if auto-allocated |
| Service CIDR | ClusterIP services | GKE-managed: 34.118.224.0/20 | Immutable after creation |
masterIpv4CidrBlock | Control plane IPs | Required only if enablePrivateNodes: true | Must be /28, cannot overlap node/pod ranges |
Node subnet sizing
| Cluster size | Recommended subnet | Example CIDR |
|---|---|---|
| ≤250 nodes | /24 (256 IPs) | 10.0.0.0/24 |
| ≤1000 nodes | /22 (1024 IPs) | 10.0.0.0/22 |
Common CIDR conflicts
Avoid these overlaps when planning IP ranges:
- Pod CIDR must not overlap VPC subnets, VPN routes, or peered VPCs
- Service CIDR must not overlap Pod CIDR or node subnets
- Master CIDR (private clusters) must not overlap any other range
Tip: Use
gcloud compute networks subnets listandgcloud compute routes listto check existing allocations before creating a cluster.
Kubernetes version selection
Check available versions in your region:
# List supported control plane versions
gcloud container get-server-config --region=${REGION} \
--format="yaml(validMasterVersions)"
# Get the latest supported version
gcloud container get-server-config --region=${REGION} \
--format="value(validMasterVersions[0])"
Warning: Hardcoded versions become unsupported. The admission webhook will reject with:
k8s version 'X.X.X-gke.XXX' is not supported for master
0. Enable required GCP APIs
gcloud config set project $PROJECT_ID
gcloud services enable \
compute.googleapis.com \
container.googleapis.com \
iam.googleapis.com \
iamcredentials.googleapis.com \
cloudresourcemanager.googleapis.com
1. Create workload identity pools
1-A. Create workload-identity-pools
Sample pool name: [user-test-pool]
gcloud iam workload-identity-pools create [user-test-pool] --display-name="user-test-pool" --location="global"
Output:
Created workload identity pool [user-test-pool].
1-B. Describe workload-identity-pools
gcloud iam workload-identity-pools describe [user-test-pool] --location="global"
Output:
displayName: user-test-pool
name: projects/[AccountID]/locations/global/workloadIdentityPools/user-test-pool
state: ACTIVE
2. Create OIDC provider
2-A. Create OIDC provider
provider : [user-test-pool-provider]<– sample name for the provider Use 1-A [user-test-pool] for “workload-identity-pool"
gcloud iam workload-identity-pools providers create-oidc [user-test-pool-provider] \
--location="global" \
--workload-identity-pool="user-test-pool" \
--attribute-mapping="google.subject=assertion.sub" \
--issuer-uri="https://console.kosmos.spcplatform.com/kosmos-oidc" \
--allowed-audiences="kosmos-operator" \
--display-name="user-test-provider"
Output:
Created workload identity pool provider [user-test-pool-provider].
2-B. List workload-identity-pools providers
gcloud iam workload-identity-pools providers list \
--location=global \
--workload-identity-pool=user-test-pool
Output:
attributeMapping:
google.subject: assertion.sub
displayName: user-test-provider
name: projects/[AccountID]/locations/global/workloadIdentityPools/[user-test-pool]/providers/[user-test-pool-provider]
oidc:
allowedAudiences:
- kosmos-operator
issuerUri: https://console.kosmos.spcplatform.com/kosmos-oidc
state: ACTIVE
3. Create service account for external workload
3-A. Create service account
Sample name for service account
service-account : [user-test-service-name]
gcloud iam service-accounts create [user-test-service-name] --display-name="[user-test-service-name]"
3-B. Describe service account
gcloud iam service-accounts list --filter=[user-test-service-name]
Output:
DISPLAY NAME EMAIL DISABLED
user-test-service-name user-test-service-name@[PROJECT_ID].iam.gserviceaccount.com False
4. Allow external identities to impersonate a service account
4-A. Get current project number
gcloud projects describe $(gcloud config get-value core/project) --format="value(projectNumber)"
4-B. Grant workload identity user role
Grant roles to external identities that meet a certain criteria.
Use 3-B Email for ${SERVICE_ACCOUNT_EMAIL}.
Use 4-A project number for ${PROJECT_NUMBER}.
Use 1-A name for ${POOL_ID}.
$ gcloud iam service-accounts add-iam-policy-binding ${SERVICE_ACCOUNT_EMAIL} \
--role=roles/iam.workloadIdentityUser \
--member="principal://iam.googleapis.com/projects/${PROJECT_NUMBER}/locations/global/workloadIdentityPools/${POOL_ID}/subject/fleet-${FLEET_ID}"
Output:
Updated IAM policy for serviceAccount [SERVICE_ACCOUNT_EMAIL].
bindings:
- members:
- principal://iam.googleapis.com/projects/[PROJECT_NUMBER]/locations/global/workloadIdentityPools/[user-test-pool]/subject/fleet-${FLEET_ID}
role: roles/iam.workloadIdentityUser
etag: BwYVoHWX4BQ=
version: 1
5. Add IAM policy with service accounts
5-A. Add IAM policy binding with roles
Use following roles for binding.
Roles:
- Compute Viewer:
roles/compute.viewer - Project Viewer:
roles/viewer - Kubernetes Engine Admin:
roles/container.admin - Service Account User:
roles/iam.serviceAccountUser
Use 3-B email for [SERVICE_ACCOUNT_EMAIL].
$ gcloud projects add-iam-policy-binding [PROJECT_ID] \
--member="serviceAccount:[SERVICE_ACCOUNT_EMAIL]" \
--role='roles/compute.viewer'
$ gcloud projects add-iam-policy-binding [PROJECT_ID] \
--member="serviceAccount:[SERVICE_ACCOUNT_EMAIL]" \
--role='roles/viewer'
$ gcloud projects add-iam-policy-binding [PROJECT_ID] \
--member="serviceAccount:[SERVICE_ACCOUNT_EMAIL]" \
--role='roles/container.admin'
$ gcloud projects add-iam-policy-binding [PROJECT_ID] \
--member="serviceAccount:[SERVICE_ACCOUNT_EMAIL]" \
--role='roles/iam.serviceAccountUser'
$ gcloud projects add-iam-policy-binding [PROJECT_ID] \
--member="serviceAccount:[SERVICE_ACCOUNT_EMAIL]" \
--role='roles/iam.workloadIdentityUser'
5-B. Describe project IAM policy
gcloud projects get-iam-policy [PROJECT_ID]
Output:
...
- members:
- serviceAccount:[SERVICE_ACCOUNT_EMAIL]
role: roles/compute.viewer
- members:
- serviceAccount:[SERVICE_ACCOUNT_EMAIL]
role: roles/viewer
- members:
- serviceAccount:[SERVICE_ACCOUNT_EMAIL]
role: roles/container.admin
- members:
- serviceAccount:[SERVICE_ACCOUNT_EMAIL]
role: roles/iam.serviceAccountUser
- members:
- serviceAccount:[SERVICE_ACCOUNT_EMAIL]
role: roles/iam.workloadIdentityUser
6. Create VPC Network
6-A. Create Simple VPC Network
gcloud compute networks create ${VPC_NAME} --subnet-mode=custom --bgp-routing-mode=regional --mtu=1460
6-B. Create Subnet
gcloud compute networks subnets create ${SUBNET_NAME} --network=${VPC_NAME} --range=${CIDR_RANGE} --region=${REGION}
GKE - Create cluster
Create a sample file
- Use
4-B${FLEET_ID}asmetadata > name. - Use target cluster’s name as
${CLUSTER_NAME}. - Use target cluster’s region as
${REGION}. - Use 3-B Email for
${SERVICE_ACCOUNT_EMAIL}. - Use 4-A project number for
${PROJECT_NUMBER}. - Use 1-B name for
${POOL_ID}. - Use 2-A [user-test-pool-provider] for
${USER_TEST_POOL_PROVIDER}. - Use 6-A name for
${VPC_NAME}. - Use 6-B name for
${SUBNET_NAME}.
FileName: gkecluster_sample.yaml
apiVersion: storage.kosmos.spcplatform.com/v1
kind: GKECluster
metadata:
labels:
app.kubernetes.io/name: ${CLUSTER_NAME}
app.kubernetes.io/instance: ${FLEET_ID}
app.kubernetes.io/part-of: kosmos
app.kubernetes.io/managed-by: kustomize
app.kubernetes.io/created-by: kosmos
name: ${CLUSTER_NAME}
namespace: ${FLEET_ID}
spec:
description: "GKE cluster created via Kosmos CLI"
name: "${CLUSTER_NAME}"
authorization:
adminUsers:
- ${USER}
gkeConfig:
clusterName: "${CLUSTER_NAME}"
description: "Kosmos managed GKE cluster"
labels: {}
region: "${REGION}"
projectID: "${PROJECT_ID}"
projectNumber: "${PROJECT_NUMBER}"
kubernetesVersion: "" # Empty = latest stable version
loggingService: ""
monitoringService: ""
enableKubernetesAlpha: false
clusterIpv4Cidr: "" # Auto-allocated if empty
ipAllocationPolicy:
useIpAliases: true
nodePools:
- name: ${CLUSTER_NAME}-nodes
autoscaling:
enabled: false
config:
machineType: "e2-medium"
diskSizeGb: 50
diskType: "pd-standard" # https://docs.cloud.google.com/compute/docs/disks/persistent-disks#disk-types
initialNodeCount: 1
maxPodsConstraint: 110
version: ""
management:
autoRepair: true
autoUpgrade: true
clusterAddons:
httpLoadBalancing: true
networkPolicyConfig: false
horizontalPodAutoscaling: true
networkPolicyEnabled: false
network: ${VPC_NAME}
subnetwork: ${SUBNET_NAME}
privateClusterConfig:
enablePrivateEndpoint: false
enablePrivateNodes: false
# Control plane CIDR - only used if enablePrivateNodes is true
masterIpv4CidrBlock: "172.16.0.0/28"
masterAuthorizedNetworks:
cidrBlocks:
# displayName is optional - human-readable label shown in GCP Console
- cidrBlock: "YOUR_IP/32" # Replace with your public IP
displayName: "my-ip"
# Kosmos platform IPs (required for management)
- cidrBlock: "3.208.49.242/32"
displayName: "kosmos-1"
- cidrBlock: "35.172.72.171/32"
displayName: "kosmos-2"
- cidrBlock: "34.214.188.139/32"
displayName: "kosmos-3"
- cidrBlock: "35.165.241.214/32"
displayName: "kosmos-4"
- cidrBlock: "13.209.31.187/32"
displayName: "kosmos-5"
- cidrBlock: "15.165.61.114/32"
displayName: "kosmos-6"
enabled: true
locations: [] # Required field - empty for regional clusters
maintenanceWindow: ""
imported: false
zone: ""
serviceAccount: ${SA_EMAIL}
workloadIdentityPoolId: ${WI_POOL_ID}
workloadIdentityProviderId: ${WI_PROVIDER_ID}
Login to Kosmos
kosmos login console.kosmos.spcplatform.com --access-key <access-key-of-cluster-owner>
Create GkeCluster
Important: Always specify the
--fleetflag explicitly to avoid using defaults from~/.kosmos/defaults.json.
kosmos create gke --file <YAML-file> --fleet ${FLEET_ID}
The GKE cluster and node are created and have status “Ready” on the GCP side. But on the Kosmos side, the cluster has status “connecting”. Execute the following command to see your cluster in Kosmos:
kosmos list gke --fleet <FLEET_ID>
Verify that you can see the new GKE cluster in GCP
gcloud container clusters list
Update the kube config file
gcloud container clusters get-credentials <cluster-name> --zone <zone>
Verify that kubectl works
kubectl get ns
Connect GkeCluster
Update the config and describe the Gkecluster and get the cluster-name in Annotations
The ${CLUSTER-NAME} is [CLUSTER_NAME]-[HASH_NUMBER]
[target-cluster-context] format is: gke_${PROJECT_NAME}${REGION}${CLUSTER_NAME}
Describe the GkeCluster:
kubectl describe gkeclusters.storage.kosmos.spcplatform.com ${CLUSTER_NAME} -n ${FLEET_ID}Extract
<CLUSTER_NAME>from annotations.Connect to the cluster:
gcloud container clusters get-credentials ${CLUSTER_NAME} --region=${REGION} kubectl config use-context [target-cluster-context]Connect the cluster with the ${CLUSTER_NAME} by the Kosmos.
kosmos join cluster <cluster-name> --fleet <FLEET_ID>After the
kosmos join clustercommand installs the platform agent successfully, the status for the cluster is changed from “connecting” to “ready” on the Kosmos side.
Open a new Terminal Window and verify cluster usage
kosmos login console.kosmos.spcplatform.com --access-key <access-key-of-cluster-owner>
kosmos use cluster <cluster-name> --fleet ${FLEET_ID}
kubectl get ns
GKE - Import cluster
Import an existing GKE cluster into Kosmos for management.
Note: When importing, the GKE cluster must already exist. Kosmos will not create the cluster, only register it for management.
Prerequisites for import
Complete steps 0-5 from the Create workflow above to set up:
- Workload Identity Pool and OIDC Provider
- Service Account with required IAM roles
Get existing cluster details
Retrieve the cluster configuration to populate the import YAML:
gcloud container clusters describe ${CLUSTER_NAME} --region=${REGION} \
--format="yaml(name,currentMasterVersion,network,subnetwork,clusterIpv4Cidr)"
Important: Copy the
currentMasterVersionvalue for thekubernetesVersionfield in your import YAML.
Update the kube config file
gcloud container clusters get-credentials ${CLUSTER_NAME} --region=${REGION}
Verify that kubectl works
kubectl get ns
Create import YAML file
Important: For imported clusters, you must set
imported: trueand match the existing cluster’s configuration (version, network, CIDR).
apiVersion: storage.kosmos.spcplatform.com/v1
kind: GKECluster
metadata:
labels:
app.kubernetes.io/name: ${CLUSTER_NAME}
app.kubernetes.io/instance: ${FLEET_ID}
app.kubernetes.io/part-of: kosmos
app.kubernetes.io/managed-by: kustomize
app.kubernetes.io/created-by: kosmos
name: ${CLUSTER_NAME}
namespace: ${FLEET_ID}
spec:
description: "Existing GKE cluster imported via Kosmos CLI"
name: "${CLUSTER_NAME}"
authorization:
adminUsers:
- ${USER}
gkeConfig:
# Cluster identity - must match existing cluster
clusterName: "${CLUSTER_NAME}"
projectID: "${PROJECT_ID}"
projectNumber: "${PROJECT_NUMBER}"
region: "${REGION}"
# IMPORTANT: imported=true tells Kosmos this cluster already exists
imported: true
# REQUIRED: Must match existing cluster version
# Get from: gcloud container clusters describe ${CLUSTER_NAME} --region=${REGION} --format="value(currentMasterVersion)"
kubernetesVersion: ""
# Network settings - must match existing cluster
network: ${VPC_NAME}
subnetwork: ${SUBNET_NAME}
clusterIpv4Cidr: "" # Get from gcloud describe output
# Workload Identity for Kosmos authentication
serviceAccount: ${SA_EMAIL}
workloadIdentityPoolId: ${WI_POOL_ID}
workloadIdentityProviderId: ${WI_PROVIDER_ID}
# Required fields (even if empty for regional clusters)
locations: []
labels: {}
zone: ""
Login to Kosmos
kosmos login console.kosmos.spcplatform.com --access-key <access-key-of-cluster-owner>
Execute CLI to import cluster
Important: Always specify the
--fleetflag explicitly.
kosmos create gke --file <YAML-file> --fleet ${FLEET_ID}
The GKE cluster is imported to Kosmos but has a status of “connecting”. Execute the following command to see your cluster in Kosmos:
kosmos list gke --fleet ${FLEET_ID}
Connect to the cluster
kosmos join cluster ${CLUSTER_NAME} --fleet ${FLEET_ID}
After the kosmos join cluster command installs the platform agent successfully, the status for the cluster is changed to “ready” on the Kosmos side.
Verify cluster usage
kosmos use cluster ${CLUSTER_NAME} --fleet ${FLEET_ID}
kubectl get ns
Key differences: Import vs Create
| Aspect | Create (imported: false) | Import (imported: true) |
|---|---|---|
| GKE Cluster | Kosmos creates it | Must pre-exist |
| Kubernetes Version | Can be empty (latest) | Must match existing |
| Network settings | Configured by Kosmos | Must match existing |
| Deletion behavior | Kosmos deletes GKE cluster | Kosmos only removes from management |
Troubleshooting
gke-gcloud-auth-plugin not found
Error:
executable gke-gcloud-auth-plugin not found
Solution: Install the plugin and add to PATH:
gcloud components install gke-gcloud-auth-plugin
# Add to ~/.bashrc or ~/.zshrc:
export USE_GKE_GCLOUD_AUTH_PLUGIN=True
export PATH="$(gcloud info --format='value(installation.sdk_root)')/bin:$PATH"
Fleet namespace mismatch
Error:
User "username" cannot create resource "gkeclusters" in namespace "wrong-fleet"
Solution: The Kosmos CLI may use a cached fleet from ~/.kosmos/defaults.json. Always specify the --fleet flag explicitly:
kosmos create gke --file cluster.yaml --fleet ${FLEET_ID}
Import validation error: locations field
Error:
spec.gkeConfig.locations: Invalid value: "null": spec.gkeConfig.locations in body must be of type array
Solution: Add locations: [] to your import YAML, even for regional clusters:
gkeConfig:
locations: []
Cleanup
Delete cluster from Kosmos
kosmos delete gke --name ${CLUSTER_NAME} --fleet ${FLEET_ID}
Note: For clusters created via Kosmos (
imported: false), this also deletes the GKE cluster in GCP. For imported clusters (imported: true), only the Kosmos registration is removed.
Wait for GKE cluster deletion (created clusters only)
# Monitor deletion (~3-5 minutes)
gcloud container clusters list --region ${REGION}
Delete VPC resources
# Delete subnet first
gcloud compute networks subnets delete ${SUBNET_NAME} --region=${REGION} --quiet
# Delete VPC
gcloud compute networks delete ${VPC_NAME} --quiet
Remove IAM bindings and service account
# Remove project IAM bindings
for role in roles/compute.viewer roles/viewer roles/container.admin roles/iam.serviceAccountUser; do
gcloud projects remove-iam-policy-binding ${PROJECT_ID} \
--member="serviceAccount:${SA_EMAIL}" \
--role="${role}" --quiet
done
# Delete service account
gcloud iam service-accounts delete ${SA_EMAIL} --quiet
Delete Workload Identity resources
# Delete OIDC Provider first
gcloud iam workload-identity-pools providers delete ${WI_PROVIDER_ID} \
--location="global" \
--workload-identity-pool=${WI_POOL_ID} \
--quiet
# Delete Workload Identity Pool
gcloud iam workload-identity-pools delete ${WI_POOL_ID} \
--location="global" \
--quiet
Verify cleanup
gcloud container clusters list --region ${REGION}
gcloud compute networks list --filter="name=${VPC_NAME}"
gcloud iam service-accounts list --filter="email:${SA_NAME}"
gcloud iam workload-identity-pools list --location=global
kosmos list gke --fleet ${FLEET_ID}